2024-06-06T03:15:58.276 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-06T03:15:58.276 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023 --name teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi --owner scheduled_teuthology@teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} -- /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023/orig.config.yaml 2024-06-06T03:15:58.333 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-06T03:15:58.418 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7743023' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 3d746ca5346f34eef7e826a1eb71e59e02b682ae owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5050 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 416/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 3d746ca5346f34eef7e826a1eb71e59e02b682ae syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBNQGGpFOTZ+QnxuIsVpNUm/Lo9GfrNstzyKFqA9ci2Bwp1892HniC6V2NT+k+/60H8BiUm/tT8MPBXkfJ6QLhk= smithi104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGRwYhkR7s0Y/2MrYcjndoUTJZKh48+wZ279Gq6gnwT89ovPTQUX/r5xEWgXCS+gjkydNdrUQnzOzq/9KEw7Ncg= smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCAJVF4hU9+jObu3b+0WPd54yiRdLziZaxqwnz9SH7d/nfNG6+9vDjBVtPdhHKbje7VJ36cbYHHsMu/6BShm1UQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - kernel_untar_build.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-05_21:24:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-06T03:15:58.418 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa; will attempt to use it 2024-06-06T03:15:58.419 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks 2024-06-06T03:15:58.419 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-06T03:15:58.421 INFO:teuthology.task.internal:Checking packages... 2024-06-06T03:15:58.441 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-06T03:15:58.441 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-06T03:15:58.441 INFO:teuthology.packaging:ref: None 2024-06-06T03:15:58.441 INFO:teuthology.packaging:tag: None 2024-06-06T03:15:58.441 INFO:teuthology.packaging:branch: squid 2024-06-06T03:15:58.441 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:15:58.441 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-06T03:15:58.656 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2960.g3d746ca5 2024-06-06T03:15:58.657 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-06T03:15:58.664 INFO:teuthology.task.internal:no buildpackages task found 2024-06-06T03:15:58.664 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-06T03:15:58.674 INFO:teuthology.task.internal:Saving configuration 2024-06-06T03:15:58.693 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-06T03:15:58.699 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-06T03:15:58.725 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 03:11:13.248827', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBNQGGpFOTZ+QnxuIsVpNUm/Lo9GfrNstzyKFqA9ci2Bwp1892HniC6V2NT+k+/60H8BiUm/tT8MPBXkfJ6QLhk='} 2024-06-06T03:15:58.750 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 03:11:13.250806', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGRwYhkR7s0Y/2MrYcjndoUTJZKh48+wZ279Gq6gnwT89ovPTQUX/r5xEWgXCS+gjkydNdrUQnzOzq/9KEw7Ncg='} 2024-06-06T03:15:58.772 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 03:11:13.252581', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCAJVF4hU9+jObu3b+0WPd54yiRdLziZaxqwnz9SH7d/nfNG6+9vDjBVtPdhHKbje7VJ36cbYHHsMu/6BShm1UQ='} 2024-06-06T03:15:58.772 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-06T03:15:58.777 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-06T03:15:58.777 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-06T03:15:58.778 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-06T03:15:58.778 INFO:teuthology.run_tasks:Running task console_log... 2024-06-06T03:15:58.883 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f19f02d3be0>, signals=[15]) 2024-06-06T03:15:58.884 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-06T03:15:58.890 INFO:teuthology.task.internal:Opening connections... 2024-06-06T03:15:58.890 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2024-06-06T03:15:58.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:15:58.967 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2024-06-06T03:15:58.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:15:59.043 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2024-06-06T03:15:59.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:15:59.118 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-06T03:15:59.125 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2024-06-06T03:15:59.144 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2024-06-06T03:15:59.144 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2024-06-06T03:15:59.202 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2024-06-06T03:15:59.202 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2024-06-06T03:15:59.202 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2024-06-06T03:15:59.202 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2024-06-06T03:15:59.202 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2024-06-06T03:15:59.202 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2024-06-06T03:15:59.202 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T03:15:59.202 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2024-06-06T03:15:59.202 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2024-06-06T03:15:59.202 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T03:15:59.202 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2024-06-06T03:15:59.203 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T03:15:59.203 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T03:15:59.203 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T03:15:59.203 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2024-06-06T03:15:59.228 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-06-06T03:15:59.246 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-06-06T03:15:59.246 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-06-06T03:15:59.303 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-06-06T03:15:59.303 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-06-06T03:15:59.304 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-06-06T03:15:59.304 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-06-06T03:15:59.304 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-06-06T03:15:59.304 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-06-06T03:15:59.304 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T03:15:59.304 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-06-06T03:15:59.304 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-06-06T03:15:59.304 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T03:15:59.304 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-06-06T03:15:59.305 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T03:15:59.305 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T03:15:59.305 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T03:15:59.305 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-06-06T03:15:59.337 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-06-06T03:15:59.354 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-06-06T03:15:59.355 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-06-06T03:15:59.413 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2024-06-06T03:15:59.413 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="9" 2024-06-06T03:15:59.413 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2024-06-06T03:15:59.413 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2024-06-06T03:15:59.413 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="9" 2024-06-06T03:15:59.413 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el9" 2024-06-06T03:15:59.414 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T03:15:59.414 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-06-06T03:15:59.414 INFO:teuthology.orchestra.run.smithi131.stdout:LOGO="fedora-logo-icon" 2024-06-06T03:15:59.414 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T03:15:59.414 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2024-06-06T03:15:59.415 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T03:15:59.415 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T03:15:59.415 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T03:15:59.415 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-06-06T03:15:59.453 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-06T03:15:59.466 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-06T03:15:59.497 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-06T03:15:59.497 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2024-06-06T03:15:59.504 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2024-06-06T03:15:59.508 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2024-06-06T03:15:59.528 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-06T03:15:59.539 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-06T03:15:59.540 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2024-06-06T03:15:59.562 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2024-06-06T03:15:59.565 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2024-06-06T03:15:59.817 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-06T03:15:59.903 INFO:teuthology.run_tasks:Running task kernel... 2024-06-06T03:15:59.919 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-06-06T03:15:59.920 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-06-06T03:15:59.920 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-06-06T03:15:59.920 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T03:15:59.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:15:59.938 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-06-06T03:15:59.996 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-437.el9.x86_64 2024-06-06T03:15:59.996 INFO:teuthology.task.kernel:Running kernel on smithi040: 5.14.0-437.el9.x86_64 2024-06-06T03:15:59.996 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum install -y kernel 2024-06-06T03:16:02.301 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-06-06T03:16:04.103 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-06-06T03:16:08.345 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-06-06T03:16:09.995 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 17 kB 00:00 2024-06-06T03:16:11.348 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-06-06T03:16:16.762 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-06-06T03:16:18.960 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-06T03:16:18.960 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-06T03:16:19.060 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-06-06T03:16:19.070 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-06-06T03:16:19.070 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-06-06T03:16:19.070 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-06-06T03:16:19.070 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-06-06T03:16:19.070 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-06T03:16:19.070 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-06-06T03:16:19.070 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-06T03:16:19.070 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-06T03:16:19.071 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-06T03:16:19.071 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:16:19.071 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-06-06T03:16:19.071 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-06-06T03:16:19.071 INFO:teuthology.orchestra.run.smithi040.stdout:Install 4 Packages 2024-06-06T03:16:19.071 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:16:19.071 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 80 M 2024-06-06T03:16:19.072 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 124 M 2024-06-06T03:16:19.072 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-06-06T03:16:19.300 INFO:teuthology.orchestra.run.smithi040.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:16:19.303 INFO:teuthology.orchestra.run.smithi040.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:16:19.308 INFO:teuthology.orchestra.run.smithi040.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:16:19.589 INFO:teuthology.orchestra.run.smithi040.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 2.3 MB/s | 893 kB 00:00 2024-06-06T03:16:19.772 INFO:teuthology.orchestra.run.smithi040.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-06-06T03:16:20.347 INFO:teuthology.orchestra.run.smithi040.stdout:(3/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 31 MB/s | 35 MB 00:01 2024-06-06T03:16:21.097 INFO:teuthology.orchestra.run.smithi040.stdout:(4/4): kernel-modules-core-5.14.0-452.el9.x86_6 19 MB/s | 29 MB 00:01 2024-06-06T03:16:21.100 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-06-06T03:16:21.100 INFO:teuthology.orchestra.run.smithi040.stdout:Total 40 MB/s | 80 MB 00:02 2024-06-06T03:16:21.830 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-06-06T03:16:22.237 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-06-06T03:16:22.237 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-06-06T03:16:23.220 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-06-06T03:16:23.220 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-06-06T03:16:25.804 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-06-06T03:16:26.446 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-06T03:16:26.540 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T03:16:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T03:16:29.494 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T03:16:34.586 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T03:16:34.633 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:16:39.746 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:17:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:17:10.913 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:17:11.220 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:17:11.221 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-06T03:17:11.221 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T03:17:11.221 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T03:17:12.033 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:17:12.033 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:17:12.033 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-06-06T03:17:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-06T03:17:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-06T03:17:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-06T03:17:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-06T03:17:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:17:12.034 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-06-06T03:17:12.395 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T03:17:12.470 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T03:17:12.471 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T03:17:12.471 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-06T03:17:12.471 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-06T03:17:12.471 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-06T03:17:12.471 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-06-06T03:17:12.471 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi040.front.sepia.ceph.com, path=None, version=distro) 2024-06-06T03:17:12.471 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T03:17:12.525 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T03:17:12.526 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T03:17:12.526 DEBUG:teuthology.orchestra.run.smithi040:> sudo rpm -qi grub2-tools 2024-06-06T03:17:12.557 INFO:teuthology.orchestra.run.smithi040.stdout:Name : grub2-tools 2024-06-06T03:17:12.557 INFO:teuthology.orchestra.run.smithi040.stdout:Epoch : 1 2024-06-06T03:17:12.557 INFO:teuthology.orchestra.run.smithi040.stdout:Version : 2.06 2024-06-06T03:17:12.557 INFO:teuthology.orchestra.run.smithi040.stdout:Release : 68.el9 2024-06-06T03:17:12.557 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: x86_64 2024-06-06T03:17:12.557 INFO:teuthology.orchestra.run.smithi040.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-06T03:17:12.557 INFO:teuthology.orchestra.run.smithi040.stdout:Group : Unspecified 2024-06-06T03:17:12.557 INFO:teuthology.orchestra.run.smithi040.stdout:Size : 8264254 2024-06-06T03:17:12.557 INFO:teuthology.orchestra.run.smithi040.stdout:License : GPLv3+ 2024-06-06T03:17:12.557 INFO:teuthology.orchestra.run.smithi040.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-06T03:17:12.557 INFO:teuthology.orchestra.run.smithi040.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-06T03:17:12.558 INFO:teuthology.orchestra.run.smithi040.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-06T03:17:12.558 INFO:teuthology.orchestra.run.smithi040.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-06T03:17:12.558 INFO:teuthology.orchestra.run.smithi040.stdout:Packager : builder@centos.org 2024-06-06T03:17:12.558 INFO:teuthology.orchestra.run.smithi040.stdout:Vendor : CentOS 2024-06-06T03:17:12.558 INFO:teuthology.orchestra.run.smithi040.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-06T03:17:12.558 INFO:teuthology.orchestra.run.smithi040.stdout:Summary : Support tools for GRUB. 2024-06-06T03:17:12.558 INFO:teuthology.orchestra.run.smithi040.stdout:Description : 2024-06-06T03:17:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:17:12.558 INFO:teuthology.orchestra.run.smithi040.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-06T03:17:12.558 INFO:teuthology.orchestra.run.smithi040.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-06T03:17:12.558 INFO:teuthology.orchestra.run.smithi040.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-06T03:17:12.558 INFO:teuthology.orchestra.run.smithi040.stdout:hardware devices. 2024-06-06T03:17:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:17:12.558 INFO:teuthology.orchestra.run.smithi040.stdout:This subpackage provides tools for support of all platforms. 2024-06-06T03:17:12.559 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-06T03:17:12.559 INFO:teuthology.task.kernel:Updating grub on smithi040 to boot 5.14.0-452.el9.x86_64 2024-06-06T03:17:12.559 DEBUG:teuthology.orchestra.run.smithi040:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-06T03:17:13.168 INFO:teuthology.orchestra.run.smithi040.stderr:Generating grub configuration file ... 2024-06-06T03:17:14.655 INFO:teuthology.orchestra.run.smithi040.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-06T03:17:14.672 INFO:teuthology.orchestra.run.smithi040.stderr:done 2024-06-06T03:17:14.673 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-06-06T03:17:14.688 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.EqG4DckXfc 2024-06-06T03:17:14.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EqG4DckXfc 2024-06-06T03:17:14.754 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.EqG4DckXfc 2024-06-06T03:17:14.886 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.EqG4DckXfc is 6KB 2024-06-06T03:17:14.935 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.EqG4DckXfc 2024-06-06T03:17:14.950 DEBUG:teuthology.orchestra.run.smithi040:> sudo /bin/ls /boot/loader/entries || true 2024-06-06T03:17:15.015 INFO:teuthology.orchestra.run.smithi040.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-06T03:17:15.015 INFO:teuthology.orchestra.run.smithi040.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-06T03:17:15.015 INFO:teuthology.orchestra.run.smithi040.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-06T03:17:15.015 INFO:teuthology.orchestra.run.smithi040.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-06T03:17:15.017 DEBUG:teuthology.orchestra.run.smithi040:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-06T03:17:15.168 DEBUG:teuthology.orchestra.run.smithi040:> sudo shutdown -r now 2024-06-06T03:17:45.171 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-06-06T03:17:45.171 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-06-06T03:17:45.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:18:03.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.40 2024-06-06T03:18:12.552 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-06-06T03:18:12.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:19:12.555 DEBUG:teuthology.orchestra.remote:timed out 2024-06-06T03:19:24.556 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-06-06T03:19:24.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:19:24.857 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-06-06T03:19:25.226 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-06-06T03:19:25.226 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-452.el9.x86_64"... 2024-06-06T03:19:25.226 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-06-06T03:19:25.243 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-452.el9.x86_64 2024-06-06T03:19:25.244 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-06T03:19:25.244 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-06T03:19:25.244 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-06T03:19:26.245 DEBUG:teuthology.parallel:result is None 2024-06-06T03:19:26.246 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-06T03:19:26.253 INFO:teuthology.task.internal:Creating test directory... 2024-06-06T03:19:26.253 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T03:19:26.255 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T03:19:26.258 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T03:19:26.275 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-06T03:19:26.282 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-06T03:19:26.321 INFO:teuthology.task.internal:Creating archive directory... 2024-06-06T03:19:26.321 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T03:19:26.324 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T03:19:26.327 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T03:19:26.362 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-06T03:19:26.369 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-06T03:19:26.369 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T03:19:26.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:19:26.412 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T03:19:26.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:19:26.429 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T03:19:26.431 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T03:19:26.455 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T03:19:26.499 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T03:19:26.500 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T03:19:26.505 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T03:19:26.526 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T03:19:26.531 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T03:19:26.542 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T03:19:26.544 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-06T03:19:26.552 INFO:teuthology.task.internal:Configuring sudo... 2024-06-06T03:19:26.552 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T03:19:26.555 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T03:19:26.557 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T03:19:26.611 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-06T03:19:26.620 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-06T03:19:26.620 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T03:19:26.626 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T03:19:26.628 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T03:19:26.668 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T03:19:26.724 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T03:19:26.775 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T03:19:26.863 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T03:19:26.901 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:19:26.901 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T03:19:26.967 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T03:19:27.013 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T03:19:27.064 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T03:19:27.150 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T03:19:27.193 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-06-06T03:19:27.193 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T03:19:27.258 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T03:19:27.301 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T03:19:27.358 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T03:19:27.439 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T03:19:27.483 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-06-06T03:19:27.483 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T03:19:27.547 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2024-06-06T03:19:27.550 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2024-06-06T03:19:27.553 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2024-06-06T03:19:27.589 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T03:19:27.606 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T03:19:27.629 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T03:19:28.064 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-06T03:19:28.073 INFO:teuthology.task.internal:Starting timer... 2024-06-06T03:19:28.073 INFO:teuthology.run_tasks:Running task pcp... 2024-06-06T03:19:28.123 INFO:teuthology.run_tasks:Running task selinux... 2024-06-06T03:19:28.151 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-06T03:19:28.152 DEBUG:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2024-06-06T03:19:28.199 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: 2024-06-06T03:19:28.201 DEBUG:teuthology.orchestra.run.smithi104:> sudo service auditd rotate 2024-06-06T03:19:28.281 INFO:teuthology.orchestra.run.smithi104.stdout:Rotating logs: 2024-06-06T03:19:28.282 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2024-06-06T03:19:28.350 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2024-06-06T03:19:28.352 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-06T03:19:28.352 DEBUG:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2024-06-06T03:19:28.398 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2024-06-06T03:19:28.399 DEBUG:teuthology.orchestra.run.smithi104:> /usr/sbin/getenforce 2024-06-06T03:19:28.438 INFO:teuthology.orchestra.run.smithi104.stdout:Permissive 2024-06-06T03:19:28.438 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2024-06-06T03:19:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2024-06-06T03:19:28.463 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi040.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi104.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi131.front.sepia.ceph.com': 'permissive'} 2024-06-06T03:19:28.463 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T03:19:28.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:19:28.493 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T03:19:28.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:19:28.521 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T03:19:28.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:19:28.549 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-06T03:19:28.550 DEBUG:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2024-06-06T03:19:28.577 DEBUG:teuthology.orchestra.run.smithi104:> sudo /usr/sbin/setenforce permissive 2024-06-06T03:19:28.615 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2024-06-06T03:19:28.642 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-06T03:19:28.652 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-06T03:19:28.655 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-06T03:19:28.790 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-06T03:19:28.801 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-06T03:19:28.801 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi104.front.sepia.ceph.com,smithi131.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-06T03:23:57.818 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi104.front.sepia.ceph.com'), Remote(name='ubuntu@smithi131.front.sepia.ceph.com')] 2024-06-06T03:23:57.819 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-06-06T03:23:57.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:23:57.895 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-06-06T03:23:57.978 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-06-06T03:23:57.979 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-06-06T03:23:57.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:23:58.055 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-06-06T03:23:58.133 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-06-06T03:23:58.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-06-06T03:23:58.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:23:58.209 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-06-06T03:23:58.303 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-06-06T03:23:58.303 INFO:teuthology.run_tasks:Running task clock... 2024-06-06T03:23:58.313 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-06T03:23:58.314 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T03:23:58.314 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T03:23:58.317 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T03:23:58.317 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T03:23:58.319 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T03:23:58.319 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T03:23:58.345 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T03:23:58.346 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T03:23:58.358 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T03:23:58.361 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T03:23:58.376 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T03:23:58.381 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: ntpd: command not found 2024-06-06T03:23:58.387 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2024-06-06T03:23:58.391 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T03:23:58.391 INFO:teuthology.orchestra.run.smithi104.stdout:506 Cannot talk to daemon 2024-06-06T03:23:58.398 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2024-06-06T03:23:58.404 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T03:23:58.413 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T03:23:58.416 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2024-06-06T03:23:58.417 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T03:23:58.428 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2024-06-06T03:23:58.428 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T03:23:58.441 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T03:23:58.454 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T03:23:58.472 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-06-06T03:23:58.476 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T03:23:58.476 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-06-06T03:23:58.476 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:23:58.476 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:23:58.476 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:23:58.477 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:23:58.488 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-06-06T03:23:58.493 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T03:23:58.493 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-06-06T03:23:58.493 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:23:58.493 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:23:58.493 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:23:58.493 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:23:58.511 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-06-06T03:23:58.515 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T03:23:58.515 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-06-06T03:23:58.515 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:23:58.515 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:23:58.515 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:23:58.515 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:23:58.516 INFO:teuthology.run_tasks:Running task install... 2024-06-06T03:23:58.524 DEBUG:teuthology.task.install:project ceph 2024-06-06T03:23:58.524 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-06T03:23:58.524 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-06T03:23:58.524 INFO:teuthology.task.install:Using flavor: default 2024-06-06T03:23:58.530 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-06T03:23:58.530 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-06T03:23:58.530 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T03:23:58.531 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:23:58.533 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T03:23:58.533 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:23:58.534 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T03:23:58.534 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:23:58.678 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T03:23:58.678 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T03:23:58.682 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T03:23:58.682 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T03:23:58.689 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T03:23:58.689 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T03:23:58.787 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T03:23:58.788 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:23:58.788 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T03:23:58.798 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T03:23:58.798 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-06-06T03:23:58.798 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T03:23:58.814 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T03:23:58.814 DEBUG:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T03:23:58.822 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T03:23:58.822 DEBUG:teuthology.orchestra.run.smithi104:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T03:23:58.856 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T03:23:58.856 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-06-06T03:23:58.857 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T03:23:58.881 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T03:23:58.881 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T03:23:58.883 DEBUG:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T03:23:58.886 DEBUG:teuthology.orchestra.run.smithi104:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T03:23:58.945 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T03:23:58.959 DEBUG:teuthology.orchestra.run.smithi104:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T03:23:58.960 DEBUG:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T03:23:59.018 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T03:23:59.022 INFO:teuthology.orchestra.run.smithi104.stdout:check_obsoletes = 1 2024-06-06T03:23:59.023 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum clean all 2024-06-06T03:23:59.025 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2024-06-06T03:23:59.027 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum clean all 2024-06-06T03:23:59.081 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2024-06-06T03:23:59.083 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2024-06-06T03:23:59.245 INFO:teuthology.orchestra.run.smithi104.stdout:45 files removed 2024-06-06T03:23:59.272 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T03:23:59.273 INFO:teuthology.orchestra.run.smithi040.stdout:45 files removed 2024-06-06T03:23:59.300 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T03:23:59.313 INFO:teuthology.orchestra.run.smithi131.stdout:45 files removed 2024-06-06T03:23:59.341 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T03:23:59.919 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 265 kB/s | 83 kB 00:00 2024-06-06T03:23:59.942 INFO:teuthology.orchestra.run.smithi104.stdout:ceph packages for x86_64 203 kB/s | 83 kB 00:00 2024-06-06T03:24:00.062 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 176 kB/s | 83 kB 00:00 2024-06-06T03:24:00.317 INFO:teuthology.orchestra.run.smithi104.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-06-06T03:24:00.360 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-06-06T03:24:00.505 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-06-06T03:24:00.687 INFO:teuthology.orchestra.run.smithi104.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-06-06T03:24:00.711 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-06-06T03:24:00.939 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-06-06T03:24:01.453 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-06-06T03:24:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.1 MB 00:01 2024-06-06T03:24:02.364 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.1 MB 00:01 2024-06-06T03:24:04.000 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-06-06T03:24:04.163 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-06-06T03:24:04.732 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-06-06T03:24:08.934 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.2 MB 00:01 2024-06-06T03:24:09.024 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 6.2 MB 00:00 2024-06-06T03:24:09.138 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.2 MB 00:01 2024-06-06T03:24:10.690 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 17 kB 00:00 2024-06-06T03:24:11.710 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 17 kB 00:01 2024-06-06T03:24:11.904 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 17 kB 00:01 2024-06-06T03:24:14.444 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 5.9 MB/s | 21 MB 00:03 2024-06-06T03:24:15.529 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 5.8 MB/s | 21 MB 00:03 2024-06-06T03:24:16.673 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 4.6 MB/s | 21 MB 00:04 2024-06-06T03:24:19.658 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-06-06T03:24:21.018 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-06-06T03:24:21.862 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-06-06T03:24:21.942 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T03:24:21.943 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T03:24:21.953 INFO:teuthology.orchestra.run.smithi131.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T03:24:21.954 INFO:teuthology.orchestra.run.smithi131.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T03:24:21.955 INFO:teuthology.orchestra.run.smithi131.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T03:24:21.956 INFO:teuthology.orchestra.run.smithi131.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T03:24:21.957 INFO:teuthology.orchestra.run.smithi131.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T03:24:21.959 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T03:24:21.960 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T03:24:22.026 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-06-06T03:24:22.045 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-06-06T03:24:22.046 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-06-06T03:24:22.046 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-06-06T03:24:22.046 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-06-06T03:24:22.046 INFO:teuthology.orchestra.run.smithi131.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T03:24:22.046 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T03:24:22.046 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T03:24:22.046 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T03:24:22.046 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T03:24:22.046 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T03:24:22.046 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T03:24:22.046 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T03:24:22.046 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T03:24:22.046 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T03:24:22.047 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T03:24:22.047 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T03:24:22.047 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T03:24:22.047 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T03:24:22.047 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T03:24:22.047 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T03:24:22.047 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T03:24:22.047 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T03:24:22.047 INFO:teuthology.orchestra.run.smithi131.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T03:24:22.047 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T03:24:22.047 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T03:24:22.047 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T03:24:22.048 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T03:24:22.048 INFO:teuthology.orchestra.run.smithi131.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T03:24:22.048 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T03:24:22.048 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T03:24:22.048 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T03:24:22.048 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T03:24:22.048 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T03:24:22.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T03:24:22.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T03:24:22.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T03:24:22.048 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T03:24:22.048 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T03:24:22.048 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T03:24:22.049 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T03:24:22.049 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T03:24:22.049 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2024-06-06T03:24:22.049 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T03:24:22.049 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T03:24:22.049 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T03:24:22.049 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T03:24:22.049 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T03:24:22.049 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T03:24:22.049 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T03:24:22.049 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T03:24:22.049 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T03:24:22.049 INFO:teuthology.orchestra.run.smithi131.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T03:24:22.049 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T03:24:22.050 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-06-06T03:24:22.050 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T03:24:22.050 INFO:teuthology.orchestra.run.smithi131.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T03:24:22.050 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T03:24:22.050 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T03:24:22.050 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T03:24:22.050 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T03:24:22.050 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T03:24:22.050 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T03:24:22.050 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T03:24:22.050 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T03:24:22.050 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T03:24:22.050 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T03:24:22.051 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T03:24:22.051 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T03:24:22.051 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T03:24:22.051 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T03:24:22.051 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T03:24:22.051 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T03:24:22.051 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T03:24:22.051 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T03:24:22.051 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T03:24:22.051 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T03:24:22.051 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T03:24:22.051 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T03:24:22.051 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T03:24:22.052 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T03:24:22.052 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T03:24:22.052 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T03:24:22.052 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T03:24:22.052 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T03:24:22.052 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T03:24:22.052 INFO:teuthology.orchestra.run.smithi131.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T03:24:22.052 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T03:24:22.052 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T03:24:22.052 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T03:24:22.052 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T03:24:22.052 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T03:24:22.052 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T03:24:22.052 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T03:24:22.053 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T03:24:22.053 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T03:24:22.053 INFO:teuthology.orchestra.run.smithi131.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T03:24:22.053 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T03:24:22.053 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T03:24:22.053 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T03:24:22.053 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T03:24:22.053 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T03:24:22.053 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T03:24:22.053 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T03:24:22.053 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T03:24:22.053 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T03:24:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T03:24:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T03:24:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T03:24:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T03:24:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T03:24:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T03:24:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T03:24:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T03:24:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T03:24:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T03:24:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T03:24:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T03:24:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T03:24:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T03:24:22.055 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T03:24:22.055 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T03:24:22.055 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T03:24:22.055 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T03:24:22.055 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T03:24:22.055 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T03:24:22.055 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T03:24:22.055 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T03:24:22.055 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T03:24:22.055 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T03:24:22.055 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T03:24:22.055 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T03:24:22.055 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T03:24:22.056 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T03:24:22.057 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T03:24:22.057 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T03:24:22.057 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T03:24:22.057 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T03:24:22.057 INFO:teuthology.orchestra.run.smithi131.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T03:24:22.057 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T03:24:22.057 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T03:24:22.057 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T03:24:22.057 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T03:24:22.057 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T03:24:22.057 INFO:teuthology.orchestra.run.smithi131.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T03:24:22.057 INFO:teuthology.orchestra.run.smithi131.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T03:24:22.057 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T03:24:22.058 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T03:24:22.058 INFO:teuthology.orchestra.run.smithi131.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T03:24:22.058 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T03:24:22.058 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T03:24:22.058 INFO:teuthology.orchestra.run.smithi131.stdout:Installing weak dependencies: 2024-06-06T03:24:22.058 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T03:24:22.058 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:24:22.058 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-06-06T03:24:22.058 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-06-06T03:24:22.058 INFO:teuthology.orchestra.run.smithi131.stdout:Install 148 Packages 2024-06-06T03:24:22.058 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 11 Packages 2024-06-06T03:24:22.058 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:24:22.059 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 225 M 2024-06-06T03:24:22.059 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-06-06T03:24:22.817 INFO:teuthology.orchestra.run.smithi131.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-06-06T03:24:23.051 INFO:teuthology.orchestra.run.smithi131.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 3.7 MB/s | 890 kB 00:00 2024-06-06T03:24:23.101 INFO:teuthology.orchestra.run.smithi131.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 2.8 MB/s | 145 kB 00:00 2024-06-06T03:24:23.335 INFO:teuthology.orchestra.run.smithi131.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 10 MB/s | 2.4 MB 00:00 2024-06-06T03:24:23.461 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T03:24:23.462 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T03:24:23.474 INFO:teuthology.orchestra.run.smithi040.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T03:24:23.474 INFO:teuthology.orchestra.run.smithi040.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T03:24:23.475 INFO:teuthology.orchestra.run.smithi040.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T03:24:23.476 INFO:teuthology.orchestra.run.smithi040.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T03:24:23.477 INFO:teuthology.orchestra.run.smithi040.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T03:24:23.480 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T03:24:23.481 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T03:24:23.502 INFO:teuthology.orchestra.run.smithi131.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 9.0 MB/s | 1.5 MB 00:00 2024-06-06T03:24:23.548 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-06-06T03:24:23.573 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-06-06T03:24:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-06-06T03:24:23.574 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-06-06T03:24:23.574 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-06-06T03:24:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T03:24:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T03:24:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T03:24:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T03:24:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T03:24:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T03:24:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T03:24:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T03:24:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T03:24:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T03:24:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T03:24:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T03:24:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T03:24:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T03:24:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T03:24:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T03:24:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T03:24:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T03:24:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T03:24:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T03:24:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T03:24:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T03:24:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T03:24:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T03:24:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T03:24:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T03:24:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T03:24:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T03:24:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T03:24:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T03:24:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T03:24:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T03:24:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T03:24:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T03:24:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T03:24:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T03:24:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T03:24:23.577 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2024-06-06T03:24:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T03:24:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T03:24:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T03:24:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T03:24:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T03:24:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T03:24:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T03:24:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T03:24:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T03:24:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T03:24:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T03:24:23.578 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-06-06T03:24:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T03:24:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T03:24:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T03:24:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T03:24:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T03:24:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T03:24:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T03:24:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T03:24:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T03:24:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T03:24:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T03:24:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T03:24:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T03:24:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T03:24:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T03:24:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T03:24:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T03:24:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T03:24:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T03:24:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T03:24:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T03:24:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T03:24:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T03:24:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T03:24:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T03:24:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T03:24:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T03:24:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T03:24:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T03:24:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T03:24:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T03:24:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T03:24:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T03:24:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T03:24:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T03:24:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T03:24:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T03:24:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T03:24:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T03:24:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T03:24:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T03:24:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T03:24:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T03:24:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T03:24:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T03:24:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T03:24:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T03:24:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T03:24:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T03:24:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T03:24:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T03:24:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T03:24:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T03:24:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T03:24:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T03:24:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T03:24:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T03:24:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T03:24:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T03:24:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T03:24:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T03:24:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T03:24:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T03:24:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T03:24:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T03:24:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T03:24:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T03:24:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T03:24:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T03:24:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T03:24:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T03:24:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T03:24:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T03:24:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T03:24:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T03:24:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T03:24:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T03:24:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T03:24:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T03:24:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T03:24:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T03:24:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T03:24:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T03:24:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T03:24:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T03:24:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T03:24:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T03:24:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T03:24:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T03:24:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T03:24:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T03:24:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T03:24:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T03:24:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T03:24:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T03:24:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T03:24:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T03:24:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T03:24:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T03:24:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T03:24:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T03:24:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T03:24:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T03:24:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T03:24:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T03:24:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T03:24:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T03:24:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T03:24:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T03:24:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T03:24:23.586 INFO:teuthology.orchestra.run.smithi040.stdout:Installing weak dependencies: 2024-06-06T03:24:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T03:24:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:24:23.586 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-06-06T03:24:23.586 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-06-06T03:24:23.586 INFO:teuthology.orchestra.run.smithi040.stdout:Install 148 Packages 2024-06-06T03:24:23.586 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 11 Packages 2024-06-06T03:24:23.587 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:24:23.587 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 225 M 2024-06-06T03:24:23.588 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-06-06T03:24:23.811 INFO:teuthology.orchestra.run.smithi131.stdout:(6/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 4.6 MB/s | 5.4 MB 00:01 2024-06-06T03:24:23.969 INFO:teuthology.orchestra.run.smithi131.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 10 MB/s | 4.7 MB 00:00 2024-06-06T03:24:24.163 INFO:teuthology.orchestra.run.smithi104.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T03:24:24.164 INFO:teuthology.orchestra.run.smithi104.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T03:24:24.174 INFO:teuthology.orchestra.run.smithi104.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T03:24:24.175 INFO:teuthology.orchestra.run.smithi104.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T03:24:24.176 INFO:teuthology.orchestra.run.smithi104.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T03:24:24.177 INFO:teuthology.orchestra.run.smithi104.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T03:24:24.179 INFO:teuthology.orchestra.run.smithi104.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T03:24:24.181 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T03:24:24.182 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T03:24:24.243 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-06-06T03:24:24.263 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-06-06T03:24:24.264 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-06-06T03:24:24.264 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-06-06T03:24:24.264 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-06-06T03:24:24.264 INFO:teuthology.orchestra.run.smithi104.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T03:24:24.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T03:24:24.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T03:24:24.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T03:24:24.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T03:24:24.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T03:24:24.264 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T03:24:24.265 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T03:24:24.265 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T03:24:24.265 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T03:24:24.265 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T03:24:24.265 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T03:24:24.265 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T03:24:24.265 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T03:24:24.265 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T03:24:24.265 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T03:24:24.265 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T03:24:24.265 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T03:24:24.266 INFO:teuthology.orchestra.run.smithi104.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T03:24:24.266 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T03:24:24.266 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T03:24:24.266 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T03:24:24.266 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T03:24:24.266 INFO:teuthology.orchestra.run.smithi104.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T03:24:24.266 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T03:24:24.266 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T03:24:24.266 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T03:24:24.266 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T03:24:24.266 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T03:24:24.266 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T03:24:24.266 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T03:24:24.267 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T03:24:24.267 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T03:24:24.267 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T03:24:24.267 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T03:24:24.267 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T03:24:24.267 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T03:24:24.267 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrading: 2024-06-06T03:24:24.267 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T03:24:24.267 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T03:24:24.267 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T03:24:24.267 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T03:24:24.267 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T03:24:24.267 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T03:24:24.268 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T03:24:24.268 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T03:24:24.268 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T03:24:24.268 INFO:teuthology.orchestra.run.smithi104.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T03:24:24.268 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T03:24:24.268 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-06-06T03:24:24.268 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T03:24:24.268 INFO:teuthology.orchestra.run.smithi104.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T03:24:24.268 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T03:24:24.268 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T03:24:24.268 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T03:24:24.269 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T03:24:24.269 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T03:24:24.269 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T03:24:24.269 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T03:24:24.269 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T03:24:24.269 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T03:24:24.269 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T03:24:24.269 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T03:24:24.269 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T03:24:24.269 INFO:teuthology.orchestra.run.smithi104.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T03:24:24.269 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T03:24:24.269 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T03:24:24.269 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T03:24:24.270 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T03:24:24.270 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T03:24:24.270 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T03:24:24.270 INFO:teuthology.orchestra.run.smithi104.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T03:24:24.270 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T03:24:24.270 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T03:24:24.270 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T03:24:24.270 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T03:24:24.270 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T03:24:24.270 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T03:24:24.270 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T03:24:24.270 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T03:24:24.270 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T03:24:24.271 INFO:teuthology.orchestra.run.smithi104.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T03:24:24.271 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T03:24:24.271 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T03:24:24.271 INFO:teuthology.orchestra.run.smithi104.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T03:24:24.271 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T03:24:24.271 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T03:24:24.271 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T03:24:24.271 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T03:24:24.271 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T03:24:24.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T03:24:24.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T03:24:24.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T03:24:24.271 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T03:24:24.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T03:24:24.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T03:24:24.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T03:24:24.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T03:24:24.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T03:24:24.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T03:24:24.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T03:24:24.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T03:24:24.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T03:24:24.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T03:24:24.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T03:24:24.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T03:24:24.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T03:24:24.272 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T03:24:24.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T03:24:24.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T03:24:24.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T03:24:24.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T03:24:24.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T03:24:24.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T03:24:24.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T03:24:24.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T03:24:24.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T03:24:24.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T03:24:24.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T03:24:24.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T03:24:24.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T03:24:24.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T03:24:24.274 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T03:24:24.274 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T03:24:24.274 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T03:24:24.274 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T03:24:24.274 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T03:24:24.274 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T03:24:24.274 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T03:24:24.274 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T03:24:24.274 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T03:24:24.274 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T03:24:24.274 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T03:24:24.274 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T03:24:24.274 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T03:24:24.274 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T03:24:24.275 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T03:24:24.275 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T03:24:24.275 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T03:24:24.275 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T03:24:24.275 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T03:24:24.275 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T03:24:24.275 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T03:24:24.275 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T03:24:24.275 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T03:24:24.276 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T03:24:24.276 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T03:24:24.276 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T03:24:24.276 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T03:24:24.276 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T03:24:24.276 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T03:24:24.276 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T03:24:24.276 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T03:24:24.276 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T03:24:24.276 INFO:teuthology.orchestra.run.smithi104.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T03:24:24.276 INFO:teuthology.orchestra.run.smithi104.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T03:24:24.276 INFO:teuthology.orchestra.run.smithi104.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T03:24:24.276 INFO:teuthology.orchestra.run.smithi104.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T03:24:24.276 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T03:24:24.277 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T03:24:24.277 INFO:teuthology.orchestra.run.smithi104.stdout:Installing weak dependencies: 2024-06-06T03:24:24.277 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T03:24:24.277 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:24:24.277 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-06-06T03:24:24.277 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-06-06T03:24:24.277 INFO:teuthology.orchestra.run.smithi104.stdout:Install 148 Packages 2024-06-06T03:24:24.277 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrade 11 Packages 2024-06-06T03:24:24.277 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:24:24.278 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 225 M 2024-06-06T03:24:24.278 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-06-06T03:24:24.403 INFO:teuthology.orchestra.run.smithi040.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-06-06T03:24:25.194 INFO:teuthology.orchestra.run.smithi040.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 890 kB 00:00 2024-06-06T03:24:25.296 INFO:teuthology.orchestra.run.smithi104.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-06-06T03:24:25.312 INFO:teuthology.orchestra.run.smithi040.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.2 MB/s | 145 kB 00:00 2024-06-06T03:24:25.728 INFO:teuthology.orchestra.run.smithi131.stdout:(8/159): ceph-common-19.0.0-2946.g3d89accf.el9. 6.8 MB/s | 21 MB 00:03 2024-06-06T03:24:25.828 INFO:teuthology.orchestra.run.smithi131.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 252 kB/s | 25 kB 00:00 2024-06-06T03:24:25.896 INFO:teuthology.orchestra.run.smithi104.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.4 MB/s | 890 kB 00:00 2024-06-06T03:24:25.962 INFO:teuthology.orchestra.run.smithi040.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 3.6 MB/s | 2.4 MB 00:00 2024-06-06T03:24:25.963 INFO:teuthology.orchestra.run.smithi104.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 2.1 MB/s | 145 kB 00:00 2024-06-06T03:24:26.003 INFO:teuthology.orchestra.run.smithi131.stdout:(10/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 7.7 MB/s | 17 MB 00:02 2024-06-06T03:24:26.170 INFO:teuthology.orchestra.run.smithi131.stdout:(11/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 4.7 MB/s | 10 MB 00:02 2024-06-06T03:24:26.195 INFO:teuthology.orchestra.run.smithi131.stdout:(12/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.1 MB/s | 221 kB 00:00 2024-06-06T03:24:26.246 INFO:teuthology.orchestra.run.smithi131.stdout:(13/159): cephfs-shell-19.0.0-2946.g3d89accf.el 359 kB/s | 27 kB 00:00 2024-06-06T03:24:26.262 INFO:teuthology.orchestra.run.smithi040.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 5.0 MB/s | 1.5 MB 00:00 2024-06-06T03:24:26.271 INFO:teuthology.orchestra.run.smithi131.stdout:(14/159): libcephfs-devel-19.0.0-2946.g3d89accf 421 kB/s | 32 kB 00:00 2024-06-06T03:24:26.379 INFO:teuthology.orchestra.run.smithi131.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.5 MB/s | 163 kB 00:00 2024-06-06T03:24:26.497 INFO:teuthology.orchestra.run.smithi131.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.1 MB/s | 127 kB 00:00 2024-06-06T03:24:26.555 INFO:teuthology.orchestra.run.smithi131.stdout:(17/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 2.4 MB/s | 747 kB 00:00 2024-06-06T03:24:27.013 INFO:teuthology.orchestra.run.smithi104.stdout:(4/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-06-06T03:24:27.072 INFO:teuthology.orchestra.run.smithi104.stdout:(5/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 2.1 MB/s | 2.4 MB 00:01 2024-06-06T03:24:27.129 INFO:teuthology.orchestra.run.smithi040.stdout:(6/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 5.4 MB/s | 4.7 MB 00:00 2024-06-06T03:24:27.372 INFO:teuthology.orchestra.run.smithi104.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 4.2 MB/s | 1.5 MB 00:00 2024-06-06T03:24:27.464 INFO:teuthology.orchestra.run.smithi131.stdout:(18/159): libradosstriper1-19.0.0-2946.g3d89acc 518 kB/s | 501 kB 00:00 2024-06-06T03:24:27.664 INFO:teuthology.orchestra.run.smithi131.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 226 kB/s | 45 kB 00:00 2024-06-06T03:24:28.248 INFO:teuthology.orchestra.run.smithi131.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 231 kB/s | 135 kB 00:00 2024-06-06T03:24:28.615 INFO:teuthology.orchestra.run.smithi131.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 441 kB/s | 162 kB 00:00 2024-06-06T03:24:29.207 INFO:teuthology.orchestra.run.smithi131.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 543 kB/s | 321 kB 00:00 2024-06-06T03:24:29.513 INFO:teuthology.orchestra.run.smithi104.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 1.9 MB/s | 4.7 MB 00:02 2024-06-06T03:24:29.657 INFO:teuthology.orchestra.run.smithi131.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 666 kB/s | 300 kB 00:00 2024-06-06T03:24:29.782 INFO:teuthology.orchestra.run.smithi131.stdout:(24/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 1.6 MB/s | 5.3 MB 00:03 2024-06-06T03:24:29.858 INFO:teuthology.orchestra.run.smithi131.stdout:(25/159): python3-rgw-19.0.0-2946.g3d89accf.el9 496 kB/s | 99 kB 00:00 2024-06-06T03:24:29.883 INFO:teuthology.orchestra.run.smithi131.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 850 kB/s | 85 kB 00:00 2024-06-06T03:24:30.033 INFO:teuthology.orchestra.run.smithi131.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.1 MB/s | 171 kB 00:00 2024-06-06T03:24:30.117 INFO:teuthology.orchestra.run.smithi131.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 379 kB/s | 32 kB 00:00 2024-06-06T03:24:30.259 INFO:teuthology.orchestra.run.smithi131.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.0 MB/s | 142 kB 00:00 2024-06-06T03:24:30.312 INFO:teuthology.orchestra.run.smithi040.stdout:(7/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 887 kB/s | 5.4 MB 00:06 2024-06-06T03:24:30.496 INFO:teuthology.orchestra.run.smithi040.stdout:(8/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x86 5.0 MB/s | 17 MB 00:03 2024-06-06T03:24:30.563 INFO:teuthology.orchestra.run.smithi040.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 380 kB/s | 25 kB 00:00 2024-06-06T03:24:30.922 INFO:teuthology.orchestra.run.smithi104.stdout:(8/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x86 4.7 MB/s | 17 MB 00:03 2024-06-06T03:24:31.014 INFO:teuthology.orchestra.run.smithi104.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 275 kB/s | 25 kB 00:00 2024-06-06T03:24:31.822 INFO:teuthology.orchestra.run.smithi104.stdout:(10/159): ceph-common-19.0.0-2946.g3d89accf.el9 3.1 MB/s | 21 MB 00:06 2024-06-06T03:24:31.931 INFO:teuthology.orchestra.run.smithi104.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.0 MB/s | 221 kB 00:00 2024-06-06T03:24:31.967 INFO:teuthology.orchestra.run.smithi131.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 2.1 MB/s | 3.5 MB 00:01 2024-06-06T03:24:32.014 INFO:teuthology.orchestra.run.smithi104.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 325 kB/s | 27 kB 00:00 2024-06-06T03:24:32.097 INFO:teuthology.orchestra.run.smithi040.stdout:(10/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 5.7 MB/s | 10 MB 00:01 2024-06-06T03:24:32.098 INFO:teuthology.orchestra.run.smithi104.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 382 kB/s | 32 kB 00:00 2024-06-06T03:24:32.163 INFO:teuthology.orchestra.run.smithi040.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 3.3 MB/s | 221 kB 00:00 2024-06-06T03:24:32.230 INFO:teuthology.orchestra.run.smithi040.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 405 kB/s | 27 kB 00:00 2024-06-06T03:24:32.297 INFO:teuthology.orchestra.run.smithi040.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 477 kB/s | 32 kB 00:00 2024-06-06T03:24:32.381 INFO:teuthology.orchestra.run.smithi104.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 2.6 MB/s | 747 kB 00:00 2024-06-06T03:24:32.473 INFO:teuthology.orchestra.run.smithi104.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.7 MB/s | 163 kB 00:00 2024-06-06T03:24:32.539 INFO:teuthology.orchestra.run.smithi040.stdout:(14/159): ceph-common-19.0.0-2946.g3d89accf.el9 2.5 MB/s | 21 MB 00:08 2024-06-06T03:24:32.548 INFO:teuthology.orchestra.run.smithi104.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.6 MB/s | 127 kB 00:00 2024-06-06T03:24:32.573 INFO:teuthology.orchestra.run.smithi040.stdout:(15/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 2.6 MB/s | 747 kB 00:00 2024-06-06T03:24:32.648 INFO:teuthology.orchestra.run.smithi040.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.7 MB/s | 127 kB 00:00 2024-06-06T03:24:32.674 INFO:teuthology.orchestra.run.smithi040.stdout:(17/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.2 MB/s | 163 kB 00:00 2024-06-06T03:24:32.774 INFO:teuthology.orchestra.run.smithi104.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 2.2 MB/s | 501 kB 00:00 2024-06-06T03:24:32.782 INFO:teuthology.orchestra.run.smithi040.stdout:(18/159): libradosstriper1-19.0.0-2946.g3d89acc 3.7 MB/s | 501 kB 00:00 2024-06-06T03:24:32.849 INFO:teuthology.orchestra.run.smithi040.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 676 kB/s | 45 kB 00:00 2024-06-06T03:24:32.924 INFO:teuthology.orchestra.run.smithi040.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.8 MB/s | 135 kB 00:00 2024-06-06T03:24:32.984 INFO:teuthology.orchestra.run.smithi131.stdout:(31/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 1.0 MB/s | 3.1 MB 00:03 2024-06-06T03:24:32.999 INFO:teuthology.orchestra.run.smithi040.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 2.1 MB/s | 162 kB 00:00 2024-06-06T03:24:33.091 INFO:teuthology.orchestra.run.smithi040.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.4 MB/s | 321 kB 00:00 2024-06-06T03:24:33.175 INFO:teuthology.orchestra.run.smithi040.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 3.5 MB/s | 300 kB 00:00 2024-06-06T03:24:33.227 INFO:teuthology.orchestra.run.smithi131.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 1.0 MB/s | 249 kB 00:00 2024-06-06T03:24:33.242 INFO:teuthology.orchestra.run.smithi040.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.5 MB/s | 99 kB 00:00 2024-06-06T03:24:33.309 INFO:teuthology.orchestra.run.smithi040.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.3 MB/s | 85 kB 00:00 2024-06-06T03:24:33.318 INFO:teuthology.orchestra.run.smithi131.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 539 kB/s | 49 kB 00:00 2024-06-06T03:24:33.419 INFO:teuthology.orchestra.run.smithi131.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 164 kB/s | 16 kB 00:00 2024-06-06T03:24:33.766 INFO:teuthology.orchestra.run.smithi104.stdout:(18/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 2.4 MB/s | 10 MB 00:04 2024-06-06T03:24:33.834 INFO:teuthology.orchestra.run.smithi040.stdout:(26/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 5.9 MB/s | 3.1 MB 00:00 2024-06-06T03:24:33.841 INFO:teuthology.orchestra.run.smithi104.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 605 kB/s | 45 kB 00:00 2024-06-06T03:24:33.885 INFO:teuthology.orchestra.run.smithi040.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 3.3 MB/s | 171 kB 00:00 2024-06-06T03:24:33.924 INFO:teuthology.orchestra.run.smithi104.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.6 MB/s | 135 kB 00:00 2024-06-06T03:24:33.943 INFO:teuthology.orchestra.run.smithi040.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 546 kB/s | 32 kB 00:00 2024-06-06T03:24:34.002 INFO:teuthology.orchestra.run.smithi040.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 2.4 MB/s | 142 kB 00:00 2024-06-06T03:24:34.025 INFO:teuthology.orchestra.run.smithi104.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.6 MB/s | 162 kB 00:00 2024-06-06T03:24:34.052 INFO:teuthology.orchestra.run.smithi131.stdout:(35/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 5.9 MB/s | 48 MB 00:08 2024-06-06T03:24:34.077 INFO:teuthology.orchestra.run.smithi131.stdout:(36/159): ceph-volume-19.0.0-2946.g3d89accf.el9 400 kB/s | 264 kB 00:00 2024-06-06T03:24:34.170 INFO:teuthology.orchestra.run.smithi131.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 199 kB/s | 18 kB 00:00 2024-06-06T03:24:34.183 INFO:teuthology.orchestra.run.smithi104.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 2.0 MB/s | 321 kB 00:00 2024-06-06T03:24:34.186 INFO:teuthology.orchestra.run.smithi040.stdout:(30/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 3.5 MB/s | 5.3 MB 00:01 2024-06-06T03:24:34.215 INFO:teuthology.orchestra.run.smithi131.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:34.300 INFO:teuthology.orchestra.run.smithi131.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 643 kB/s | 80 kB 00:00 2024-06-06T03:24:34.320 INFO:teuthology.orchestra.run.smithi131.stdout:(39/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 2.8 MB/s | 763 kB 00:00 2024-06-06T03:24:34.321 INFO:teuthology.orchestra.run.smithi131.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:34.324 INFO:teuthology.orchestra.run.smithi131.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:34.359 INFO:teuthology.orchestra.run.smithi104.stdout:(23/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 3.4 MB/s | 5.3 MB 00:01 2024-06-06T03:24:34.384 INFO:teuthology.orchestra.run.smithi104.stdout:(24/159): python3-rbd-19.0.0-2946.g3d89accf.el9 1.5 MB/s | 300 kB 00:00 2024-06-06T03:24:34.412 INFO:teuthology.orchestra.run.smithi131.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-06-06T03:24:34.434 INFO:teuthology.orchestra.run.smithi104.stdout:(25/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.3 MB/s | 99 kB 00:00 2024-06-06T03:24:34.437 INFO:teuthology.orchestra.run.smithi131.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-06-06T03:24:34.460 INFO:teuthology.orchestra.run.smithi104.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 85 kB 00:00 2024-06-06T03:24:34.464 INFO:teuthology.orchestra.run.smithi131.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-06-06T03:24:34.512 INFO:teuthology.orchestra.run.smithi040.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 6.9 MB/s | 3.5 MB 00:00 2024-06-06T03:24:34.513 INFO:teuthology.orchestra.run.smithi131.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-06-06T03:24:34.535 INFO:teuthology.orchestra.run.smithi104.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.2 MB/s | 171 kB 00:00 2024-06-06T03:24:34.555 INFO:teuthology.orchestra.run.smithi131.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-06-06T03:24:34.570 INFO:teuthology.orchestra.run.smithi040.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 4.2 MB/s | 249 kB 00:00 2024-06-06T03:24:34.610 INFO:teuthology.orchestra.run.smithi104.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 423 kB/s | 32 kB 00:00 2024-06-06T03:24:34.620 INFO:teuthology.orchestra.run.smithi040.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 991 kB/s | 49 kB 00:00 2024-06-06T03:24:34.664 INFO:teuthology.orchestra.run.smithi131.stdout:(45/159): ceph-mgr-diskprediction-local-19.0.0- 2.7 MB/s | 7.4 MB 00:02 2024-06-06T03:24:34.677 INFO:teuthology.orchestra.run.smithi104.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 2.1 MB/s | 142 kB 00:00 2024-06-06T03:24:34.678 INFO:teuthology.orchestra.run.smithi040.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 282 kB/s | 16 kB 00:00 2024-06-06T03:24:34.689 INFO:teuthology.orchestra.run.smithi131.stdout:(46/159): python3-pycparser-2.20-6.el9.noarch.r 769 kB/s | 135 kB 00:00 2024-06-06T03:24:34.714 INFO:teuthology.orchestra.run.smithi131.stdout:(47/159): python3-requests-2.25.1-8.el9.noarch. 786 kB/s | 125 kB 00:00 2024-06-06T03:24:34.715 INFO:teuthology.orchestra.run.smithi131.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T03:24:34.717 INFO:teuthology.orchestra.run.smithi131.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:34.762 INFO:teuthology.orchestra.run.smithi040.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 3.1 MB/s | 264 kB 00:00 2024-06-06T03:24:34.765 INFO:teuthology.orchestra.run.smithi131.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 8.9 MB/s | 689 kB 00:00 2024-06-06T03:24:34.799 INFO:teuthology.orchestra.run.smithi131.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-06-06T03:24:34.799 INFO:teuthology.orchestra.run.smithi131.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:34.824 INFO:teuthology.orchestra.run.smithi131.stdout:(50/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.3 MB/s | 215 kB 00:00 2024-06-06T03:24:34.824 INFO:teuthology.orchestra.run.smithi131.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:34.849 INFO:teuthology.orchestra.run.smithi131.stdout:(51/159): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-06-06T03:24:34.874 INFO:teuthology.orchestra.run.smithi131.stdout:(52/159): elfutils-libelf-devel-0.191-4.el9.x86 287 kB/s | 22 kB 00:00 2024-06-06T03:24:34.900 INFO:teuthology.orchestra.run.smithi131.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 646 kB/s | 33 kB 00:00 2024-06-06T03:24:34.904 INFO:teuthology.orchestra.run.smithi040.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 5.3 MB/s | 763 kB 00:00 2024-06-06T03:24:34.925 INFO:teuthology.orchestra.run.smithi131.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-06-06T03:24:34.950 INFO:teuthology.orchestra.run.smithi131.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-06-06T03:24:34.963 INFO:teuthology.orchestra.run.smithi040.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 313 kB/s | 18 kB 00:00 2024-06-06T03:24:34.976 INFO:teuthology.orchestra.run.smithi131.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-06-06T03:24:35.001 INFO:teuthology.orchestra.run.smithi131.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-06-06T03:24:35.027 INFO:teuthology.orchestra.run.smithi131.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-06-06T03:24:35.027 INFO:teuthology.orchestra.run.smithi040.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:35.068 INFO:teuthology.orchestra.run.smithi131.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-06-06T03:24:35.113 INFO:teuthology.orchestra.run.smithi040.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-06-06T03:24:35.118 INFO:teuthology.orchestra.run.smithi040.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:35.136 INFO:teuthology.orchestra.run.smithi104.stdout:(30/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 4.4 MB/s | 3.1 MB 00:00 2024-06-06T03:24:35.149 INFO:teuthology.orchestra.run.smithi131.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 4.7 MB/s | 582 kB 00:00 2024-06-06T03:24:35.178 INFO:teuthology.orchestra.run.smithi131.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-06-06T03:24:35.197 INFO:teuthology.orchestra.run.smithi040.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2024-06-06T03:24:35.200 INFO:teuthology.orchestra.run.smithi040.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:35.203 INFO:teuthology.orchestra.run.smithi131.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 885 kB/s | 47 kB 00:00 2024-06-06T03:24:35.228 INFO:teuthology.orchestra.run.smithi131.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-06T03:24:35.239 INFO:teuthology.orchestra.run.smithi040.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-06-06T03:24:35.245 INFO:teuthology.orchestra.run.smithi131.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-06-06T03:24:35.279 INFO:teuthology.orchestra.run.smithi131.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-06-06T03:24:35.281 INFO:teuthology.orchestra.run.smithi040.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-06-06T03:24:35.329 INFO:teuthology.orchestra.run.smithi131.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.6 MB/s | 3.0 MB 00:00 2024-06-06T03:24:35.348 INFO:teuthology.orchestra.run.smithi040.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-06-06T03:24:35.381 INFO:teuthology.orchestra.run.smithi040.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-06-06T03:24:35.413 INFO:teuthology.orchestra.run.smithi131.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-06-06T03:24:35.423 INFO:teuthology.orchestra.run.smithi040.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-06-06T03:24:35.455 INFO:teuthology.orchestra.run.smithi131.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-06-06T03:24:35.466 INFO:teuthology.orchestra.run.smithi040.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-06-06T03:24:35.497 INFO:teuthology.orchestra.run.smithi131.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-06-06T03:24:35.507 INFO:teuthology.orchestra.run.smithi040.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-06-06T03:24:35.511 INFO:teuthology.orchestra.run.smithi040.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T03:24:35.522 INFO:teuthology.orchestra.run.smithi131.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-06-06T03:24:35.553 INFO:teuthology.orchestra.run.smithi104.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 4.0 MB/s | 3.5 MB 00:00 2024-06-06T03:24:35.589 INFO:teuthology.orchestra.run.smithi131.stdout:(71/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-06-06T03:24:35.623 INFO:teuthology.orchestra.run.smithi131.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-06-06T03:24:35.648 INFO:teuthology.orchestra.run.smithi131.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-06-06T03:24:35.661 INFO:teuthology.orchestra.run.smithi104.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.2 MB/s | 249 kB 00:00 2024-06-06T03:24:35.690 INFO:teuthology.orchestra.run.smithi131.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-06-06T03:24:35.716 INFO:teuthology.orchestra.run.smithi040.stdout:(47/159): ceph-mgr-diskprediction-local-19.0.0- 4.8 MB/s | 7.4 MB 00:01 2024-06-06T03:24:35.719 INFO:teuthology.orchestra.run.smithi040.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:35.728 INFO:teuthology.orchestra.run.smithi104.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 739 kB/s | 49 kB 00:00 2024-06-06T03:24:35.774 INFO:teuthology.orchestra.run.smithi131.stdout:(75/159): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-06-06T03:24:35.787 INFO:teuthology.orchestra.run.smithi104.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 282 kB/s | 16 kB 00:00 2024-06-06T03:24:35.799 INFO:teuthology.orchestra.run.smithi131.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-06-06T03:24:35.824 INFO:teuthology.orchestra.run.smithi131.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-06-06T03:24:35.850 INFO:teuthology.orchestra.run.smithi131.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-06-06T03:24:35.875 INFO:teuthology.orchestra.run.smithi131.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 379 kB/s | 19 kB 00:00 2024-06-06T03:24:35.901 INFO:teuthology.orchestra.run.smithi131.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-06-06T03:24:35.903 INFO:teuthology.orchestra.run.smithi104.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.2 MB/s | 264 kB 00:00 2024-06-06T03:24:35.918 INFO:teuthology.orchestra.run.smithi131.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 828 kB/s | 35 kB 00:00 2024-06-06T03:24:36.001 INFO:teuthology.orchestra.run.smithi131.stdout:(82/159): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-06-06T03:24:36.026 INFO:teuthology.orchestra.run.smithi131.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-06-06T03:24:36.052 INFO:teuthology.orchestra.run.smithi131.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-06-06T03:24:36.077 INFO:teuthology.orchestra.run.smithi131.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-06T03:24:36.103 INFO:teuthology.orchestra.run.smithi131.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-06-06T03:24:36.104 INFO:teuthology.orchestra.run.smithi104.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 3.7 MB/s | 763 kB 00:00 2024-06-06T03:24:36.128 INFO:teuthology.orchestra.run.smithi131.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-06-06T03:24:36.150 INFO:teuthology.orchestra.run.smithi040.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-06-06T03:24:36.153 INFO:teuthology.orchestra.run.smithi040.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:36.153 INFO:teuthology.orchestra.run.smithi131.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-06T03:24:36.179 INFO:teuthology.orchestra.run.smithi104.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 244 kB/s | 18 kB 00:00 2024-06-06T03:24:36.187 INFO:teuthology.orchestra.run.smithi131.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-06-06T03:24:36.212 INFO:teuthology.orchestra.run.smithi131.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 873 kB/s | 51 kB 00:00 2024-06-06T03:24:36.221 INFO:teuthology.orchestra.run.smithi104.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:36.238 INFO:teuthology.orchestra.run.smithi131.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-06T03:24:36.250 INFO:teuthology.orchestra.run.smithi040.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-06-06T03:24:36.279 INFO:teuthology.orchestra.run.smithi131.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-06-06T03:24:36.296 INFO:teuthology.orchestra.run.smithi104.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 689 kB/s | 80 kB 00:00 2024-06-06T03:24:36.299 INFO:teuthology.orchestra.run.smithi104.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:36.308 INFO:teuthology.orchestra.run.smithi040.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 373 kB/s | 22 kB 00:00 2024-06-06T03:24:36.346 INFO:teuthology.orchestra.run.smithi131.stdout:(93/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-06-06T03:24:36.372 INFO:teuthology.orchestra.run.smithi131.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 481 kB/s | 44 kB 00:00 2024-06-06T03:24:36.388 INFO:teuthology.orchestra.run.smithi104.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-06-06T03:24:36.392 INFO:teuthology.orchestra.run.smithi040.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 1.4 MB/s | 943 kB 00:00 2024-06-06T03:24:36.397 INFO:teuthology.orchestra.run.smithi131.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-06-06T03:24:36.422 INFO:teuthology.orchestra.run.smithi131.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-06-06T03:24:36.443 INFO:teuthology.orchestra.run.smithi040.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-06-06T03:24:36.448 INFO:teuthology.orchestra.run.smithi131.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-06-06T03:24:36.468 INFO:teuthology.orchestra.run.smithi040.stdout:(53/159): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 313 kB 00:00 2024-06-06T03:24:36.472 INFO:teuthology.orchestra.run.smithi104.stdout:(40/159): ceph-mgr-diskprediction-local-19.0.0- 5.5 MB/s | 7.4 MB 00:01 2024-06-06T03:24:36.472 INFO:teuthology.orchestra.run.smithi104.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:36.473 INFO:teuthology.orchestra.run.smithi131.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-06-06T03:24:36.474 INFO:teuthology.orchestra.run.smithi104.stdout:[MIRROR] python3-cffi-1.14.5-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-cffi-1.14.5-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:36.498 INFO:teuthology.orchestra.run.smithi131.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-06T03:24:36.514 INFO:teuthology.orchestra.run.smithi104.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-06-06T03:24:36.518 INFO:teuthology.orchestra.run.smithi040.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-06-06T03:24:36.524 INFO:teuthology.orchestra.run.smithi131.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-06T03:24:36.549 INFO:teuthology.orchestra.run.smithi131.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-06T03:24:36.574 INFO:teuthology.orchestra.run.smithi131.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-06-06T03:24:36.575 INFO:teuthology.orchestra.run.smithi131.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:36.577 INFO:teuthology.orchestra.run.smithi040.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-06-06T03:24:36.577 INFO:teuthology.orchestra.run.smithi131.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:36.580 INFO:teuthology.orchestra.run.smithi104.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-06-06T03:24:36.623 INFO:teuthology.orchestra.run.smithi104.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-06-06T03:24:36.648 INFO:teuthology.orchestra.run.smithi104.stdout:(44/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-06-06T03:24:36.673 INFO:teuthology.orchestra.run.smithi104.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-06-06T03:24:36.699 INFO:teuthology.orchestra.run.smithi104.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-06-06T03:24:36.724 INFO:teuthology.orchestra.run.smithi104.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-06-06T03:24:36.724 INFO:teuthology.orchestra.run.smithi104.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T03:24:36.727 INFO:teuthology.orchestra.run.smithi104.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:36.794 INFO:teuthology.orchestra.run.smithi040.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 739 kB/s | 160 kB 00:00 2024-06-06T03:24:36.800 INFO:teuthology.orchestra.run.smithi131.stdout:(103/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-06-06T03:24:36.825 INFO:teuthology.orchestra.run.smithi131.stdout:(104/159): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-06-06T03:24:36.861 INFO:teuthology.orchestra.run.smithi040.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 679 kB/s | 45 kB 00:00 2024-06-06T03:24:36.976 INFO:teuthology.orchestra.run.smithi131.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 3.1 MB/s | 548 kB 00:00 2024-06-06T03:24:37.001 INFO:teuthology.orchestra.run.smithi131.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-06-06T03:24:37.026 INFO:teuthology.orchestra.run.smithi131.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-06-06T03:24:37.060 INFO:teuthology.orchestra.run.smithi131.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 787 kB/s | 26 kB 00:00 2024-06-06T03:24:37.070 INFO:teuthology.orchestra.run.smithi040.stdout:(58/159): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-06-06T03:24:37.102 INFO:teuthology.orchestra.run.smithi131.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-06-06T03:24:37.155 INFO:teuthology.orchestra.run.smithi131.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-06-06T03:24:37.220 INFO:teuthology.orchestra.run.smithi040.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-06-06T03:24:37.244 INFO:teuthology.orchestra.run.smithi131.stdout:(111/159): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-06-06T03:24:37.266 INFO:teuthology.orchestra.run.smithi104.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 1.2 MB/s | 689 kB 00:00 2024-06-06T03:24:37.268 INFO:teuthology.orchestra.run.smithi104.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:37.269 INFO:teuthology.orchestra.run.smithi131.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-06-06T03:24:37.278 INFO:teuthology.orchestra.run.smithi040.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 438 kB/s | 25 kB 00:00 2024-06-06T03:24:37.341 INFO:teuthology.orchestra.run.smithi104.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 1.5 MB/s | 943 kB 00:00 2024-06-06T03:24:37.345 INFO:teuthology.orchestra.run.smithi040.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 713 kB/s | 47 kB 00:00 2024-06-06T03:24:37.366 INFO:teuthology.orchestra.run.smithi104.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-06-06T03:24:37.392 INFO:teuthology.orchestra.run.smithi104.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-06-06T03:24:37.446 INFO:teuthology.orchestra.run.smithi040.stdout:(62/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-06-06T03:24:37.459 INFO:teuthology.orchestra.run.smithi104.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-06-06T03:24:37.511 INFO:teuthology.orchestra.run.smithi131.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-06-06T03:24:37.530 INFO:teuthology.orchestra.run.smithi040.stdout:(63/159): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-06-06T03:24:37.553 INFO:teuthology.orchestra.run.smithi131.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-06-06T03:24:37.587 INFO:teuthology.orchestra.run.smithi131.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 970 kB/s | 32 kB 00:00 2024-06-06T03:24:37.588 INFO:teuthology.orchestra.run.smithi040.stdout:(64/159): openblas-0.3.21-2.el9.x86_64.rpm 604 kB/s | 35 kB 00:00 2024-06-06T03:24:37.612 INFO:teuthology.orchestra.run.smithi131.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 598 kB/s | 15 kB 00:00 2024-06-06T03:24:37.646 INFO:teuthology.orchestra.run.smithi131.stdout:(117/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.0 MB/s | 839 kB 00:00 2024-06-06T03:24:37.680 INFO:teuthology.orchestra.run.smithi131.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-06-06T03:24:37.755 INFO:teuthology.orchestra.run.smithi131.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-06-06T03:24:37.780 INFO:teuthology.orchestra.run.smithi131.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-06-06T03:24:37.856 INFO:teuthology.orchestra.run.smithi131.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-06-06T03:24:37.909 INFO:teuthology.orchestra.run.smithi104.stdout:(53/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 7.0 MB/s | 48 MB 00:06 2024-06-06T03:24:37.923 INFO:teuthology.orchestra.run.smithi131.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-06-06T03:24:37.934 INFO:teuthology.orchestra.run.smithi104.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 553 kB/s | 313 kB 00:00 2024-06-06T03:24:37.939 INFO:teuthology.orchestra.run.smithi131.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 650 kB/s | 11 kB 00:00 2024-06-06T03:24:37.965 INFO:teuthology.orchestra.run.smithi131.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-06-06T03:24:37.984 INFO:teuthology.orchestra.run.smithi104.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-06-06T03:24:37.990 INFO:teuthology.orchestra.run.smithi131.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 924 kB/s | 23 kB 00:00 2024-06-06T03:24:38.009 INFO:teuthology.orchestra.run.smithi104.stdout:(56/159): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-06-06T03:24:38.013 INFO:teuthology.orchestra.run.smithi040.stdout:(65/159): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-06-06T03:24:38.015 INFO:teuthology.orchestra.run.smithi131.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 779 kB/s | 19 kB 00:00 2024-06-06T03:24:38.041 INFO:teuthology.orchestra.run.smithi131.stdout:(127/159): python3-jaraco-text-3.2.0-6.el9.noar 796 kB/s | 20 kB 00:00 2024-06-06T03:24:38.064 INFO:teuthology.orchestra.run.smithi040.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.8 MB/s | 3.0 MB 00:01 2024-06-06T03:24:38.101 INFO:teuthology.orchestra.run.smithi104.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 494 kB/s | 45 kB 00:00 2024-06-06T03:24:38.127 INFO:teuthology.orchestra.run.smithi104.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-06-06T03:24:38.236 INFO:teuthology.orchestra.run.smithi104.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-06-06T03:24:38.281 INFO:teuthology.orchestra.run.smithi040.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-06-06T03:24:38.285 INFO:teuthology.orchestra.run.smithi104.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 511 kB/s | 25 kB 00:00 2024-06-06T03:24:38.336 INFO:teuthology.orchestra.run.smithi104.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-06-06T03:24:38.390 INFO:teuthology.orchestra.run.smithi040.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-06-06T03:24:38.400 INFO:teuthology.orchestra.run.smithi131.stdout:(128/159): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2024-06-06T03:24:38.403 INFO:teuthology.orchestra.run.smithi104.stdout:(62/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-06-06T03:24:38.436 INFO:teuthology.orchestra.run.smithi131.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.3 MB/s | 46 kB 00:00 2024-06-06T03:24:38.448 INFO:teuthology.orchestra.run.smithi040.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-06T03:24:38.461 INFO:teuthology.orchestra.run.smithi104.stdout:(63/159): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-06-06T03:24:38.486 INFO:teuthology.orchestra.run.smithi131.stdout:(130/159): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-06-06T03:24:38.511 INFO:teuthology.orchestra.run.smithi131.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-06-06T03:24:38.512 INFO:teuthology.orchestra.run.smithi104.stdout:(64/159): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-06-06T03:24:38.536 INFO:teuthology.orchestra.run.smithi131.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-06-06T03:24:38.537 INFO:teuthology.orchestra.run.smithi104.stdout:(65/159): libtool-2.4.6-45.el9.x86_64.rpm 1.4 MB/s | 582 kB 00:00 2024-06-06T03:24:38.603 INFO:teuthology.orchestra.run.smithi131.stdout:(133/159): libarrow-9.0.0-11.el9.x86_64.rpm 2.8 MB/s | 4.4 MB 00:01 2024-06-06T03:24:38.629 INFO:teuthology.orchestra.run.smithi131.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-06-06T03:24:38.654 INFO:teuthology.orchestra.run.smithi131.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-06-06T03:24:38.654 INFO:teuthology.orchestra.run.smithi104.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-06-06T03:24:38.679 INFO:teuthology.orchestra.run.smithi131.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-06-06T03:24:38.704 INFO:teuthology.orchestra.run.smithi131.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-06-06T03:24:38.715 INFO:teuthology.orchestra.run.smithi040.stdout:(70/159): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-06-06T03:24:38.738 INFO:teuthology.orchestra.run.smithi131.stdout:(138/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-06-06T03:24:38.746 INFO:teuthology.orchestra.run.smithi104.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-06-06T03:24:38.763 INFO:teuthology.orchestra.run.smithi131.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-06-06T03:24:38.788 INFO:teuthology.orchestra.run.smithi131.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-06-06T03:24:38.807 INFO:teuthology.orchestra.run.smithi040.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-06-06T03:24:38.814 INFO:teuthology.orchestra.run.smithi131.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-06T03:24:38.821 INFO:teuthology.orchestra.run.smithi104.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-06-06T03:24:38.839 INFO:teuthology.orchestra.run.smithi131.stdout:(142/159): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-06-06T03:24:38.866 INFO:teuthology.orchestra.run.smithi040.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-06-06T03:24:38.872 INFO:teuthology.orchestra.run.smithi131.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-06-06T03:24:38.880 INFO:teuthology.orchestra.run.smithi104.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-06T03:24:38.906 INFO:teuthology.orchestra.run.smithi131.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 600 kB/s | 20 kB 00:00 2024-06-06T03:24:38.940 INFO:teuthology.orchestra.run.smithi131.stdout:(145/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-06-06T03:24:38.974 INFO:teuthology.orchestra.run.smithi040.stdout:(73/159): postgresql-13.14-1.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2024-06-06T03:24:38.982 INFO:teuthology.orchestra.run.smithi131.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-06-06T03:24:39.013 INFO:teuthology.orchestra.run.smithi104.stdout:(70/159): postgresql-13.14-1.el9.x86_64.rpm 4.4 MB/s | 1.6 MB 00:00 2024-06-06T03:24:39.032 INFO:teuthology.orchestra.run.smithi131.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.2 MB/s | 427 kB 00:00 2024-06-06T03:24:39.083 INFO:teuthology.orchestra.run.smithi040.stdout:(74/159): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2024-06-06T03:24:39.108 INFO:teuthology.orchestra.run.smithi040.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 459 kB/s | 61 kB 00:00 2024-06-06T03:24:39.122 INFO:teuthology.orchestra.run.smithi104.stdout:(71/159): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-06-06T03:24:39.142 INFO:teuthology.orchestra.run.smithi040.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-06-06T03:24:39.172 INFO:teuthology.orchestra.run.smithi104.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-06-06T03:24:39.200 INFO:teuthology.orchestra.run.smithi040.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-06-06T03:24:39.231 INFO:teuthology.orchestra.run.smithi104.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 4.6 MB/s | 1.0 MB 00:00 2024-06-06T03:24:39.259 INFO:teuthology.orchestra.run.smithi040.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-06-06T03:24:39.281 INFO:teuthology.orchestra.run.smithi104.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-06-06T03:24:39.284 INFO:teuthology.orchestra.run.smithi040.stdout:(79/159): python3-jinja2-2.11.3-6.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-06-06T03:24:39.318 INFO:teuthology.orchestra.run.smithi040.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-06-06T03:24:39.344 INFO:teuthology.orchestra.run.smithi040.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-06-06T03:24:39.373 INFO:teuthology.orchestra.run.smithi104.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-06-06T03:24:39.391 INFO:teuthology.orchestra.run.smithi131.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2024-06-06T03:24:39.398 INFO:teuthology.orchestra.run.smithi104.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.0 MB/s | 244 kB 00:00 2024-06-06T03:24:39.424 INFO:teuthology.orchestra.run.smithi104.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-06-06T03:24:39.441 INFO:teuthology.orchestra.run.smithi104.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 615 kB/s | 26 kB 00:00 2024-06-06T03:24:39.458 INFO:teuthology.orchestra.run.smithi104.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 574 kB/s | 19 kB 00:00 2024-06-06T03:24:39.494 INFO:teuthology.orchestra.run.smithi040.stdout:(82/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-06-06T03:24:39.499 INFO:teuthology.orchestra.run.smithi104.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-06-06T03:24:39.516 INFO:teuthology.orchestra.run.smithi104.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-06-06T03:24:39.533 INFO:teuthology.orchestra.run.smithi131.stdout:(149/159): librados2-19.0.0-2946.g3d89accf.el9. 6.2 MB/s | 3.4 MB 00:00 2024-06-06T03:24:39.553 INFO:teuthology.orchestra.run.smithi040.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-06-06T03:24:39.578 INFO:teuthology.orchestra.run.smithi040.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-06-06T03:24:39.583 INFO:teuthology.orchestra.run.smithi131.stdout:(150/159): NetworkManager-1.47.91-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-06-06T03:24:39.600 INFO:teuthology.orchestra.run.smithi104.stdout:(82/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-06-06T03:24:39.608 INFO:teuthology.orchestra.run.smithi131.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 1.6 MB/s | 41 kB 00:00 2024-06-06T03:24:39.620 INFO:teuthology.orchestra.run.smithi040.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-06-06T03:24:39.625 INFO:teuthology.orchestra.run.smithi104.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-06-06T03:24:39.645 INFO:teuthology.orchestra.run.smithi040.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 631 kB/s | 42 kB 00:00 2024-06-06T03:24:39.650 INFO:teuthology.orchestra.run.smithi131.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-06-06T03:24:39.651 INFO:teuthology.orchestra.run.smithi104.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-06-06T03:24:39.667 INFO:teuthology.orchestra.run.smithi104.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-06-06T03:24:39.675 INFO:teuthology.orchestra.run.smithi131.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 1.4 MB/s | 37 kB 00:00 2024-06-06T03:24:39.693 INFO:teuthology.orchestra.run.smithi104.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-06-06T03:24:39.712 INFO:teuthology.orchestra.run.smithi040.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-06-06T03:24:39.717 INFO:teuthology.orchestra.run.smithi131.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.9 MB/s | 207 kB 00:00 2024-06-06T03:24:39.718 INFO:teuthology.orchestra.run.smithi104.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-06-06T03:24:39.759 INFO:teuthology.orchestra.run.smithi131.stdout:(155/159): NetworkManager-libnm-1.47.91-1.el9.x 8.0 MB/s | 1.8 MB 00:00 2024-06-06T03:24:39.760 INFO:teuthology.orchestra.run.smithi104.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-06-06T03:24:39.785 INFO:teuthology.orchestra.run.smithi131.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.8 MB/s | 260 kB 00:00 2024-06-06T03:24:39.794 INFO:teuthology.orchestra.run.smithi104.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-06-06T03:24:39.796 INFO:teuthology.orchestra.run.smithi040.stdout:(88/159): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-06-06T03:24:39.819 INFO:teuthology.orchestra.run.smithi104.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 873 kB/s | 51 kB 00:00 2024-06-06T03:24:39.821 INFO:teuthology.orchestra.run.smithi040.stdout:(89/159): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-06-06T03:24:39.843 INFO:teuthology.orchestra.run.smithi131.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-06-06T03:24:39.854 INFO:teuthology.orchestra.run.smithi040.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 869 kB/s | 51 kB 00:00 2024-06-06T03:24:39.880 INFO:teuthology.orchestra.run.smithi040.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 915 kB/s | 54 kB 00:00 2024-06-06T03:24:39.893 INFO:teuthology.orchestra.run.smithi131.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-06-06T03:24:39.939 INFO:teuthology.orchestra.run.smithi040.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-06-06T03:24:39.997 INFO:teuthology.orchestra.run.smithi040.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 761 kB/s | 44 kB 00:00 2024-06-06T03:24:40.044 INFO:teuthology.orchestra.run.smithi104.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 214 kB/s | 54 kB 00:00 2024-06-06T03:24:40.278 INFO:teuthology.orchestra.run.smithi104.stdout:(92/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.6 MB/s | 5.1 MB 00:00 2024-06-06T03:24:40.303 INFO:teuthology.orchestra.run.smithi104.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 161 kB/s | 42 kB 00:00 2024-06-06T03:24:40.329 INFO:teuthology.orchestra.run.smithi104.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-06-06T03:24:40.370 INFO:teuthology.orchestra.run.smithi104.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 867 kB/s | 36 kB 00:00 2024-06-06T03:24:40.381 INFO:teuthology.orchestra.run.smithi040.stdout:(94/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-06-06T03:24:40.406 INFO:teuthology.orchestra.run.smithi040.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 540 kB/s | 221 kB 00:00 2024-06-06T03:24:40.412 INFO:teuthology.orchestra.run.smithi104.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-06-06T03:24:40.440 INFO:teuthology.orchestra.run.smithi040.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-06-06T03:24:40.446 INFO:teuthology.orchestra.run.smithi104.stdout:(97/159): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-06-06T03:24:40.473 INFO:teuthology.orchestra.run.smithi040.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-06-06T03:24:40.488 INFO:teuthology.orchestra.run.smithi104.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-06-06T03:24:40.493 INFO:teuthology.orchestra.run.smithi131.stdout:(159/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 2.1 MB/s | 3.1 MB 00:01 2024-06-06T03:24:40.499 INFO:teuthology.orchestra.run.smithi040.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-06-06T03:24:40.500 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-06-06T03:24:40.500 INFO:teuthology.orchestra.run.smithi131.stdout:Total 12 MB/s | 225 MB 00:18 2024-06-06T03:24:40.522 INFO:teuthology.orchestra.run.smithi104.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.8 MB/s | 94 kB 00:00 2024-06-06T03:24:40.541 INFO:teuthology.orchestra.run.smithi040.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 863 kB/s | 58 kB 00:00 2024-06-06T03:24:40.566 INFO:teuthology.orchestra.run.smithi040.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-06-06T03:24:40.572 INFO:teuthology.orchestra.run.smithi104.stdout:(100/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-06T03:24:40.614 INFO:teuthology.orchestra.run.smithi104.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-06-06T03:24:40.639 INFO:teuthology.orchestra.run.smithi104.stdout:(102/159): uuid-1.6.2-55.el9.x86_64.rpm 300 kB/s | 58 kB 00:00 2024-06-06T03:24:40.639 INFO:teuthology.orchestra.run.smithi104.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:40.642 INFO:teuthology.orchestra.run.smithi104.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:40.681 INFO:teuthology.orchestra.run.smithi104.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 334 kB/s | 22 kB 00:00 2024-06-06T03:24:40.865 INFO:teuthology.orchestra.run.smithi104.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-06-06T03:24:40.983 INFO:teuthology.orchestra.run.smithi040.stdout:(101/159): ceph-test-19.0.0-2946.g3d89accf.el9. 4.6 MB/s | 48 MB 00:10 2024-06-06T03:24:40.998 INFO:teuthology.orchestra.run.smithi104.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-06-06T03:24:41.008 INFO:teuthology.orchestra.run.smithi040.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 101 kB/s | 45 kB 00:00 2024-06-06T03:24:41.009 INFO:teuthology.orchestra.run.smithi040.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:41.024 INFO:teuthology.orchestra.run.smithi104.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.9 MB/s | 309 kB 00:00 2024-06-06T03:24:41.033 INFO:teuthology.orchestra.run.smithi040.stdout:(103/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 129 kB/s | 64 kB 00:00 2024-06-06T03:24:41.034 INFO:teuthology.orchestra.run.smithi040.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:24:41.049 INFO:teuthology.orchestra.run.smithi104.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-06-06T03:24:41.082 INFO:teuthology.orchestra.run.smithi104.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 785 kB/s | 26 kB 00:00 2024-06-06T03:24:41.109 INFO:teuthology.orchestra.run.smithi040.stdout:(104/159): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-06-06T03:24:41.116 INFO:teuthology.orchestra.run.smithi104.stdout:(109/159): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-06-06T03:24:41.141 INFO:teuthology.orchestra.run.smithi104.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 829 kB/s | 49 kB 00:00 2024-06-06T03:24:41.184 INFO:teuthology.orchestra.run.smithi040.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 3.6 MB/s | 548 kB 00:00 2024-06-06T03:24:41.217 INFO:teuthology.orchestra.run.smithi040.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 587 kB/s | 19 kB 00:00 2024-06-06T03:24:41.233 INFO:teuthology.orchestra.run.smithi104.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-06-06T03:24:41.258 INFO:teuthology.orchestra.run.smithi104.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.7 MB/s | 151 kB 00:00 2024-06-06T03:24:41.260 INFO:teuthology.orchestra.run.smithi040.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-06-06T03:24:41.285 INFO:teuthology.orchestra.run.smithi040.stdout:(108/159): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-06-06T03:24:41.310 INFO:teuthology.orchestra.run.smithi040.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 516 kB/s | 26 kB 00:00 2024-06-06T03:24:41.386 INFO:teuthology.orchestra.run.smithi040.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-06-06T03:24:41.411 INFO:teuthology.orchestra.run.smithi040.stdout:(111/159): liboath-2.6.7-2.el9.x86_64.rpm 386 kB/s | 49 kB 00:00 2024-06-06T03:24:41.425 INFO:teuthology.orchestra.run.smithi104.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-06-06T03:24:41.444 INFO:teuthology.orchestra.run.smithi040.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-06-06T03:24:41.467 INFO:teuthology.orchestra.run.smithi104.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-06-06T03:24:41.501 INFO:teuthology.orchestra.run.smithi104.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 967 kB/s | 32 kB 00:00 2024-06-06T03:24:41.535 INFO:teuthology.orchestra.run.smithi104.stdout:(116/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.7 MB/s | 839 kB 00:00 2024-06-06T03:24:41.560 INFO:teuthology.orchestra.run.smithi104.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-06-06T03:24:41.595 INFO:teuthology.orchestra.run.smithi040.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-06-06T03:24:41.610 INFO:teuthology.orchestra.run.smithi104.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-06-06T03:24:41.628 INFO:teuthology.orchestra.run.smithi040.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-06-06T03:24:41.654 INFO:teuthology.orchestra.run.smithi040.stdout:(115/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-06-06T03:24:41.677 INFO:teuthology.orchestra.run.smithi104.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-06-06T03:24:41.679 INFO:teuthology.orchestra.run.smithi040.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-06-06T03:24:41.702 INFO:teuthology.orchestra.run.smithi104.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-06-06T03:24:41.704 INFO:teuthology.orchestra.run.smithi040.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-06-06T03:24:41.729 INFO:teuthology.orchestra.run.smithi040.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-06-06T03:24:41.734 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-06-06T03:24:41.736 INFO:teuthology.orchestra.run.smithi104.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-06-06T03:24:41.772 INFO:teuthology.orchestra.run.smithi040.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-06-06T03:24:41.794 INFO:teuthology.orchestra.run.smithi104.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-06-06T03:24:41.805 INFO:teuthology.orchestra.run.smithi040.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-06-06T03:24:41.819 INFO:teuthology.orchestra.run.smithi104.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-06-06T03:24:41.839 INFO:teuthology.orchestra.run.smithi040.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-06-06T03:24:41.845 INFO:teuthology.orchestra.run.smithi104.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-06-06T03:24:41.879 INFO:teuthology.orchestra.run.smithi104.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 695 kB/s | 23 kB 00:00 2024-06-06T03:24:41.889 INFO:teuthology.orchestra.run.smithi040.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-06-06T03:24:41.894 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-06-06T03:24:41.894 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-06-06T03:24:41.912 INFO:teuthology.orchestra.run.smithi104.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 586 kB/s | 19 kB 00:00 2024-06-06T03:24:41.914 INFO:teuthology.orchestra.run.smithi040.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-06-06T03:24:41.946 INFO:teuthology.orchestra.run.smithi104.stdout:(127/159): python3-jaraco-text-3.2.0-6.el9.noar 590 kB/s | 20 kB 00:00 2024-06-06T03:24:41.948 INFO:teuthology.orchestra.run.smithi040.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 532 kB/s | 18 kB 00:00 2024-06-06T03:24:41.974 INFO:teuthology.orchestra.run.smithi040.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-06-06T03:24:42.007 INFO:teuthology.orchestra.run.smithi040.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 584 kB/s | 19 kB 00:00 2024-06-06T03:24:42.032 INFO:teuthology.orchestra.run.smithi040.stdout:(127/159): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-06-06T03:24:42.213 INFO:teuthology.orchestra.run.smithi104.stdout:(128/159): python3-kubernetes-26.1.0-2.el9.noar 3.8 MB/s | 1.0 MB 00:00 2024-06-06T03:24:42.271 INFO:teuthology.orchestra.run.smithi104.stdout:(129/159): libarrow-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-06-06T03:24:42.297 INFO:teuthology.orchestra.run.smithi104.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-06-06T03:24:42.333 INFO:teuthology.orchestra.run.smithi040.stdout:(128/159): python3-kubernetes-26.1.0-2.el9.noar 3.4 MB/s | 1.0 MB 00:00 2024-06-06T03:24:42.339 INFO:teuthology.orchestra.run.smithi104.stdout:(131/159): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-06-06T03:24:42.350 INFO:teuthology.orchestra.run.smithi040.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 2.8 MB/s | 46 kB 00:00 2024-06-06T03:24:42.364 INFO:teuthology.orchestra.run.smithi104.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 864 kB/s | 58 kB 00:00 2024-06-06T03:24:42.389 INFO:teuthology.orchestra.run.smithi104.stdout:(133/159): python3-more-itertools-8.12.0-2.el9. 671 kB/s | 79 kB 00:00 2024-06-06T03:24:42.391 INFO:teuthology.orchestra.run.smithi040.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-06-06T03:24:42.414 INFO:teuthology.orchestra.run.smithi104.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-06-06T03:24:42.434 INFO:teuthology.orchestra.run.smithi040.stdout:(131/159): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-06-06T03:24:42.448 INFO:teuthology.orchestra.run.smithi104.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-06-06T03:24:42.450 INFO:teuthology.orchestra.run.smithi040.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-06-06T03:24:42.468 INFO:teuthology.orchestra.run.smithi040.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 983 kB/s | 16 kB 00:00 2024-06-06T03:24:42.473 INFO:teuthology.orchestra.run.smithi104.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 480 kB/s | 28 kB 00:00 2024-06-06T03:24:42.499 INFO:teuthology.orchestra.run.smithi104.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-06-06T03:24:42.501 INFO:teuthology.orchestra.run.smithi040.stdout:(134/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.6 MB/s | 90 kB 00:00 2024-06-06T03:24:42.524 INFO:teuthology.orchestra.run.smithi104.stdout:(138/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-06-06T03:24:42.549 INFO:teuthology.orchestra.run.smithi104.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-06-06T03:24:42.560 INFO:teuthology.orchestra.run.smithi040.stdout:(135/159): libarrow-9.0.0-11.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-06-06T03:24:42.566 INFO:teuthology.orchestra.run.smithi104.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 857 kB/s | 36 kB 00:00 2024-06-06T03:24:42.583 INFO:teuthology.orchestra.run.smithi104.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.5 MB/s | 51 kB 00:00 2024-06-06T03:24:42.585 INFO:teuthology.orchestra.run.smithi040.stdout:(136/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-06-06T03:24:42.608 INFO:teuthology.orchestra.run.smithi104.stdout:(142/159): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-06-06T03:24:42.610 INFO:teuthology.orchestra.run.smithi040.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-06-06T03:24:42.636 INFO:teuthology.orchestra.run.smithi040.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-06-06T03:24:42.642 INFO:teuthology.orchestra.run.smithi104.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-06-06T03:24:42.661 INFO:teuthology.orchestra.run.smithi040.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-06-06T03:24:42.667 INFO:teuthology.orchestra.run.smithi104.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-06-06T03:24:42.686 INFO:teuthology.orchestra.run.smithi040.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-06-06T03:24:42.692 INFO:teuthology.orchestra.run.smithi104.stdout:(145/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-06-06T03:24:42.712 INFO:teuthology.orchestra.run.smithi040.stdout:(141/159): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-06-06T03:24:42.737 INFO:teuthology.orchestra.run.smithi040.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-06-06T03:24:42.751 INFO:teuthology.orchestra.run.smithi104.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-06-06T03:24:42.770 INFO:teuthology.orchestra.run.smithi040.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-06-06T03:24:42.796 INFO:teuthology.orchestra.run.smithi040.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-06-06T03:24:42.801 INFO:teuthology.orchestra.run.smithi104.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.2 MB/s | 427 kB 00:00 2024-06-06T03:24:42.821 INFO:teuthology.orchestra.run.smithi040.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-06-06T03:24:42.888 INFO:teuthology.orchestra.run.smithi040.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-06-06T03:24:42.947 INFO:teuthology.orchestra.run.smithi040.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.0 MB/s | 427 kB 00:00 2024-06-06T03:24:43.093 INFO:teuthology.orchestra.run.smithi104.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-06-06T03:24:43.230 INFO:teuthology.orchestra.run.smithi040.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-06-06T03:24:43.260 INFO:teuthology.orchestra.run.smithi104.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 14 MB/s | 2.3 MB 00:00 2024-06-06T03:24:43.327 INFO:teuthology.orchestra.run.smithi104.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-06-06T03:24:43.361 INFO:teuthology.orchestra.run.smithi104.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-06-06T03:24:43.389 INFO:teuthology.orchestra.run.smithi040.stdout:(149/159): librados2-19.0.0-2946.g3d89accf.el9. 6.8 MB/s | 3.4 MB 00:00 2024-06-06T03:24:43.411 INFO:teuthology.orchestra.run.smithi104.stdout:(152/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 5.1 MB/s | 3.1 MB 00:00 2024-06-06T03:24:43.431 INFO:teuthology.orchestra.run.smithi040.stdout:(150/159): NetworkManager-1.47.91-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-06-06T03:24:43.436 INFO:teuthology.orchestra.run.smithi104.stdout:(153/159): NetworkManager-tui-1.47.91-1.el9.x86 3.2 MB/s | 248 kB 00:00 2024-06-06T03:24:43.462 INFO:teuthology.orchestra.run.smithi104.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 731 kB/s | 37 kB 00:00 2024-06-06T03:24:43.464 INFO:teuthology.orchestra.run.smithi040.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-06-06T03:24:43.487 INFO:teuthology.orchestra.run.smithi104.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.0 MB/s | 207 kB 00:00 2024-06-06T03:24:43.515 INFO:teuthology.orchestra.run.smithi040.stdout:(152/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 5.5 MB/s | 3.1 MB 00:00 2024-06-06T03:24:43.540 INFO:teuthology.orchestra.run.smithi040.stdout:(153/159): NetworkManager-tui-1.47.91-1.el9.x86 3.2 MB/s | 248 kB 00:00 2024-06-06T03:24:43.545 INFO:teuthology.orchestra.run.smithi104.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 23 MB/s | 1.3 MB 00:00 2024-06-06T03:24:43.571 INFO:teuthology.orchestra.run.smithi104.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.3 MB/s | 260 kB 00:00 2024-06-06T03:24:43.582 INFO:teuthology.orchestra.run.smithi040.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.9 MB/s | 207 kB 00:00 2024-06-06T03:24:43.607 INFO:teuthology.orchestra.run.smithi040.stdout:(155/159): elfutils-debuginfod-client-0.191-4.e 398 kB/s | 37 kB 00:00 2024-06-06T03:24:43.621 INFO:teuthology.orchestra.run.smithi104.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-06-06T03:24:43.633 INFO:teuthology.orchestra.run.smithi040.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 5.0 MB/s | 260 kB 00:00 2024-06-06T03:24:43.674 INFO:teuthology.orchestra.run.smithi040.stdout:(157/159): NetworkManager-libnm-1.47.91-1.el9.x 6.3 MB/s | 1.8 MB 00:00 2024-06-06T03:24:43.705 INFO:teuthology.orchestra.run.smithi104.stdout:(159/159): librados2-19.0.0-2946.g3d89accf.el9. 3.6 MB/s | 3.4 MB 00:00 2024-06-06T03:24:43.710 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-06-06T03:24:43.711 INFO:teuthology.orchestra.run.smithi104.stdout:Total 12 MB/s | 225 MB 00:19 2024-06-06T03:24:43.741 INFO:teuthology.orchestra.run.smithi040.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-06-06T03:24:43.776 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-06-06T03:24:43.777 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-06-06T03:24:43.783 INFO:teuthology.orchestra.run.smithi040.stdout:(159/159): openssl-3.2.1-1.el9.x86_64.rpm 7.7 MB/s | 1.3 MB 00:00 2024-06-06T03:24:43.790 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-06-06T03:24:43.790 INFO:teuthology.orchestra.run.smithi040.stdout:Total 11 MB/s | 225 MB 00:20 2024-06-06T03:24:44.922 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-06-06T03:24:45.076 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-06-06T03:24:45.077 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-06-06T03:24:45.077 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-06-06T03:24:45.256 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-06-06T03:24:45.256 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-06-06T03:24:46.863 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-06-06T03:24:46.863 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-06-06T03:24:47.482 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-06-06T03:24:47.483 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-06-06T03:24:49.997 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-06-06T03:24:50.073 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T03:24:50.111 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T03:24:50.172 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T03:24:50.206 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T03:24:50.232 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T03:24:50.275 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T03:24:50.597 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T03:24:50.716 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T03:24:50.839 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T03:24:50.867 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T03:24:50.959 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T03:24:50.997 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T03:24:51.029 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T03:24:51.203 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T03:24:51.253 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T03:24:51.281 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T03:24:51.385 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T03:24:51.407 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T03:24:51.470 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T03:24:51.493 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T03:24:51.510 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T03:24:51.659 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T03:24:51.789 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T03:24:51.964 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T03:24:52.015 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T03:24:52.109 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T03:24:52.193 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T03:24:52.406 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T03:24:52.529 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T03:24:52.620 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T03:24:52.649 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T03:24:52.692 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T03:24:52.765 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T03:24:52.799 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T03:24:52.843 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T03:24:52.875 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T03:24:52.903 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T03:24:52.972 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T03:24:53.026 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T03:24:53.064 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T03:24:53.100 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T03:24:53.176 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T03:24:53.197 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-06-06T03:24:53.211 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T03:24:53.254 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T03:24:53.275 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T03:24:53.282 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T03:24:53.313 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T03:24:53.352 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T03:24:53.366 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T03:24:53.397 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T03:24:53.400 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T03:24:53.426 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T03:24:53.468 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T03:24:53.472 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T03:24:53.604 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T03:24:53.658 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T03:24:53.808 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T03:24:53.935 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T03:24:54.056 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T03:24:54.077 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T03:24:54.168 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T03:24:54.173 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T03:24:54.198 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T03:24:54.230 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-06-06T03:24:54.231 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T03:24:54.255 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T03:24:54.282 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T03:24:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T03:24:54.318 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T03:24:54.343 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T03:24:54.376 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T03:24:54.410 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T03:24:54.422 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T03:24:54.424 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T03:24:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T03:24:54.463 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T03:24:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T03:24:54.488 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T03:24:54.516 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T03:24:54.528 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T03:24:54.613 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T03:24:54.619 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T03:24:54.642 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T03:24:54.667 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T03:24:54.696 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T03:24:54.718 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T03:24:54.720 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T03:24:54.743 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T03:24:54.787 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T03:24:54.817 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T03:24:54.854 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T03:24:54.876 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T03:24:54.901 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T03:24:54.926 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T03:24:54.936 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T03:24:54.979 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T03:24:54.984 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T03:24:55.030 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T03:24:55.062 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T03:24:55.126 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T03:24:55.152 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T03:24:55.167 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T03:24:55.207 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T03:24:55.243 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T03:24:55.248 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T03:24:55.251 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T03:24:55.282 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T03:24:55.292 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T03:24:55.332 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T03:24:55.343 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T03:24:55.382 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T03:24:55.400 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T03:24:55.421 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T03:24:55.447 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T03:24:55.468 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T03:24:55.472 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T03:24:55.531 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T03:24:55.547 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T03:24:55.589 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T03:24:55.616 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T03:24:55.666 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T03:24:55.672 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T03:24:55.704 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T03:24:55.705 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T03:24:55.732 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T03:24:55.733 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T03:24:55.774 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T03:24:55.778 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T03:24:55.814 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T03:24:55.814 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T03:24:55.835 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T03:24:55.845 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T03:24:55.859 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T03:24:55.871 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T03:24:55.875 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T03:24:55.909 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T03:24:55.915 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T03:24:55.934 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T03:24:55.959 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T03:24:55.968 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T03:24:56.002 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T03:24:56.022 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T03:24:56.033 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T03:24:56.066 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T03:24:56.089 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T03:24:56.110 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T03:24:56.142 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T03:24:56.170 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T03:24:56.176 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T03:24:56.239 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T03:24:56.293 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T03:24:56.323 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T03:24:56.333 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T03:24:56.358 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T03:24:56.365 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T03:24:56.376 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T03:24:56.435 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T03:24:56.469 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T03:24:56.481 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T03:24:56.512 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T03:24:56.532 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T03:24:56.541 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T03:24:56.553 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T03:24:56.611 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T03:24:56.664 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T03:24:56.681 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T03:24:56.738 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T03:24:56.757 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T03:24:56.869 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T03:24:56.873 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T03:24:56.925 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T03:24:56.950 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T03:24:56.966 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T03:24:57.008 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T03:24:57.025 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T03:24:57.061 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T03:24:57.136 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T03:24:57.192 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T03:24:57.236 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T03:24:57.270 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T03:24:57.296 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T03:24:57.369 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T03:24:57.420 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T03:24:57.450 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T03:24:57.455 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T03:24:57.494 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T03:24:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T03:24:57.566 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T03:24:57.572 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T03:24:57.602 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T03:24:57.604 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T03:24:57.648 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T03:24:57.651 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T03:24:57.676 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T03:24:57.691 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T03:24:57.727 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T03:24:57.732 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T03:24:57.749 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T03:24:57.771 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T03:24:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T03:24:57.808 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T03:24:57.869 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T03:24:57.885 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T03:24:57.907 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T03:24:57.952 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T03:24:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T03:24:58.007 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T03:24:58.013 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T03:24:58.014 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T03:24:58.070 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T03:24:58.080 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T03:24:58.100 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T03:24:58.111 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T03:24:58.132 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T03:24:58.148 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T03:24:58.194 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T03:24:58.230 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T03:24:58.301 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T03:24:58.310 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T03:24:58.338 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T03:24:58.357 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T03:24:58.416 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T03:24:58.494 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T03:24:58.627 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T03:24:58.627 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T03:24:58.692 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T03:24:58.716 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T03:24:58.751 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T03:24:58.787 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T03:24:58.788 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T03:24:58.825 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T03:24:58.829 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T03:24:58.865 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T03:24:58.866 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T03:24:58.891 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T03:24:58.908 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T03:24:58.915 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T03:24:58.990 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T03:24:59.062 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T03:24:59.076 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T03:24:59.119 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T03:24:59.128 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T03:24:59.156 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T03:24:59.178 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T03:24:59.184 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T03:24:59.184 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T03:24:59.218 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T03:24:59.257 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T03:24:59.267 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T03:24:59.289 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T03:24:59.303 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T03:24:59.319 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T03:24:59.340 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T03:24:59.353 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T03:24:59.386 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T03:24:59.388 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T03:24:59.423 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T03:24:59.429 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T03:24:59.463 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T03:24:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T03:24:59.508 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T03:24:59.550 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T03:24:59.593 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T03:24:59.624 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T03:24:59.654 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T03:24:59.671 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T03:24:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T03:24:59.744 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T03:24:59.770 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T03:24:59.817 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T03:24:59.842 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T03:24:59.876 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T03:24:59.918 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T03:24:59.950 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T03:24:59.974 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T03:25:00.067 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T03:25:00.199 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T03:25:00.232 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T03:25:00.261 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T03:25:00.304 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T03:25:00.343 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T03:25:00.383 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T03:25:00.405 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T03:25:00.412 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T03:25:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T03:25:00.480 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T03:25:00.492 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T03:25:00.520 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T03:25:00.522 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T03:25:00.547 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T03:25:00.553 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T03:25:00.556 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T03:25:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T03:25:00.640 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T03:25:00.657 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T03:25:00.712 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T03:25:00.742 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T03:25:00.751 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T03:25:00.795 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T03:25:00.799 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T03:25:00.839 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T03:25:00.862 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T03:25:00.907 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T03:25:00.910 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T03:25:00.927 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T03:25:00.959 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T03:25:00.962 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T03:25:00.997 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T03:25:01.028 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T03:25:01.065 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T03:25:01.138 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T03:25:01.181 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T03:25:01.259 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T03:25:01.534 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T03:25:01.560 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T03:25:01.640 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T03:25:01.670 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T03:25:01.726 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T03:25:01.783 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T03:25:01.817 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T03:25:01.904 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T03:25:01.919 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T03:25:01.975 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T03:25:02.021 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T03:25:02.028 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T03:25:02.118 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T03:25:02.155 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T03:25:02.339 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T03:25:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T03:25:02.456 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T03:25:03.149 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T03:25:03.249 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T03:25:03.271 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T03:25:03.335 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T03:25:03.363 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T03:25:03.396 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T03:25:03.716 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T03:25:03.767 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T03:25:03.842 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T03:25:03.872 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T03:25:03.900 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T03:25:03.908 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T03:25:03.933 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T03:25:04.379 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T03:25:04.403 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T03:25:04.471 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T03:25:04.475 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T03:25:04.505 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T03:25:04.511 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T03:25:04.585 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T03:25:04.585 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T03:25:04.585 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T03:25:04.586 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:25:04.612 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T03:25:04.630 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T03:25:04.723 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T03:25:04.751 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T03:25:04.784 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T03:25:05.113 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T03:25:05.155 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T03:25:05.231 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T03:25:05.260 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T03:25:05.297 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T03:25:05.795 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T03:25:05.816 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T03:25:05.888 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T03:25:05.918 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T03:25:07.155 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T03:25:07.185 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T03:25:07.727 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T03:25:07.763 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T03:25:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T03:25:07.854 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T03:25:07.854 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T03:25:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:25:07.880 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T03:25:08.311 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T03:25:08.348 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T03:25:08.923 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T03:25:08.959 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T03:25:09.051 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T03:25:09.051 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T03:25:09.052 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T03:25:09.052 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:25:09.086 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T03:25:16.263 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T03:25:16.263 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2024-06-06T03:25:16.263 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2024-06-06T03:25:16.263 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2024-06-06T03:25:16.263 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2024-06-06T03:25:16.263 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2024-06-06T03:25:16.263 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /root 2024-06-06T03:25:16.263 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2024-06-06T03:25:16.263 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:25:16.438 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T03:25:16.454 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T03:25:16.454 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:16.454 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T03:25:16.454 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:25:17.959 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T03:25:17.985 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T03:25:17.985 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:17.985 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T03:25:17.985 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T03:25:17.985 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T03:25:17.985 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:25:18.218 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T03:25:18.241 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T03:25:18.241 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:18.241 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T03:25:18.241 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T03:25:18.241 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T03:25:18.242 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:25:18.685 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T03:25:18.708 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T03:25:18.708 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:18.708 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T03:25:18.708 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T03:25:18.708 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T03:25:18.709 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:25:18.816 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T03:25:18.864 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T03:25:18.920 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T03:25:19.105 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T03:25:19.153 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T03:25:19.239 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T03:25:19.327 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T03:25:19.327 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /sys 2024-06-06T03:25:19.327 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /proc 2024-06-06T03:25:19.328 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /mnt 2024-06-06T03:25:19.328 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /var/tmp 2024-06-06T03:25:19.328 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /home 2024-06-06T03:25:19.328 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /root 2024-06-06T03:25:19.328 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /tmp 2024-06-06T03:25:19.328 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:25:19.328 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T03:25:19.458 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T03:25:19.490 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T03:25:19.503 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T03:25:19.505 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T03:25:19.505 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:19.505 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T03:25:19.505 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:25:19.535 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T03:25:19.563 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T03:25:19.999 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T03:25:20.028 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T03:25:20.059 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T03:25:20.121 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T03:25:20.151 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T03:25:20.992 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T03:25:21.018 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T03:25:21.018 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:21.018 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T03:25:21.019 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T03:25:21.019 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T03:25:21.019 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:25:21.086 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T03:25:21.087 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2024-06-06T03:25:21.087 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2024-06-06T03:25:21.087 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2024-06-06T03:25:21.087 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2024-06-06T03:25:21.087 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2024-06-06T03:25:21.087 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /root 2024-06-06T03:25:21.087 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2024-06-06T03:25:21.087 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:25:21.179 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T03:25:21.220 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T03:25:21.252 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T03:25:21.260 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T03:25:21.275 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T03:25:21.275 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:21.275 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T03:25:21.275 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T03:25:21.275 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T03:25:21.275 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:25:21.277 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T03:25:21.277 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:21.277 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T03:25:21.277 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:25:21.720 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T03:25:21.743 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T03:25:21.743 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:21.743 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T03:25:21.743 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T03:25:21.743 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T03:25:21.743 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:25:21.850 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T03:25:21.915 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T03:25:21.971 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T03:25:22.157 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T03:25:22.186 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T03:25:22.204 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T03:25:22.220 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T03:25:22.281 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T03:25:22.323 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T03:25:22.380 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T03:25:22.484 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T03:25:22.514 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T03:25:22.518 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T03:25:22.537 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T03:25:22.537 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:22.537 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T03:25:22.538 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T03:25:22.538 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T03:25:22.538 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:25:22.570 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T03:25:22.581 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T03:25:22.593 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T03:25:22.602 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T03:25:22.638 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T03:25:22.690 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T03:25:22.747 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T03:25:22.813 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T03:25:22.841 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T03:25:22.842 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:22.842 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T03:25:22.842 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T03:25:22.842 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T03:25:22.842 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:25:22.965 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T03:25:22.995 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T03:25:23.026 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T03:25:23.080 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T03:25:23.089 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T03:25:23.105 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T03:25:23.106 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:23.106 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T03:25:23.106 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T03:25:23.106 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T03:25:23.106 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:25:23.121 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T03:25:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T03:25:23.606 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T03:25:23.606 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:23.606 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T03:25:23.606 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T03:25:23.606 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T03:25:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:25:23.641 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T03:25:23.667 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T03:25:23.690 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T03:25:23.690 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:23.691 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T03:25:23.691 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T03:25:23.691 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T03:25:23.691 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:25:23.720 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T03:25:23.735 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T03:25:23.757 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T03:25:23.758 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:23.758 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T03:25:23.758 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:25:23.779 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T03:25:23.803 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T03:25:23.825 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T03:25:23.825 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:23.825 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T03:25:23.825 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T03:25:23.825 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T03:25:23.826 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:25:23.868 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T03:25:24.072 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T03:25:24.124 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T03:25:24.128 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T03:25:24.149 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T03:25:24.150 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T03:25:24.150 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:24.150 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T03:25:24.150 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T03:25:24.150 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T03:25:24.151 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:25:24.186 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T03:25:24.206 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T03:25:24.296 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T03:25:24.454 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T03:25:24.507 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T03:25:24.539 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T03:25:24.566 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T03:25:25.014 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T03:25:25.041 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T03:25:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T03:25:25.145 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T03:25:25.155 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T03:25:25.186 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T03:25:25.188 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T03:25:25.288 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T03:25:25.449 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T03:25:25.472 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T03:25:25.495 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T03:25:25.496 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:25.496 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T03:25:25.496 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T03:25:25.496 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T03:25:25.496 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:25:25.547 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T03:25:25.559 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T03:25:25.649 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T03:25:25.695 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T03:25:26.280 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T03:25:26.315 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T03:25:26.599 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T03:25:26.633 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T03:25:26.656 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T03:25:26.656 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:26.656 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T03:25:26.656 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T03:25:26.656 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T03:25:26.656 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:25:26.700 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T03:25:26.722 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T03:25:26.722 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:26.723 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T03:25:26.723 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:25:26.767 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T03:25:26.789 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T03:25:26.790 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:26.790 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T03:25:26.790 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T03:25:26.790 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T03:25:26.790 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:25:27.093 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T03:25:27.116 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T03:25:27.116 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:27.116 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T03:25:27.116 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T03:25:27.116 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T03:25:27.116 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:25:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T03:25:27.366 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T03:25:27.475 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T03:25:27.639 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T03:25:27.670 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T03:25:27.695 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T03:25:27.696 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:27.696 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T03:25:27.696 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T03:25:27.696 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T03:25:27.696 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:25:27.748 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T03:25:27.761 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T03:25:27.861 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T03:25:27.919 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T03:25:28.648 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T03:25:28.689 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T03:25:28.730 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T03:25:28.762 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T03:25:28.798 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T03:25:28.873 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T03:25:28.900 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T03:25:28.925 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T03:25:28.925 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:28.925 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T03:25:28.926 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T03:25:28.926 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T03:25:28.926 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:25:28.980 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T03:25:28.984 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T03:25:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T03:25:29.009 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:29.009 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T03:25:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:25:29.027 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T03:25:29.077 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T03:25:29.117 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T03:25:29.155 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T03:25:29.177 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T03:25:29.193 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T03:25:29.202 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T03:25:29.202 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:29.202 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T03:25:29.202 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T03:25:29.202 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T03:25:29.202 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:25:29.519 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T03:25:29.544 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T03:25:29.544 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T03:25:29.544 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T03:25:29.544 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T03:25:29.544 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T03:25:29.544 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:25:29.708 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T03:25:29.750 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T03:25:29.770 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T03:25:30.604 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T03:25:30.688 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T03:25:30.730 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T03:25:30.811 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T03:25:30.850 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T03:25:30.875 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T03:25:30.909 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T03:25:30.935 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T03:25:30.935 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T03:25:31.020 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T03:25:31.053 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T03:25:31.053 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T03:25:31.121 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T03:25:31.145 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T03:25:31.151 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T03:25:31.151 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T03:25:31.331 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T03:25:31.331 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T03:25:31.415 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T03:25:31.440 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T03:25:31.504 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T03:25:31.621 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T03:25:31.671 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T03:25:31.711 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T03:25:31.743 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T03:25:31.770 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T03:25:31.955 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T03:25:31.999 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T03:25:32.031 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T03:25:32.064 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T03:25:32.094 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T03:25:32.140 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T03:25:32.752 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T03:25:32.795 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T03:25:32.815 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T03:25:33.645 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T03:25:33.724 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T03:25:33.757 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T03:25:33.847 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T03:25:33.894 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T03:25:33.919 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T03:25:33.944 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T03:25:33.971 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T03:25:33.971 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T03:25:34.055 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T03:25:34.088 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T03:25:34.088 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T03:25:34.173 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T03:25:34.198 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T03:25:34.203 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T03:25:34.203 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T03:25:34.365 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T03:25:34.383 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T03:25:34.383 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T03:25:34.406 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T03:25:34.447 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T03:25:34.459 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T03:25:34.483 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T03:25:34.483 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T03:25:34.506 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T03:25:34.548 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T03:25:34.710 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T03:25:34.752 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T03:25:34.785 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T03:25:34.825 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T03:25:34.854 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T03:25:34.918 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T03:25:35.511 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T03:25:35.549 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T03:25:35.570 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T03:25:36.491 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T03:25:36.586 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T03:25:36.629 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T03:25:36.721 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T03:25:36.757 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T03:25:36.782 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T03:25:36.807 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T03:25:36.834 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T03:25:36.834 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T03:25:36.918 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T03:25:36.959 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T03:25:36.959 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T03:25:37.036 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T03:25:37.060 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T03:25:37.080 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T03:25:37.080 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T03:25:37.194 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T03:25:37.194 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T03:25:37.194 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T03:25:37.194 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T03:25:37.194 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T03:25:37.194 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T03:25:37.194 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T03:25:37.194 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T03:25:37.194 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T03:25:37.194 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T03:25:37.194 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T03:25:37.195 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T03:25:37.195 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T03:25:37.195 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T03:25:37.195 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T03:25:37.195 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T03:25:37.195 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T03:25:37.195 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T03:25:37.195 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T03:25:37.195 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T03:25:37.195 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T03:25:37.195 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T03:25:37.195 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T03:25:37.195 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T03:25:37.196 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T03:25:37.196 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T03:25:37.196 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T03:25:37.196 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T03:25:37.196 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T03:25:37.196 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T03:25:37.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T03:25:37.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T03:25:37.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T03:25:37.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T03:25:37.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T03:25:37.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T03:25:37.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T03:25:37.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T03:25:37.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T03:25:37.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T03:25:37.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T03:25:37.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T03:25:37.209 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T03:25:37.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T03:25:37.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T03:25:37.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T03:25:37.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T03:25:37.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T03:25:37.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T03:25:37.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T03:25:37.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T03:25:37.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T03:25:37.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T03:25:37.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T03:25:37.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T03:25:37.210 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T03:25:37.211 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T03:25:37.212 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T03:25:37.213 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T03:25:37.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T03:25:37.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T03:25:37.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T03:25:37.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T03:25:37.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T03:25:37.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T03:25:37.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T03:25:37.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T03:25:37.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T03:25:37.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T03:25:37.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T03:25:37.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T03:25:37.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T03:25:37.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T03:25:37.215 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T03:25:37.216 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T03:25:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T03:25:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T03:25:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T03:25:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T03:25:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T03:25:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T03:25:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T03:25:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T03:25:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T03:25:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T03:25:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T03:25:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T03:25:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T03:25:37.217 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T03:25:37.218 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T03:25:37.218 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T03:25:37.218 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T03:25:37.218 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T03:25:37.218 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T03:25:37.218 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T03:25:37.218 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T03:25:37.218 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T03:25:37.218 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T03:25:37.218 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T03:25:37.218 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T03:25:37.270 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T03:25:37.271 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T03:25:37.355 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T03:25:37.380 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T03:25:37.449 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T03:25:39.774 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T03:25:39.774 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:25:39.774 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2024-06-06T03:25:39.774 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T03:25:39.774 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T03:25:39.774 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T03:25:39.775 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T03:25:39.775 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T03:25:39.775 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T03:25:39.775 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T03:25:39.775 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.775 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.775 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T03:25:39.775 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T03:25:39.775 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-06-06T03:25:39.775 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T03:25:39.775 INFO:teuthology.orchestra.run.smithi131.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T03:25:39.775 INFO:teuthology.orchestra.run.smithi131.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T03:25:39.775 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T03:25:39.776 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.776 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.776 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.776 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.776 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:39.776 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.776 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.776 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.776 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:39.776 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:39.776 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:39.776 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:39.776 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T03:25:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T03:25:39.777 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T03:25:39.778 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T03:25:39.778 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T03:25:39.778 INFO:teuthology.orchestra.run.smithi131.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T03:25:39.778 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T03:25:39.778 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T03:25:39.778 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T03:25:39.778 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T03:25:39.778 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T03:25:39.778 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.778 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.778 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.778 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T03:25:39.778 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T03:25:39.779 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T03:25:39.779 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T03:25:39.779 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T03:25:39.779 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.779 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.779 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T03:25:39.779 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.779 INFO:teuthology.orchestra.run.smithi131.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T03:25:39.779 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T03:25:39.779 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T03:25:39.779 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T03:25:39.779 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T03:25:39.779 INFO:teuthology.orchestra.run.smithi131.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T03:25:39.780 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T03:25:39.780 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T03:25:39.780 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T03:25:39.780 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T03:25:39.780 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T03:25:39.780 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T03:25:39.780 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T03:25:39.780 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T03:25:39.780 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T03:25:39.780 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T03:25:39.780 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T03:25:39.780 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T03:25:39.780 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T03:25:39.780 INFO:teuthology.orchestra.run.smithi131.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T03:25:39.781 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T03:25:39.781 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T03:25:39.781 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T03:25:39.781 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.781 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.781 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.781 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T03:25:39.781 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T03:25:39.781 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T03:25:39.781 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T03:25:39.781 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T03:25:39.781 INFO:teuthology.orchestra.run.smithi131.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T03:25:39.782 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T03:25:39.782 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T03:25:39.782 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T03:25:39.782 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T03:25:39.782 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T03:25:39.782 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T03:25:39.782 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T03:25:39.782 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T03:25:39.782 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T03:25:39.782 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T03:25:39.782 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T03:25:39.782 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T03:25:39.782 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T03:25:39.783 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T03:25:39.783 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T03:25:39.783 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T03:25:39.783 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T03:25:39.783 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T03:25:39.783 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T03:25:39.783 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T03:25:39.783 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T03:25:39.783 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T03:25:39.783 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T03:25:39.783 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T03:25:39.783 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T03:25:39.783 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T03:25:39.783 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T03:25:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T03:25:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T03:25:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T03:25:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T03:25:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T03:25:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T03:25:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T03:25:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T03:25:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T03:25:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T03:25:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.784 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T03:25:39.785 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T03:25:39.785 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T03:25:39.785 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T03:25:39.785 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T03:25:39.785 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T03:25:39.785 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T03:25:39.785 INFO:teuthology.orchestra.run.smithi131.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T03:25:39.785 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T03:25:39.785 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T03:25:39.785 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T03:25:39.785 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T03:25:39.785 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T03:25:39.785 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T03:25:39.785 INFO:teuthology.orchestra.run.smithi131.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T03:25:39.786 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.786 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.786 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:39.786 INFO:teuthology.orchestra.run.smithi131.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T03:25:39.786 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T03:25:39.786 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T03:25:39.786 INFO:teuthology.orchestra.run.smithi131.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T03:25:39.786 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T03:25:39.786 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T03:25:39.787 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T03:25:39.787 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:25:39.787 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-06-06T03:25:40.071 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T03:25:40.071 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T03:25:40.071 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T03:25:40.071 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T03:25:40.071 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T03:25:40.071 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T03:25:40.072 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T03:25:40.072 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T03:25:40.072 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T03:25:40.072 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T03:25:40.072 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T03:25:40.072 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T03:25:40.072 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T03:25:40.072 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T03:25:40.072 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T03:25:40.072 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T03:25:40.072 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T03:25:40.072 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T03:25:40.072 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T03:25:40.072 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T03:25:40.073 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T03:25:40.073 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T03:25:40.073 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T03:25:40.073 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T03:25:40.073 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T03:25:40.073 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T03:25:40.073 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T03:25:40.073 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T03:25:40.085 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T03:25:40.086 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T03:25:40.086 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T03:25:40.086 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T03:25:40.086 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T03:25:40.086 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T03:25:40.086 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T03:25:40.086 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T03:25:40.086 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T03:25:40.086 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T03:25:40.087 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T03:25:40.087 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T03:25:40.087 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T03:25:40.087 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T03:25:40.087 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T03:25:40.087 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T03:25:40.087 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T03:25:40.087 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T03:25:40.087 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T03:25:40.087 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T03:25:40.087 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T03:25:40.087 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T03:25:40.087 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T03:25:40.087 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T03:25:40.088 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T03:25:40.088 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T03:25:40.088 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T03:25:40.088 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T03:25:40.088 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T03:25:40.088 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T03:25:40.088 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T03:25:40.088 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T03:25:40.088 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T03:25:40.088 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T03:25:40.088 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T03:25:40.088 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T03:25:40.088 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T03:25:40.089 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T03:25:40.089 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T03:25:40.089 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T03:25:40.089 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T03:25:40.089 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T03:25:40.089 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T03:25:40.089 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T03:25:40.089 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T03:25:40.089 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T03:25:40.089 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T03:25:40.089 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T03:25:40.089 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T03:25:40.089 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T03:25:40.090 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T03:25:40.090 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T03:25:40.090 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T03:25:40.090 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T03:25:40.090 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T03:25:40.090 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T03:25:40.090 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T03:25:40.090 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T03:25:40.090 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T03:25:40.090 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T03:25:40.090 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T03:25:40.090 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T03:25:40.090 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T03:25:40.090 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T03:25:40.091 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T03:25:40.091 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T03:25:40.091 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T03:25:40.091 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T03:25:40.091 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T03:25:40.091 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T03:25:40.091 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T03:25:40.091 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T03:25:40.091 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T03:25:40.091 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T03:25:40.091 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T03:25:40.091 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T03:25:40.091 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T03:25:40.091 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T03:25:40.092 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T03:25:40.092 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T03:25:40.092 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T03:25:40.092 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T03:25:40.092 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T03:25:40.092 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T03:25:40.092 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T03:25:40.092 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T03:25:40.092 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T03:25:40.092 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T03:25:40.092 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T03:25:40.092 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T03:25:40.092 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T03:25:40.092 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T03:25:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T03:25:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T03:25:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T03:25:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T03:25:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T03:25:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T03:25:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T03:25:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T03:25:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T03:25:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T03:25:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T03:25:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T03:25:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T03:25:40.093 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T03:25:40.094 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T03:25:40.094 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T03:25:40.094 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T03:25:40.094 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T03:25:40.094 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T03:25:40.094 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T03:25:40.094 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T03:25:40.094 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T03:25:40.094 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T03:25:40.094 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T03:25:40.094 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T03:25:40.094 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T03:25:40.094 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T03:25:40.094 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T03:25:40.095 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T03:25:40.096 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T03:25:40.096 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T03:25:40.096 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T03:25:40.096 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T03:25:40.096 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T03:25:40.096 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T03:25:40.096 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T03:25:40.242 DEBUG:teuthology.parallel:result is None 2024-06-06T03:25:42.894 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T03:25:42.894 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:25:42.894 INFO:teuthology.orchestra.run.smithi104.stdout:Upgraded: 2024-06-06T03:25:42.895 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T03:25:42.895 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T03:25:42.895 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T03:25:42.895 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T03:25:42.895 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T03:25:42.895 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T03:25:42.895 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T03:25:42.895 INFO:teuthology.orchestra.run.smithi104.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.895 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.895 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T03:25:42.895 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T03:25:42.896 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-06-06T03:25:42.896 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T03:25:42.896 INFO:teuthology.orchestra.run.smithi104.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T03:25:42.896 INFO:teuthology.orchestra.run.smithi104.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T03:25:42.896 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T03:25:42.896 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.896 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.896 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.896 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.896 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:42.896 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.897 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.897 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.897 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:42.897 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:42.897 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:42.897 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:42.897 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:42.897 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.897 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.897 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:42.897 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.897 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.898 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.898 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:42.898 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:42.898 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.898 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.898 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:42.898 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T03:25:42.898 INFO:teuthology.orchestra.run.smithi104.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T03:25:42.898 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T03:25:42.898 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T03:25:42.898 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T03:25:42.899 INFO:teuthology.orchestra.run.smithi104.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T03:25:42.899 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T03:25:42.899 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T03:25:42.899 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T03:25:42.899 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T03:25:42.899 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T03:25:42.899 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.899 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.899 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.899 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T03:25:42.900 INFO:teuthology.orchestra.run.smithi104.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T03:25:42.900 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T03:25:42.900 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T03:25:42.900 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T03:25:42.900 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.900 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.900 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T03:25:42.900 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.900 INFO:teuthology.orchestra.run.smithi104.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T03:25:42.900 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T03:25:42.900 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T03:25:42.900 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T03:25:42.901 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T03:25:42.901 INFO:teuthology.orchestra.run.smithi104.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T03:25:42.901 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T03:25:42.901 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T03:25:42.901 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T03:25:42.901 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T03:25:42.901 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T03:25:42.901 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T03:25:42.901 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T03:25:42.901 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T03:25:42.901 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T03:25:42.902 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T03:25:42.902 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T03:25:42.902 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T03:25:42.902 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T03:25:42.902 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T03:25:42.902 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T03:25:42.902 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T03:25:42.902 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T03:25:42.902 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.902 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.902 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T03:25:42.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T03:25:42.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T03:25:42.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T03:25:42.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T03:25:42.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T03:25:42.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T03:25:42.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T03:25:42.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T03:25:42.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T03:25:42.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T03:25:42.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T03:25:42.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T03:25:42.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T03:25:42.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T03:25:42.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T03:25:42.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T03:25:42.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T03:25:42.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T03:25:42.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T03:25:42.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T03:25:42.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T03:25:42.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T03:25:42.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T03:25:42.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T03:25:42.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T03:25:42.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T03:25:42.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T03:25:42.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T03:25:42.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T03:25:42.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T03:25:42.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T03:25:42.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T03:25:42.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T03:25:42.906 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T03:25:42.906 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T03:25:42.906 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T03:25:42.906 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T03:25:42.906 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T03:25:42.906 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T03:25:42.906 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.906 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.906 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T03:25:42.906 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T03:25:42.906 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T03:25:42.907 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.907 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T03:25:42.907 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T03:25:42.907 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T03:25:42.907 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T03:25:42.907 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T03:25:42.907 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T03:25:42.907 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T03:25:42.907 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T03:25:42.907 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T03:25:42.908 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T03:25:42.908 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T03:25:42.908 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T03:25:42.908 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T03:25:42.908 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T03:25:42.908 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T03:25:42.908 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.908 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.908 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:42.908 INFO:teuthology.orchestra.run.smithi104.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T03:25:42.908 INFO:teuthology.orchestra.run.smithi104.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T03:25:42.908 INFO:teuthology.orchestra.run.smithi104.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T03:25:42.908 INFO:teuthology.orchestra.run.smithi104.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T03:25:42.909 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T03:25:42.909 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T03:25:42.909 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T03:25:42.909 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-06-06T03:25:42.909 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-06-06T03:25:43.007 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T03:25:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T03:25:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T03:25:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T03:25:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T03:25:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T03:25:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T03:25:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T03:25:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T03:25:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T03:25:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T03:25:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T03:25:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T03:25:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T03:25:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T03:25:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T03:25:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T03:25:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T03:25:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T03:25:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T03:25:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T03:25:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T03:25:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T03:25:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T03:25:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T03:25:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T03:25:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T03:25:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T03:25:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T03:25:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T03:25:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T03:25:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T03:25:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T03:25:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T03:25:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T03:25:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T03:25:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T03:25:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T03:25:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T03:25:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T03:25:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T03:25:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T03:25:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T03:25:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T03:25:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T03:25:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T03:25:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T03:25:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T03:25:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T03:25:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T03:25:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T03:25:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T03:25:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T03:25:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T03:25:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T03:25:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T03:25:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T03:25:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T03:25:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T03:25:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T03:25:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T03:25:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T03:25:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T03:25:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T03:25:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T03:25:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T03:25:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T03:25:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T03:25:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T03:25:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T03:25:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T03:25:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T03:25:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T03:25:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T03:25:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T03:25:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T03:25:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T03:25:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T03:25:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T03:25:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T03:25:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T03:25:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T03:25:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T03:25:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T03:25:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T03:25:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T03:25:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T03:25:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T03:25:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T03:25:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T03:25:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T03:25:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T03:25:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T03:25:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T03:25:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T03:25:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T03:25:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T03:25:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T03:25:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T03:25:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T03:25:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T03:25:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T03:25:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T03:25:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T03:25:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T03:25:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T03:25:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T03:25:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T03:25:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T03:25:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T03:25:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T03:25:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T03:25:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T03:25:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T03:25:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T03:25:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T03:25:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T03:25:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T03:25:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T03:25:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T03:25:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T03:25:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T03:25:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T03:25:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T03:25:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T03:25:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T03:25:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T03:25:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T03:25:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T03:25:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T03:25:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T03:25:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T03:25:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T03:25:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T03:25:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T03:25:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T03:25:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T03:25:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T03:25:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T03:25:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T03:25:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T03:25:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T03:25:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T03:25:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T03:25:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T03:25:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T03:25:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T03:25:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T03:25:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T03:25:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T03:25:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T03:25:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T03:25:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T03:25:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T03:25:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T03:25:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T03:25:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T03:25:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T03:25:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T03:25:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T03:25:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T03:25:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T03:25:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T03:25:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T03:25:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T03:25:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T03:25:43.023 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T03:25:43.023 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T03:25:43.023 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T03:25:43.023 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T03:25:43.330 DEBUG:teuthology.parallel:result is None 2024-06-06T03:25:46.062 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T03:25:46.063 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:25:46.063 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2024-06-06T03:25:46.063 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T03:25:46.063 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T03:25:46.063 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T03:25:46.063 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T03:25:46.063 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T03:25:46.063 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T03:25:46.063 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T03:25:46.063 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.063 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.064 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T03:25:46.064 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T03:25:46.064 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-06-06T03:25:46.064 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T03:25:46.064 INFO:teuthology.orchestra.run.smithi040.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T03:25:46.064 INFO:teuthology.orchestra.run.smithi040.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T03:25:46.064 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T03:25:46.064 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.064 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.064 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.064 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.064 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:46.064 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.065 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.065 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.065 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:46.065 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:46.065 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:46.065 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:46.065 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:46.065 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.065 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.065 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:46.065 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.065 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.065 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.065 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:46.066 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:46.066 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.066 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.066 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T03:25:46.066 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T03:25:46.066 INFO:teuthology.orchestra.run.smithi040.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T03:25:46.066 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T03:25:46.066 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T03:25:46.066 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T03:25:46.066 INFO:teuthology.orchestra.run.smithi040.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T03:25:46.066 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T03:25:46.066 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T03:25:46.067 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T03:25:46.067 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T03:25:46.067 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T03:25:46.067 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.067 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.067 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.067 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T03:25:46.067 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T03:25:46.067 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T03:25:46.067 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T03:25:46.067 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T03:25:46.067 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.067 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.067 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T03:25:46.068 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T03:25:46.069 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T03:25:46.069 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T03:25:46.069 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T03:25:46.069 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T03:25:46.069 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T03:25:46.069 INFO:teuthology.orchestra.run.smithi040.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T03:25:46.069 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T03:25:46.069 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T03:25:46.069 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T03:25:46.069 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.069 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.069 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.069 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T03:25:46.070 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T03:25:46.070 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T03:25:46.070 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T03:25:46.070 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T03:25:46.070 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T03:25:46.070 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T03:25:46.070 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T03:25:46.070 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T03:25:46.070 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T03:25:46.070 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T03:25:46.070 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T03:25:46.070 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T03:25:46.070 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T03:25:46.071 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T03:25:46.071 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T03:25:46.071 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T03:25:46.071 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T03:25:46.071 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T03:25:46.071 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T03:25:46.071 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T03:25:46.071 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T03:25:46.071 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T03:25:46.071 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T03:25:46.071 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T03:25:46.071 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T03:25:46.071 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T03:25:46.071 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T03:25:46.072 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T03:25:46.072 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T03:25:46.072 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T03:25:46.072 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T03:25:46.072 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T03:25:46.072 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T03:25:46.072 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T03:25:46.072 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T03:25:46.072 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T03:25:46.072 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T03:25:46.072 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T03:25:46.072 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T03:25:46.072 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.072 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.073 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T03:25:46.073 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T03:25:46.073 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T03:25:46.073 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.073 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T03:25:46.073 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T03:25:46.073 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T03:25:46.073 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T03:25:46.073 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T03:25:46.073 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T03:25:46.073 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T03:25:46.073 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T03:25:46.073 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T03:25:46.074 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T03:25:46.074 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T03:25:46.074 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T03:25:46.074 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T03:25:46.074 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T03:25:46.074 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T03:25:46.074 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.074 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.074 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T03:25:46.074 INFO:teuthology.orchestra.run.smithi040.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T03:25:46.074 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T03:25:46.074 INFO:teuthology.orchestra.run.smithi040.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T03:25:46.075 INFO:teuthology.orchestra.run.smithi040.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T03:25:46.075 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T03:25:46.075 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T03:25:46.075 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T03:25:46.075 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:25:46.075 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-06-06T03:25:46.586 DEBUG:teuthology.parallel:result is None 2024-06-06T03:25:46.587 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:25:46.728 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T03:25:46.753 INFO:teuthology.orchestra.run.smithi040.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T03:25:46.753 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T03:25:46.753 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T03:25:46.755 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:25:46.962 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T03:25:46.988 INFO:teuthology.orchestra.run.smithi104.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T03:25:46.988 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T03:25:46.988 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T03:25:46.990 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:25:47.126 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T03:25:47.151 INFO:teuthology.orchestra.run.smithi131.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T03:25:47.152 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T03:25:47.152 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T03:25:47.153 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-06T03:25:47.154 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:25:47.154 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T03:25:47.187 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-06-06T03:25:47.187 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T03:25:47.218 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-06-06T03:25:47.218 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T03:25:47.248 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-06T03:25:47.248 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:25:47.248 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T03:25:47.277 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T03:25:47.344 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-06-06T03:25:47.345 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T03:25:47.374 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T03:25:47.441 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-06-06T03:25:47.441 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T03:25:47.469 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T03:25:47.537 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-06T03:25:47.538 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:25:47.538 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T03:25:47.566 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T03:25:47.635 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-06-06T03:25:47.635 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T03:25:47.662 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T03:25:47.728 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-06-06T03:25:47.729 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T03:25:47.756 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T03:25:47.824 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-06T03:25:47.824 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:25:47.824 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T03:25:47.855 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T03:25:47.925 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-06-06T03:25:47.926 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T03:25:47.953 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T03:25:48.019 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-06-06T03:25:48.020 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T03:25:48.047 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T03:25:48.115 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-06T03:25:48.207 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-06-06T03:25:48.208 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:25:48.208 INFO:tasks.cephadm:Cluster fsid is 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:25:48.208 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-06T03:25:48.209 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.40', 'mon.b': '172.21.15.104', 'mon.c': '172.21.15.131'} 2024-06-06T03:25:48.209 INFO:tasks.cephadm:First mon is mon.a on smithi040 2024-06-06T03:25:48.209 INFO:tasks.cephadm:First mgr is x 2024-06-06T03:25:48.209 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-06T03:25:48.209 DEBUG:teuthology.orchestra.run.smithi040:> sudo hostname $(hostname -s) 2024-06-06T03:25:48.241 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname $(hostname -s) 2024-06-06T03:25:48.272 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname $(hostname -s) 2024-06-06T03:25:48.307 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-06T03:25:48.307 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:25:48.459 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-06T03:25:48.680 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:25:48.681 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-06T03:25:48.681 DEBUG:teuthology.orchestra.run.smithi040:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T03:25:49.099 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 03:25 /home/ubuntu/cephtest/cephadm 2024-06-06T03:25:49.099 DEBUG:teuthology.orchestra.run.smithi104:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T03:25:49.489 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 03:25 /home/ubuntu/cephtest/cephadm 2024-06-06T03:25:49.489 DEBUG:teuthology.orchestra.run.smithi131:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T03:25:49.890 INFO:teuthology.orchestra.run.smithi131.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 03:25 /home/ubuntu/cephtest/cephadm 2024-06-06T03:25:49.890 DEBUG:teuthology.orchestra.run.smithi040:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T03:25:49.910 DEBUG:teuthology.orchestra.run.smithi104:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T03:25:49.929 DEBUG:teuthology.orchestra.run.smithi131:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T03:25:49.962 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-06T03:25:49.962 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T03:25:49.965 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T03:25:49.971 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T03:25:50.763 INFO:teuthology.orchestra.run.smithi104.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T03:25:50.776 INFO:teuthology.orchestra.run.smithi131.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T03:25:50.883 INFO:teuthology.orchestra.run.smithi040.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T03:26:20.865 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:26:20.865 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T03:26:20.865 INFO:teuthology.orchestra.run.smithi040.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T03:26:20.865 INFO:teuthology.orchestra.run.smithi040.stdout: "repo_digests": [ 2024-06-06T03:26:20.865 INFO:teuthology.orchestra.run.smithi040.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T03:26:20.865 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-06-06T03:26:20.865 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:26:41.281 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-06-06T03:26:41.281 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T03:26:41.281 INFO:teuthology.orchestra.run.smithi131.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T03:26:41.282 INFO:teuthology.orchestra.run.smithi131.stdout: "repo_digests": [ 2024-06-06T03:26:41.282 INFO:teuthology.orchestra.run.smithi131.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T03:26:41.282 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-06-06T03:26:41.282 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-06-06T03:26:43.038 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-06-06T03:26:43.038 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T03:26:43.038 INFO:teuthology.orchestra.run.smithi104.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T03:26:43.038 INFO:teuthology.orchestra.run.smithi104.stdout: "repo_digests": [ 2024-06-06T03:26:43.038 INFO:teuthology.orchestra.run.smithi104.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T03:26:43.038 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-06-06T03:26:43.038 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-06-06T03:26:43.064 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph 2024-06-06T03:26:43.100 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph 2024-06-06T03:26:43.126 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph 2024-06-06T03:26:43.158 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /etc/ceph 2024-06-06T03:26:43.189 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /etc/ceph 2024-06-06T03:26:43.212 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /etc/ceph 2024-06-06T03:26:43.240 INFO:tasks.cephadm:Writing seed config... 2024-06-06T03:26:43.242 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-06T03:26:43.243 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-06T03:26:43.243 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-06T03:26:43.243 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-06T03:26:43.243 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-06T03:26:43.243 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-06-06T03:26:43.243 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-06T03:26:43.244 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-06T03:26:43.244 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-06T03:26:43.244 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-06T03:26:43.244 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-06T03:26:43.244 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-06T03:26:43.244 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-06T03:26:43.244 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-06T03:26:43.245 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-06T03:26:43.245 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-06T03:26:43.245 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-06T03:26:43.245 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-06T03:26:43.245 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-06T03:26:43.245 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-06T03:26:43.245 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-06T03:26:43.246 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-06-06T03:26:43.246 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-06T03:26:43.246 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-06T03:26:43.246 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-06T03:26:43.246 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-06T03:26:43.246 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-06T03:26:43.247 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-06T03:26:43.247 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-06T03:26:43.247 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-06T03:26:43.247 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-06T03:26:43.247 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-06-06T03:26:43.247 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-06T03:26:43.247 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-06T03:26:43.248 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-06T03:26:43.248 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-06T03:26:43.248 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-06T03:26:43.248 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-06T03:26:43.248 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-06T03:26:43.248 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-06T03:26:43.248 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-06T03:26:43.249 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-06T03:26:43.249 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-06T03:26:43.249 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-06T03:26:43.249 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-06T03:26:43.249 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-06-06T03:26:43.250 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:26:43.251 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-06T03:26:43.271 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 77c25d68-23b4-11ef-bc9b-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-06T03:26:43.272 DEBUG:teuthology.orchestra.run.smithi040:mon.a> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mon.a.service 2024-06-06T03:26:43.315 DEBUG:teuthology.orchestra.run.smithi040:mgr.x> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-06T03:26:43.358 INFO:tasks.cephadm:Bootstrapping... 2024-06-06T03:26:43.358 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.40 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-06T03:26:43.563 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-06-06T03:26:43.563 INFO:teuthology.orchestra.run.smithi040.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', '77c25d68-23b4-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.40', '--skip-admin-label'] 2024-06-06T03:26:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-06T03:26:43.564 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying podman|docker is present... 2024-06-06T03:26:43.590 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stdout 5.0.2 2024-06-06T03:26:43.590 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying lvm2 is present... 2024-06-06T03:26:43.590 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying time synchronization is in place... 2024-06-06T03:26:43.598 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-06T03:26:43.598 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-06T03:26:43.604 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-06T03:26:43.605 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2024-06-06T03:26:43.612 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout enabled 2024-06-06T03:26:43.619 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout active 2024-06-06T03:26:43.619 INFO:teuthology.orchestra.run.smithi040.stdout:Unit chronyd.service is enabled and running 2024-06-06T03:26:43.619 INFO:teuthology.orchestra.run.smithi040.stdout:Repeating the final host check... 2024-06-06T03:26:43.644 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stdout 5.0.2 2024-06-06T03:26:43.644 INFO:teuthology.orchestra.run.smithi040.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-06T03:26:43.645 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl is present 2024-06-06T03:26:43.645 INFO:teuthology.orchestra.run.smithi040.stdout:lvcreate is present 2024-06-06T03:26:43.651 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-06T03:26:43.651 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-06T03:26:43.657 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-06T03:26:43.657 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2024-06-06T03:26:43.664 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout enabled 2024-06-06T03:26:43.671 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout active 2024-06-06T03:26:43.671 INFO:teuthology.orchestra.run.smithi040.stdout:Unit chronyd.service is enabled and running 2024-06-06T03:26:43.671 INFO:teuthology.orchestra.run.smithi040.stdout:Host looks OK 2024-06-06T03:26:43.671 INFO:teuthology.orchestra.run.smithi040.stdout:Cluster fsid: 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:26:43.671 INFO:teuthology.orchestra.run.smithi040.stdout:Acquiring lock 140477129074048 on /run/cephadm/77c25d68-23b4-11ef-bc9b-c7b262605968.lock 2024-06-06T03:26:43.671 INFO:teuthology.orchestra.run.smithi040.stdout:Lock 140477129074048 acquired on /run/cephadm/77c25d68-23b4-11ef-bc9b-c7b262605968.lock 2024-06-06T03:26:43.672 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying IP 172.21.15.40 port 3300 ... 2024-06-06T03:26:43.672 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying IP 172.21.15.40 port 6789 ... 2024-06-06T03:26:43.672 INFO:teuthology.orchestra.run.smithi040.stdout:Base mon IP(s) is [172.21.15.40:3300, 172.21.15.40:6789], mon addrv is [v2:172.21.15.40:3300,v1:172.21.15.40:6789] 2024-06-06T03:26:43.675 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.40 metric 100 2024-06-06T03:26:43.675 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.40 metric 100 2024-06-06T03:26:43.678 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-06T03:26:43.678 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-06-06T03:26:43.678 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-06-06T03:26:43.680 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-06T03:26:43.681 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-06T03:26:43.681 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-06T03:26:43.681 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-06-06T03:26:43.681 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-06-06T03:26:43.681 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-06T03:26:43.681 INFO:teuthology.orchestra.run.smithi040.stdout:Mon IP `172.21.15.40` is in CIDR network `172.21.0.0/20` 2024-06-06T03:26:43.681 INFO:teuthology.orchestra.run.smithi040.stdout:Mon IP `172.21.15.40` is in CIDR network `172.21.0.0/20` 2024-06-06T03:26:43.682 INFO:teuthology.orchestra.run.smithi040.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-06T03:26:43.682 INFO:teuthology.orchestra.run.smithi040.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-06T03:26:43.683 INFO:teuthology.orchestra.run.smithi040.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T03:26:44.930 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-06T03:26:44.930 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T03:26:44.930 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Getting image source signatures 2024-06-06T03:26:44.930 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-06T03:26:44.930 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-06T03:26:44.930 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-06T03:26:44.930 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-06T03:26:45.908 INFO:teuthology.orchestra.run.smithi040.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T03:26:45.908 INFO:teuthology.orchestra.run.smithi040.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T03:26:45.908 INFO:teuthology.orchestra.run.smithi040.stdout:Extracting ceph user uid/gid from container image... 2024-06-06T03:26:46.931 INFO:teuthology.orchestra.run.smithi040.stdout:stat: stdout 167 167 2024-06-06T03:26:46.932 INFO:teuthology.orchestra.run.smithi040.stdout:Creating initial keys... 2024-06-06T03:26:47.845 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph-authtool: stdout AQB3LGFm9OT7DhAAEdp+Ts/MHp7VdC/tkyCztA== 2024-06-06T03:26:48.759 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph-authtool: stdout AQB4LGFmh5JwCxAAgeYpxjyftDD/i/hmvxUOpQ== 2024-06-06T03:26:49.614 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph-authtool: stdout AQB5LGFmzUBYBBAADBMHlhnANInt/LRZTSpLbQ== 2024-06-06T03:26:49.618 INFO:teuthology.orchestra.run.smithi040.stdout:Creating initial monmap... 2024-06-06T03:26:50.829 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-06T03:26:50.830 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-06T03:26:50.830 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:26:50.830 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-06T03:26:50.830 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool for a [v2:172.21.15.40:3300,v1:172.21.15.40:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-06T03:26:50.830 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = quincy 2024-06-06T03:26:50.830 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: set fsid to 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:26:50.830 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-06T03:26:50.830 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:26:50.830 INFO:teuthology.orchestra.run.smithi040.stdout:Creating mon... 2024-06-06T03:26:51.760 INFO:teuthology.orchestra.run.smithi040.stdout:create mon.a on 2024-06-06T03:26:51.969 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-06T03:26:52.111 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-06T03:26:52.310 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-77c25d68-23b4-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-77c25d68-23b4-11ef-bc9b-c7b262605968.target. 2024-06-06T03:26:52.310 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-77c25d68-23b4-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-77c25d68-23b4-11ef-bc9b-c7b262605968.target. 2024-06-06T03:26:52.531 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mon.a 2024-06-06T03:26:52.531 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Failed to reset failed state of unit ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-06T03:26:52.703 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-77c25d68-23b4-11ef-bc9b-c7b262605968.target.wants/ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@.service. 2024-06-06T03:26:53.187 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:53 smithi040 ceph-mon[31847]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-06T03:26:53.188 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:53 smithi040 bash[31827]: 0c05c2b9acd496f9a9df385b3ff3f5d8cb54f73e39c58d937ce9a79b9f3bc204 2024-06-06T03:26:53.200 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T03:26:53.201 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout disabled 2024-06-06T03:26:53.208 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T03:26:53.208 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2024-06-06T03:26:53.208 INFO:teuthology.orchestra.run.smithi040.stdout:firewalld.service is not enabled 2024-06-06T03:26:53.208 INFO:teuthology.orchestra.run.smithi040.stdout:Not possible to enable service . firewalld.service is not available 2024-06-06T03:26:53.209 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mon to start... 2024-06-06T03:26:53.209 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mon... 2024-06-06T03:26:53.515 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:53 smithi040 systemd[1]: Started Ceph mon.a for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:26:54.194 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout cluster: 2024-06-06T03:26:54.194 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout id: 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:26:54.194 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-06T03:26:54.194 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-06-06T03:26:54.194 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout services: 2024-06-06T03:26:54.194 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.536028s) 2024-06-06T03:26:54.194 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-06T03:26:54.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-06T03:26:54.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-06-06T03:26:54.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout data: 2024-06-06T03:26:54.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-06T03:26:54.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-06T03:26:54.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-06T03:26:54.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout pgs: 2024-06-06T03:26:54.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-06-06T03:26:54.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.618+0000 7f50c285a700 1 Processor -- start 2024-06-06T03:26:54.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.618+0000 7f50c285a700 1 -- start start 2024-06-06T03:26:54.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.618+0000 7f50c285a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50bc106620 0x7f50bc106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:26:54.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.618+0000 7f50c285a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc107010 con 0x7f50bc106620 2024-06-06T03:26:54.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.619+0000 7f50bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50bc106620 0x7f50bc106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:26:54.196 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.619+0000 7f50bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50bc106620 0x7f50bc106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35036/0 (socket says 172.21.15.40:35036) 2024-06-06T03:26:54.196 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.619+0000 7f50bbfff700 1 -- 172.21.15.40:0/996704015 learned_addr learned my addr 172.21.15.40:0/996704015 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:26:54.196 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.619+0000 7f50bbfff700 1 -- 172.21.15.40:0/996704015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc107820 con 0x7f50bc106620 2024-06-06T03:26:54.196 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.619+0000 7f50bbfff700 1 --2- 172.21.15.40:0/996704015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50bc106620 0x7f50bc106a40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f50ac0098a0 tx=0x7f50ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbd6a222613d9d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:26:54.196 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.619+0000 7f50bb7fe700 1 -- 172.21.15.40:0/996704015 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ac004160 con 0x7f50bc106620 2024-06-06T03:26:54.196 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.619+0000 7f50bb7fe700 1 -- 172.21.15.40:0/996704015 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ac004300 con 0x7f50bc106620 2024-06-06T03:26:54.196 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.619+0000 7f50bb7fe700 1 -- 172.21.15.40:0/996704015 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ac0036a0 con 0x7f50bc106620 2024-06-06T03:26:54.196 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.620+0000 7f50c285a700 1 -- 172.21.15.40:0/996704015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50bc106620 msgr2=0x7f50bc106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:26:54.196 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.620+0000 7f50c285a700 1 --2- 172.21.15.40:0/996704015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50bc106620 0x7f50bc106a40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f50ac0098a0 tx=0x7f50ac009bb0 comp rx=0 tx=0).stop 2024-06-06T03:26:54.196 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.620+0000 7f50c285a700 1 -- 172.21.15.40:0/996704015 shutdown_connections 2024-06-06T03:26:54.196 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.620+0000 7f50c285a700 1 --2- 172.21.15.40:0/996704015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50bc106620 0x7f50bc106a40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:26:54.196 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.620+0000 7f50c285a700 1 -- 172.21.15.40:0/996704015 >> 172.21.15.40:0/996704015 conn(0x7f50bc101bc0 msgr2=0x7f50bc104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:26:54.196 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.620+0000 7f50c285a700 1 -- 172.21.15.40:0/996704015 shutdown_connections 2024-06-06T03:26:54.196 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.620+0000 7f50c285a700 1 -- 172.21.15.40:0/996704015 wait complete. 2024-06-06T03:26:54.197 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.621+0000 7f50c285a700 1 Processor -- start 2024-06-06T03:26:54.197 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.621+0000 7f50c285a700 1 -- start start 2024-06-06T03:26:54.197 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.621+0000 7f50c285a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50bc106620 0x7f50bc19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:26:54.197 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.621+0000 7f50c285a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc107d50 con 0x7f50bc106620 2024-06-06T03:26:54.197 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.621+0000 7f50bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50bc106620 0x7f50bc19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:26:54.197 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.621+0000 7f50bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50bc106620 0x7f50bc19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35038/0 (socket says 172.21.15.40:35038) 2024-06-06T03:26:54.197 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.621+0000 7f50bbfff700 1 -- 172.21.15.40:0/1784784429 learned_addr learned my addr 172.21.15.40:0/1784784429 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:26:54.197 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.621+0000 7f50bbfff700 1 -- 172.21.15.40:0/1784784429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc19aae0 con 0x7f50bc106620 2024-06-06T03:26:54.197 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.622+0000 7f50bbfff700 1 --2- 172.21.15.40:0/1784784429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50bc106620 0x7f50bc19a5a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f50ac000c00 tx=0x7f50ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:26:54.197 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.622+0000 7f50b9ffb700 1 -- 172.21.15.40:0/1784784429 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ac003db0 con 0x7f50bc106620 2024-06-06T03:26:54.197 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.622+0000 7f50b9ffb700 1 -- 172.21.15.40:0/1784784429 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ac011630 con 0x7f50bc106620 2024-06-06T03:26:54.197 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.622+0000 7f50b9ffb700 1 -- 172.21.15.40:0/1784784429 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ac011950 con 0x7f50bc106620 2024-06-06T03:26:54.197 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.622+0000 7f50c285a700 1 -- 172.21.15.40:0/1784784429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50bc19ad70 con 0x7f50bc106620 2024-06-06T03:26:54.197 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.622+0000 7f50c285a700 1 -- 172.21.15.40:0/1784784429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50bc19da60 con 0x7f50bc106620 2024-06-06T03:26:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.622+0000 7f50b9ffb700 1 -- 172.21.15.40:0/1784784429 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f50ac01a460 con 0x7f50bc106620 2024-06-06T03:26:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.622+0000 7f50b9ffb700 1 -- 172.21.15.40:0/1784784429 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f50ac01b780 con 0x7f50bc106620 2024-06-06T03:26:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.623+0000 7f50c285a700 1 -- 172.21.15.40:0/1784784429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50bc063af0 con 0x7f50bc106620 2024-06-06T03:26:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.625+0000 7f50b9ffb700 1 -- 172.21.15.40:0/1784784429 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f50ac01b980 con 0x7f50bc106620 2024-06-06T03:26:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.699+0000 7f50c285a700 1 -- 172.21.15.40:0/1784784429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f50bc02def0 con 0x7f50bc106620 2024-06-06T03:26:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.699+0000 7f50b9ffb700 1 -- 172.21.15.40:0/1784784429 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f50ac01bb60 con 0x7f50bc106620 2024-06-06T03:26:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.700+0000 7f50c285a700 1 -- 172.21.15.40:0/1784784429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50bc106620 msgr2=0x7f50bc19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:26:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.700+0000 7f50c285a700 1 --2- 172.21.15.40:0/1784784429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50bc106620 0x7f50bc19a5a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f50ac000c00 tx=0x7f50ac004750 comp rx=0 tx=0).stop 2024-06-06T03:26:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.700+0000 7f50c285a700 1 -- 172.21.15.40:0/1784784429 shutdown_connections 2024-06-06T03:26:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.700+0000 7f50c285a700 1 --2- 172.21.15.40:0/1784784429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50bc106620 0x7f50bc19a5a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:26:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.700+0000 7f50c285a700 1 -- 172.21.15.40:0/1784784429 >> 172.21.15.40:0/1784784429 conn(0x7f50bc101bc0 msgr2=0x7f50bc191100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:26:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.700+0000 7f50c285a700 1 -- 172.21.15.40:0/1784784429 shutdown_connections 2024-06-06T03:26:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:53.700+0000 7f50c285a700 1 -- 172.21.15.40:0/1784784429 wait complete. 2024-06-06T03:26:54.199 INFO:teuthology.orchestra.run.smithi040.stdout:mon is available 2024-06-06T03:26:54.199 INFO:teuthology.orchestra.run.smithi040.stdout:Assimilating anything we can from ceph.conf... 2024-06-06T03:26:54.430 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:54 smithi040 ceph-mon[31847]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-06T03:26:54.430 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:54 smithi040 ceph-mon[31847]: monmap epoch 1 2024-06-06T03:26:54.430 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:54 smithi040 ceph-mon[31847]: fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:26:54.430 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:54 smithi040 ceph-mon[31847]: last_changed 2024-06-06T03:26:49.934047+0000 2024-06-06T03:26:54.430 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:54 smithi040 ceph-mon[31847]: created 2024-06-06T03:26:49.934047+0000 2024-06-06T03:26:54.430 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:54 smithi040 ceph-mon[31847]: min_mon_release 19 (squid) 2024-06-06T03:26:54.430 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:54 smithi040 ceph-mon[31847]: election_strategy: 1 2024-06-06T03:26:54.431 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:54 smithi040 ceph-mon[31847]: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-06-06T03:26:54.431 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:54 smithi040 ceph-mon[31847]: fsmap 2024-06-06T03:26:54.431 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:54 smithi040 ceph-mon[31847]: osdmap e1: 0 total, 0 up, 0 in 2024-06-06T03:26:54.431 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:54 smithi040 ceph-mon[31847]: mgrmap e1: no daemons active 2024-06-06T03:26:54.431 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:54 smithi040 ceph-mon[31847]: from='client.? 172.21.15.40:0/1784784429' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-06T03:26:55.292 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-06-06T03:26:55.292 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [global] 2024-06-06T03:26:55.292 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout fsid = 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:26:55.292 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.40:3300,v1:172.21.15.40:6789] 2024-06-06T03:26:55.293 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-06T03:26:55.293 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-06T03:26:55.293 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-06T03:26:55.293 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-06T03:26:55.293 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-06-06T03:26:55.293 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-06T03:26:55.293 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-06T03:26:55.293 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-06-06T03:26:55.293 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [osd] 2024-06-06T03:26:55.293 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-06T03:26:55.293 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-06T03:26:55.293 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-06T03:26:55.293 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-06T03:26:55.294 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-06T03:26:55.294 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.641+0000 7f897acb3700 1 Processor -- start 2024-06-06T03:26:55.294 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.641+0000 7f897acb3700 1 -- start start 2024-06-06T03:26:55.294 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.641+0000 7f897acb3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8974107f00 0x7f8974108340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:26:55.294 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.641+0000 7f897acb3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8974108880 con 0x7f8974107f00 2024-06-06T03:26:55.294 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.641+0000 7f8978a4f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8974107f00 0x7f8974108340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:26:55.294 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.641+0000 7f8978a4f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8974107f00 0x7f8974108340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35048/0 (socket says 172.21.15.40:35048) 2024-06-06T03:26:55.294 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.641+0000 7f8978a4f700 1 -- 172.21.15.40:0/2993264686 learned_addr learned my addr 172.21.15.40:0/2993264686 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:26:55.294 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.641+0000 7f8978a4f700 1 -- 172.21.15.40:0/2993264686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89741090e0 con 0x7f8974107f00 2024-06-06T03:26:55.294 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.642+0000 7f8978a4f700 1 --2- 172.21.15.40:0/2993264686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8974107f00 0x7f8974108340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f89680098a0 tx=0x7f8968009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e581d073c6a1c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:26:55.294 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.642+0000 7f89737fe700 1 -- 172.21.15.40:0/2993264686 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8968004160 con 0x7f8974107f00 2024-06-06T03:26:55.294 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.642+0000 7f89737fe700 1 -- 172.21.15.40:0/2993264686 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8968004300 con 0x7f8974107f00 2024-06-06T03:26:55.294 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.642+0000 7f89737fe700 1 -- 172.21.15.40:0/2993264686 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89680036a0 con 0x7f8974107f00 2024-06-06T03:26:55.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.642+0000 7f897acb3700 1 -- 172.21.15.40:0/2993264686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8974107f00 msgr2=0x7f8974108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:26:55.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.642+0000 7f897acb3700 1 --2- 172.21.15.40:0/2993264686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8974107f00 0x7f8974108340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f89680098a0 tx=0x7f8968009bb0 comp rx=0 tx=0).stop 2024-06-06T03:26:55.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.642+0000 7f897acb3700 1 -- 172.21.15.40:0/2993264686 shutdown_connections 2024-06-06T03:26:55.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.642+0000 7f897acb3700 1 --2- 172.21.15.40:0/2993264686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8974107f00 0x7f8974108340 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:26:55.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.642+0000 7f897acb3700 1 -- 172.21.15.40:0/2993264686 >> 172.21.15.40:0/2993264686 conn(0x7f8974103530 msgr2=0x7f8974105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:26:55.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.642+0000 7f897acb3700 1 -- 172.21.15.40:0/2993264686 shutdown_connections 2024-06-06T03:26:55.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.643+0000 7f897acb3700 1 -- 172.21.15.40:0/2993264686 wait complete. 2024-06-06T03:26:55.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.643+0000 7f897acb3700 1 Processor -- start 2024-06-06T03:26:55.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.643+0000 7f897acb3700 1 -- start start 2024-06-06T03:26:55.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.643+0000 7f897acb3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8974107f00 0x7f897407e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:26:55.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.643+0000 7f897acb3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8974109490 con 0x7f8974107f00 2024-06-06T03:26:55.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.643+0000 7f8978a4f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8974107f00 0x7f897407e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:26:55.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.643+0000 7f8978a4f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8974107f00 0x7f897407e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35060/0 (socket says 172.21.15.40:35060) 2024-06-06T03:26:55.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.643+0000 7f8978a4f700 1 -- 172.21.15.40:0/361607092 learned_addr learned my addr 172.21.15.40:0/361607092 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:26:55.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.644+0000 7f8978a4f700 1 -- 172.21.15.40:0/361607092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897407e910 con 0x7f8974107f00 2024-06-06T03:26:55.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.644+0000 7f8978a4f700 1 --2- 172.21.15.40:0/361607092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8974107f00 0x7f897407e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8968000c00 tx=0x7f8968004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:26:55.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.644+0000 7f8971ffb700 1 -- 172.21.15.40:0/361607092 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8968003db0 con 0x7f8974107f00 2024-06-06T03:26:55.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.644+0000 7f8971ffb700 1 -- 172.21.15.40:0/361607092 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8968011630 con 0x7f8974107f00 2024-06-06T03:26:55.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.644+0000 7f8971ffb700 1 -- 172.21.15.40:0/361607092 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8968011950 con 0x7f8974107f00 2024-06-06T03:26:55.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.644+0000 7f897acb3700 1 -- 172.21.15.40:0/361607092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897407cb00 con 0x7f8974107f00 2024-06-06T03:26:55.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.644+0000 7f897acb3700 1 -- 172.21.15.40:0/361607092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897407d040 con 0x7f8974107f00 2024-06-06T03:26:55.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.645+0000 7f8971ffb700 1 -- 172.21.15.40:0/361607092 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8968011af0 con 0x7f8974107f00 2024-06-06T03:26:55.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.645+0000 7f8971ffb700 1 -- 172.21.15.40:0/361607092 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f896801c450 con 0x7f8974107f00 2024-06-06T03:26:55.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.645+0000 7f897acb3700 1 -- 172.21.15.40:0/361607092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8974063af0 con 0x7f8974107f00 2024-06-06T03:26:55.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.648+0000 7f8971ffb700 1 -- 172.21.15.40:0/361607092 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8968032020 con 0x7f8974107f00 2024-06-06T03:26:55.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.718+0000 7f897acb3700 1 -- 172.21.15.40:0/361607092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f8974194ae0 con 0x7f8974107f00 2024-06-06T03:26:55.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.721+0000 7f8971ffb700 1 -- 172.21.15.40:0/361607092 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f8974194ae0 con 0x7f8974107f00 2024-06-06T03:26:55.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.721+0000 7f8971ffb700 1 -- 172.21.15.40:0/361607092 <== mon.0 v2:172.21.15.40:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8968004260 con 0x7f8974107f00 2024-06-06T03:26:55.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.722+0000 7f897acb3700 1 -- 172.21.15.40:0/361607092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8974107f00 msgr2=0x7f897407e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:26:55.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.722+0000 7f897acb3700 1 --2- 172.21.15.40:0/361607092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8974107f00 0x7f897407e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8968000c00 tx=0x7f8968004750 comp rx=0 tx=0).stop 2024-06-06T03:26:55.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.722+0000 7f897acb3700 1 -- 172.21.15.40:0/361607092 shutdown_connections 2024-06-06T03:26:55.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.722+0000 7f897acb3700 1 --2- 172.21.15.40:0/361607092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8974107f00 0x7f897407e3d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:26:55.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.722+0000 7f897acb3700 1 -- 172.21.15.40:0/361607092 >> 172.21.15.40:0/361607092 conn(0x7f8974103530 msgr2=0x7f8974192190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:26:55.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.722+0000 7f897acb3700 1 -- 172.21.15.40:0/361607092 shutdown_connections 2024-06-06T03:26:55.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:54.722+0000 7f897acb3700 1 -- 172.21.15.40:0/361607092 wait complete. 2024-06-06T03:26:55.297 INFO:teuthology.orchestra.run.smithi040.stdout:Generating new minimal ceph.conf... 2024-06-06T03:26:55.414 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:55 smithi040 ceph-mon[31847]: from='client.? 172.21.15.40:0/361607092' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-06T03:26:55.414 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:55 smithi040 ceph-mon[31847]: from='client.? 172.21.15.40:0/361607092' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-06T03:26:56.372 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.758+0000 7f5691c82700 1 Processor -- start 2024-06-06T03:26:56.372 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.758+0000 7f5691c82700 1 -- start start 2024-06-06T03:26:56.372 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.758+0000 7f5691c82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c106e70 0x7f568c107290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:26:56.372 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.758+0000 7f5691c82700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1077d0 con 0x7f568c106e70 2024-06-06T03:26:56.372 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.758+0000 7f568b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c106e70 0x7f568c107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:26:56.373 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.758+0000 7f568b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c106e70 0x7f568c107290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35064/0 (socket says 172.21.15.40:35064) 2024-06-06T03:26:56.373 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.758+0000 7f568b7fe700 1 -- 172.21.15.40:0/1381873877 learned_addr learned my addr 172.21.15.40:0/1381873877 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:26:56.373 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.759+0000 7f568b7fe700 1 -- 172.21.15.40:0/1381873877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c107fd0 con 0x7f568c106e70 2024-06-06T03:26:56.373 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.759+0000 7f568b7fe700 1 --2- 172.21.15.40:0/1381873877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c106e70 0x7f568c107290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f567c0098a0 tx=0x7f567c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb472070083f3ca server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:26:56.373 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.759+0000 7f568a7fc700 1 -- 172.21.15.40:0/1381873877 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f567c004160 con 0x7f568c106e70 2024-06-06T03:26:56.373 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.759+0000 7f568a7fc700 1 -- 172.21.15.40:0/1381873877 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f567c004300 con 0x7f568c106e70 2024-06-06T03:26:56.373 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.759+0000 7f568a7fc700 1 -- 172.21.15.40:0/1381873877 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f567c011b50 con 0x7f568c106e70 2024-06-06T03:26:56.373 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.760+0000 7f5691c82700 1 -- 172.21.15.40:0/1381873877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c106e70 msgr2=0x7f568c107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:26:56.373 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.760+0000 7f5691c82700 1 --2- 172.21.15.40:0/1381873877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c106e70 0x7f568c107290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f567c0098a0 tx=0x7f567c009bb0 comp rx=0 tx=0).stop 2024-06-06T03:26:56.373 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.760+0000 7f5691c82700 1 -- 172.21.15.40:0/1381873877 shutdown_connections 2024-06-06T03:26:56.373 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.760+0000 7f5691c82700 1 --2- 172.21.15.40:0/1381873877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c106e70 0x7f568c107290 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:26:56.374 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.760+0000 7f5691c82700 1 -- 172.21.15.40:0/1381873877 >> 172.21.15.40:0/1381873877 conn(0x7f568c1023f0 msgr2=0x7f568c104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:26:56.374 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.760+0000 7f5691c82700 1 -- 172.21.15.40:0/1381873877 shutdown_connections 2024-06-06T03:26:56.374 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.760+0000 7f5691c82700 1 -- 172.21.15.40:0/1381873877 wait complete. 2024-06-06T03:26:56.374 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.760+0000 7f5691c82700 1 Processor -- start 2024-06-06T03:26:56.374 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.761+0000 7f5691c82700 1 -- start start 2024-06-06T03:26:56.374 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.761+0000 7f5691c82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c106e70 0x7f568c07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:26:56.375 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.761+0000 7f5691c82700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f568c10b290 con 0x7f568c106e70 2024-06-06T03:26:56.375 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.761+0000 7f568b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c106e70 0x7f568c07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:26:56.375 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.761+0000 7f568b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c106e70 0x7f568c07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35068/0 (socket says 172.21.15.40:35068) 2024-06-06T03:26:56.375 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.761+0000 7f568b7fe700 1 -- 172.21.15.40:0/1614534415 learned_addr learned my addr 172.21.15.40:0/1614534415 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:26:56.375 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.761+0000 7f568b7fe700 1 -- 172.21.15.40:0/1614534415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c19eb80 con 0x7f568c106e70 2024-06-06T03:26:56.375 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.761+0000 7f568b7fe700 1 --2- 172.21.15.40:0/1614534415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c106e70 0x7f568c07c2b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f567c011b60 tx=0x7f567c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:26:56.375 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.761+0000 7f5688ff9700 1 -- 172.21.15.40:0/1614534415 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f567c003b20 con 0x7f568c106e70 2024-06-06T03:26:56.376 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.762+0000 7f5691c82700 1 -- 172.21.15.40:0/1614534415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568c19ed50 con 0x7f568c106e70 2024-06-06T03:26:56.376 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.762+0000 7f5688ff9700 1 -- 172.21.15.40:0/1614534415 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f567c003cc0 con 0x7f568c106e70 2024-06-06T03:26:56.376 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.762+0000 7f5688ff9700 1 -- 172.21.15.40:0/1614534415 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f567c024840 con 0x7f568c106e70 2024-06-06T03:26:56.376 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.762+0000 7f5691c82700 1 -- 172.21.15.40:0/1614534415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568c19f0f0 con 0x7f568c106e70 2024-06-06T03:26:56.376 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.762+0000 7f5688ff9700 1 -- 172.21.15.40:0/1614534415 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f567c022020 con 0x7f568c106e70 2024-06-06T03:26:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.762+0000 7f5688ff9700 1 -- 172.21.15.40:0/1614534415 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f567c011e00 con 0x7f568c106e70 2024-06-06T03:26:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.763+0000 7f5691c82700 1 -- 172.21.15.40:0/1614534415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f568c063af0 con 0x7f568c106e70 2024-06-06T03:26:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.765+0000 7f5688ff9700 1 -- 172.21.15.40:0/1614534415 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f567c043dd0 con 0x7f568c106e70 2024-06-06T03:26:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.836+0000 7f5691c82700 1 -- 172.21.15.40:0/1614534415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f568c19f530 con 0x7f568c106e70 2024-06-06T03:26:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.836+0000 7f5688ff9700 1 -- 172.21.15.40:0/1614534415 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f567c032030 con 0x7f568c106e70 2024-06-06T03:26:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.837+0000 7f5691c82700 1 -- 172.21.15.40:0/1614534415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c106e70 msgr2=0x7f568c07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:26:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.837+0000 7f5691c82700 1 --2- 172.21.15.40:0/1614534415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c106e70 0x7f568c07c2b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f567c011b60 tx=0x7f567c011c40 comp rx=0 tx=0).stop 2024-06-06T03:26:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.837+0000 7f5691c82700 1 -- 172.21.15.40:0/1614534415 shutdown_connections 2024-06-06T03:26:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.837+0000 7f5691c82700 1 --2- 172.21.15.40:0/1614534415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c106e70 0x7f568c07c2b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:26:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.837+0000 7f5691c82700 1 -- 172.21.15.40:0/1614534415 >> 172.21.15.40:0/1614534415 conn(0x7f568c1023f0 msgr2=0x7f568c102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:26:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.837+0000 7f5691c82700 1 -- 172.21.15.40:0/1614534415 shutdown_connections 2024-06-06T03:26:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:55.837+0000 7f5691c82700 1 -- 172.21.15.40:0/1614534415 wait complete. 2024-06-06T03:26:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:Restarting the monitor... 2024-06-06T03:26:56.391 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:56 smithi040 ceph-mon[31847]: from='client.? 172.21.15.40:0/1614534415' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:26:56.644 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:56 smithi040 systemd[1]: Stopping Ceph mon.a for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:26:57.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:56 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a[31843]: 2024-06-06T03:26:56.644+0000 7ff5df81f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:26:57.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:56 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a[31843]: 2024-06-06T03:26:56.644+0000 7ff5df81f700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-06T03:26:57.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:56 smithi040 podman[32094]: 2024-06-06 03:26:56.880467775 +0000 UTC m=+0.335950316 container died 0c05c2b9acd496f9a9df385b3ff3f5d8cb54f73e39c58d937ce9a79b9f3bc204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T03:26:57.486 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:57 smithi040 podman[32094]: 2024-06-06 03:26:57.228024557 +0000 UTC m=+0.683507093 container cleanup 0c05c2b9acd496f9a9df385b3ff3f5d8cb54f73e39c58d937ce9a79b9f3bc204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-06T03:26:57.486 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:57 smithi040 bash[32094]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a 2024-06-06T03:26:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:57 smithi040 podman[32106]: 2024-06-06 03:26:57.487214074 +0000 UTC m=+0.606290766 container remove 0c05c2b9acd496f9a9df385b3ff3f5d8cb54f73e39c58d937ce9a79b9f3bc204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T03:26:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:57 smithi040 systemd[1]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-06T03:26:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:57 smithi040 systemd[1]: Stopped Ceph mon.a for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:26:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:57 smithi040 systemd[1]: Starting Ceph mon.a for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:26:58.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:57 smithi040 podman[32201]: 2024-06-06 03:26:57.871435429 +0000 UTC m=+0.110099083 container create f7dcb07db357d152afe758de41ab9ed6bcd6a6e4896708ab56508ef10cefe9eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:26:58.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:57 smithi040 podman[32201]: 2024-06-06 03:26:57.778625311 +0000 UTC m=+0.017288987 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:26:58.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 podman[32201]: 2024-06-06 03:26:58.090334236 +0000 UTC m=+0.328997894 container init f7dcb07db357d152afe758de41ab9ed6bcd6a6e4896708ab56508ef10cefe9eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , ceph=True) 2024-06-06T03:26:58.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 podman[32201]: 2024-06-06 03:26:58.093655907 +0000 UTC m=+0.332319566 container start f7dcb07db357d152afe758de41ab9ed6bcd6a6e4896708ab56508ef10cefe9eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-06T03:26:58.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: set uid:gid to 167:167 (ceph:ceph) 2024-06-06T03:26:58.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-06T03:26:58.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: pidfile_write: ignore empty --pid-file 2024-06-06T03:26:58.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: load: jerasure load: lrc 2024-06-06T03:26:58.176 INFO:teuthology.orchestra.run.smithi040.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-06T03:26:58.405 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: RocksDB version: 7.9.2 2024-06-06T03:26:58.405 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Git sha 0 2024-06-06T03:26:58.405 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-06T03:26:58.405 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: DB SUMMARY 2024-06-06T03:26:58.405 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: DB Session ID: B1YALELWDRIMIGG3FKY4 2024-06-06T03:26:58.405 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: CURRENT file: CURRENT 2024-06-06T03:26:58.405 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: IDENTITY file: IDENTITY 2024-06-06T03:26:58.405 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-06T03:26:58.405 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-06T03:26:58.406 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104718 ; 2024-06-06T03:26:58.406 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.error_if_exists: 0 2024-06-06T03:26:58.406 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.create_if_missing: 0 2024-06-06T03:26:58.406 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.paranoid_checks: 1 2024-06-06T03:26:58.406 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-06T03:26:58.406 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-06T03:26:58.406 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-06T03:26:58.406 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.env: 0x5573c023dc60 2024-06-06T03:26:58.406 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.fs: PosixFileSystem 2024-06-06T03:26:58.406 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.info_log: 0x5573c14ebb20 2024-06-06T03:26:58.406 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_file_opening_threads: 16 2024-06-06T03:26:58.406 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.statistics: (nil) 2024-06-06T03:26:58.406 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.use_fsync: 0 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_log_file_size: 0 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.keep_log_file_num: 1000 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.recycle_log_file_num: 0 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.allow_fallocate: 1 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.allow_mmap_reads: 0 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.allow_mmap_writes: 0 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.use_direct_reads: 0 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.create_missing_column_families: 0 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.db_log_dir: 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.wal_dir: 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-06T03:26:58.407 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.advise_random_on_open: 1 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.db_write_buffer_size: 0 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.write_buffer_manager: 0x5573c14bc1e0 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.rate_limiter: (nil) 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.wal_recovery_mode: 2 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.enable_thread_tracking: 0 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.enable_pipelined_write: 0 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.unordered_write: 0 2024-06-06T03:26:58.408 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.row_cache: None 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.wal_filter: None 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.allow_ingest_behind: 0 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.two_write_queues: 0 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.manual_wal_flush: 0 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.wal_compression: 0 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.atomic_flush: 0 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-06T03:26:58.409 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.log_readahead_size: 0 2024-06-06T03:26:58.410 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-06T03:26:58.410 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.best_efforts_recovery: 0 2024-06-06T03:26:58.410 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-06T03:26:58.410 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-06T03:26:58.410 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.allow_data_in_errors: 0 2024-06-06T03:26:58.410 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.db_host_id: __hostname__ 2024-06-06T03:26:58.410 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-06T03:26:58.410 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_background_jobs: 2 2024-06-06T03:26:58.410 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_background_compactions: -1 2024-06-06T03:26:58.410 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_subcompactions: 1 2024-06-06T03:26:58.411 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-06T03:26:58.411 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-06T03:26:58.411 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_total_wal_size: 0 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_open_files: -1 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.bytes_per_sync: 0 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compaction_readahead_size: 0 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_background_flushes: -1 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Compression algorithms supported: 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: kZSTD supported: 0 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: kXpressCompression supported: 0 2024-06-06T03:26:58.412 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: kLZ4HCCompression supported: 1 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: kZlibCompression supported: 1 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: kSnappyCompression supported: 1 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: kLZ4Compression supported: 1 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: kBZip2Compression supported: 0 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.merge_operator: 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compaction_filter: None 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compaction_filter_factory: None 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.sst_partitioner_factory: None 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-06T03:26:58.413 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5573c14ebc40) 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: cache_index_and_filter_blocks: 1 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: pin_top_level_index_and_filter: 1 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: index_type: 0 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: data_block_index_type: 0 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: index_shortening: 1 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: checksum: 4 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: no_block_cache: 0 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_cache: 0x5573c14ed090 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_cache_name: BinnedLRUCache 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_cache_options: 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: capacity : 536870912 2024-06-06T03:26:58.414 INFO:journalctl@ceph.mon.a.smithi040.stdout: num_shard_bits : 4 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: strict_capacity_limit : 0 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: high_pri_pool_ratio: 0.000 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_cache_compressed: (nil) 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: persistent_cache: (nil) 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_size: 4096 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_size_deviation: 10 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_restart_interval: 16 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: index_block_restart_interval: 1 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: metadata_block_size: 4096 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: partition_filters: 0 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: use_delta_encoding: 1 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: filter_policy: bloomfilter 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: whole_key_filtering: 1 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: verify_compression: 0 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: read_amp_bytes_per_bit: 0 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: format_version: 5 2024-06-06T03:26:58.415 INFO:journalctl@ceph.mon.a.smithi040.stdout: enable_index_compression: 1 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_align: 0 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout: max_auto_readahead_size: 262144 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout: prepopulate_block_cache: 0 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout: initial_auto_readahead_size: 8192 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout: num_file_reads_for_auto_readahead: 2 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.write_buffer_size: 33554432 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_write_buffer_number: 2 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compression: NoCompression 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.bottommost_compression: Disabled 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.prefix_extractor: nullptr 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.num_levels: 7 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-06T03:26:58.416 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compression_opts.level: 32767 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compression_opts.strategy: 0 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-06T03:26:58.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compression_opts.enabled: false 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.target_file_size_base: 67108864 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-06T03:26:58.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.arena_block_size: 1048576 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.disable_auto_compactions: 0 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-06T03:26:58.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.table_properties_collectors: 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.inplace_update_support: 0 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.bloom_locality: 0 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.max_successive_merges: 0 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.paranoid_file_checks: 0 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.force_consistency_checks: 1 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.report_bg_io_stats: 0 2024-06-06T03:26:58.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.ttl: 2592000 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.enable_blob_files: false 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.min_blob_size: 0 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.blob_file_size: 268435456 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.blob_file_starting_level: 0 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e9ed99aa-6f9e-4a81-a8ed-094b3e0152f8 2024-06-06T03:26:58.421 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717644418124259, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-06T03:26:58.422 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-06T03:26:58.422 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717644418125249, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99807, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97719, "index_size": 306, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14165, "raw_average_key_size": 49, "raw_value_size": 90058, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717644418, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e9ed99aa-6f9e-4a81-a8ed-094b3e0152f8", "db_session_id": "B1YALELWDRIMIGG3FKY4", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-06T03:26:58.422 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717644418125338, "job": 1, "event": "recovery_finished"} 2024-06-06T03:26:58.422 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-06T03:26:58.422 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-06T03:26:58.422 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5573c15dc000 2024-06-06T03:26:58.422 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: DB pointer 0x5573c15c6000 2024-06-06T03:26:58.422 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-06T03:26:58.422 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-06T03:26:58.422 INFO:journalctl@ceph.mon.a.smithi040.stdout: ** DB Stats ** 2024-06-06T03:26:58.422 INFO:journalctl@ceph.mon.a.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-06T03:26:58.422 INFO:journalctl@ceph.mon.a.smithi040.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-06T03:26:58.422 INFO:journalctl@ceph.mon.a.smithi040.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-06T03:26:58.422 INFO:journalctl@ceph.mon.a.smithi040.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: ** Compaction Stats [default] ** 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: L0 2/0 99.30 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 167.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: Sum 2/0 99.30 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 167.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 167.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: ** Compaction Stats [default] ** 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 167.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T03:26:58.423 INFO:journalctl@ceph.mon.a.smithi040.stdout: 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: Cumulative compaction: 0.00 GB write, 31.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: Interval compaction: 0.00 GB write, 31.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: Block cache BinnedLRUCache@0x5573c14ed090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: 2024-06-06T03:26:58.424 INFO:journalctl@ceph.mon.a.smithi040.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: starting mon.a rank 0 at public addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] at bind addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: mon.a@-1(???) e1 preinit fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: mon.a@-1(???).mds e1 new map 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: mon.a@-1(???).mds e1 print_map 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout: e1 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout: legacy client fscid: -1 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout: 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout: No filesystems configured 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-06T03:26:58.425 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 bash[32201]: f7dcb07db357d152afe758de41ab9ed6bcd6a6e4896708ab56508ef10cefe9eb 2024-06-06T03:26:58.426 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 systemd[1]: Started Ceph mon.a for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:26:58.426 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-06T03:26:58.426 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: monmap epoch 1 2024-06-06T03:26:58.426 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:26:58.426 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: last_changed 2024-06-06T03:26:49.934047+0000 2024-06-06T03:26:58.426 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: created 2024-06-06T03:26:49.934047+0000 2024-06-06T03:26:58.426 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: min_mon_release 19 (squid) 2024-06-06T03:26:58.426 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: election_strategy: 1 2024-06-06T03:26:58.426 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-06-06T03:26:58.426 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: fsmap 2024-06-06T03:26:58.426 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: osdmap e1: 0 total, 0 up, 0 in 2024-06-06T03:26:58.426 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:58 smithi040 ceph-mon[32220]: mgrmap e1: no daemons active 2024-06-06T03:26:59.186 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.625+0000 7f580116e700 1 Processor -- start 2024-06-06T03:26:59.186 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.625+0000 7f580116e700 1 -- start start 2024-06-06T03:26:59.186 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.625+0000 7f580116e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57fc108a10 0x7f57fc10ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:26:59.186 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.625+0000 7f580116e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc07c710 con 0x7f57fc108a10 2024-06-06T03:26:59.186 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.625+0000 7f57fad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57fc108a10 0x7f57fc10ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:26:59.186 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.625+0000 7f57fad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57fc108a10 0x7f57fc10ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35078/0 (socket says 172.21.15.40:35078) 2024-06-06T03:26:59.186 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.625+0000 7f57fad9d700 1 -- 172.21.15.40:0/3975602659 learned_addr learned my addr 172.21.15.40:0/3975602659 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:26:59.186 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.626+0000 7f57fad9d700 1 -- 172.21.15.40:0/3975602659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57fc10b340 con 0x7f57fc108a10 2024-06-06T03:26:59.186 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.626+0000 7f57fad9d700 1 --2- 172.21.15.40:0/3975602659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57fc108a10 0x7f57fc10ae00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0098a0 tx=0x7f57ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bd56746bfa5c31 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.626+0000 7f57f9d9b700 1 -- 172.21.15.40:0/3975602659 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57ec004160 con 0x7f57fc108a10 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.626+0000 7f57f9d9b700 1 -- 172.21.15.40:0/3975602659 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f57ec004300 con 0x7f57fc108a10 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.626+0000 7f57f9d9b700 1 -- 172.21.15.40:0/3975602659 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57ec011b50 con 0x7f57fc108a10 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.627+0000 7f580116e700 1 -- 172.21.15.40:0/3975602659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57fc108a10 msgr2=0x7f57fc10ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.627+0000 7f580116e700 1 --2- 172.21.15.40:0/3975602659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57fc108a10 0x7f57fc10ae00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0098a0 tx=0x7f57ec009bb0 comp rx=0 tx=0).stop 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.627+0000 7f580116e700 1 -- 172.21.15.40:0/3975602659 shutdown_connections 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.627+0000 7f580116e700 1 --2- 172.21.15.40:0/3975602659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57fc108a10 0x7f57fc10ae00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.627+0000 7f580116e700 1 -- 172.21.15.40:0/3975602659 >> 172.21.15.40:0/3975602659 conn(0x7f57fc102440 msgr2=0x7f57fc104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.627+0000 7f580116e700 1 -- 172.21.15.40:0/3975602659 shutdown_connections 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.627+0000 7f580116e700 1 -- 172.21.15.40:0/3975602659 wait complete. 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.627+0000 7f580116e700 1 Processor -- start 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.628+0000 7f580116e700 1 -- start start 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.628+0000 7f580116e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57fc108a10 0x7f57fc108240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.628+0000 7f580116e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc10c290 con 0x7f57fc108a10 2024-06-06T03:26:59.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.628+0000 7f57fad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57fc108a10 0x7f57fc108240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:26:59.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.628+0000 7f57fad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57fc108a10 0x7f57fc108240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35092/0 (socket says 172.21.15.40:35092) 2024-06-06T03:26:59.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.628+0000 7f57fad9d700 1 -- 172.21.15.40:0/1635238525 learned_addr learned my addr 172.21.15.40:0/1635238525 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:26:59.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.628+0000 7f57fad9d700 1 -- 172.21.15.40:0/1635238525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57fc108780 con 0x7f57fc108a10 2024-06-06T03:26:59.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.628+0000 7f57fad9d700 1 --2- 172.21.15.40:0/1635238525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57fc108a10 0x7f57fc108240 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009fb0 tx=0x7f57ec011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:26:59.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.628+0000 7f57e3fff700 1 -- 172.21.15.40:0/1635238525 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57ec0038b0 con 0x7f57fc108a10 2024-06-06T03:26:59.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.629+0000 7f57e3fff700 1 -- 172.21.15.40:0/1635238525 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f57ec011e50 con 0x7f57fc108a10 2024-06-06T03:26:59.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.629+0000 7f580116e700 1 -- 172.21.15.40:0/1635238525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57fc1069c0 con 0x7f57fc108a10 2024-06-06T03:26:59.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.629+0000 7f57e3fff700 1 -- 172.21.15.40:0/1635238525 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57ec019420 con 0x7f57fc108a10 2024-06-06T03:26:59.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.629+0000 7f580116e700 1 -- 172.21.15.40:0/1635238525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57fc106ea0 con 0x7f57fc108a10 2024-06-06T03:26:59.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.629+0000 7f57e3fff700 1 -- 172.21.15.40:0/1635238525 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f57ec019ad0 con 0x7f57fc108a10 2024-06-06T03:26:59.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.629+0000 7f57e3fff700 1 -- 172.21.15.40:0/1635238525 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f57ec024730 con 0x7f57fc108a10 2024-06-06T03:26:59.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.630+0000 7f580116e700 1 -- 172.21.15.40:0/1635238525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57fc063af0 con 0x7f57fc108a10 2024-06-06T03:26:59.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.632+0000 7f57e3fff700 1 -- 172.21.15.40:0/1635238525 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f57ec043dd0 con 0x7f57fc108a10 2024-06-06T03:26:59.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.702+0000 7f580116e700 1 -- 172.21.15.40:0/1635238525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f57fc107120 con 0x7f57fc108a10 2024-06-06T03:26:59.189 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.704+0000 7f57e3fff700 1 -- 172.21.15.40:0/1635238525 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f57ec032030 con 0x7f57fc108a10 2024-06-06T03:26:59.189 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.704+0000 7f580116e700 1 -- 172.21.15.40:0/1635238525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57fc108a10 msgr2=0x7f57fc108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:26:59.189 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.704+0000 7f580116e700 1 --2- 172.21.15.40:0/1635238525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57fc108a10 0x7f57fc108240 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009fb0 tx=0x7f57ec011bf0 comp rx=0 tx=0).stop 2024-06-06T03:26:59.189 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.705+0000 7f580116e700 1 -- 172.21.15.40:0/1635238525 shutdown_connections 2024-06-06T03:26:59.189 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.705+0000 7f580116e700 1 --2- 172.21.15.40:0/1635238525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57fc108a10 0x7f57fc108240 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:26:59.189 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.705+0000 7f580116e700 1 -- 172.21.15.40:0/1635238525 >> 172.21.15.40:0/1635238525 conn(0x7f57fc102440 msgr2=0x7f57fc102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:26:59.189 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.705+0000 7f580116e700 1 -- 172.21.15.40:0/1635238525 shutdown_connections 2024-06-06T03:26:59.189 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:26:58.705+0000 7f580116e700 1 -- 172.21.15.40:0/1635238525 wait complete. 2024-06-06T03:26:59.189 INFO:teuthology.orchestra.run.smithi040.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-06T03:26:59.211 INFO:teuthology.orchestra.run.smithi040.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-06T03:26:59.211 INFO:teuthology.orchestra.run.smithi040.stdout:Creating mgr... 2024-06-06T03:26:59.212 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-06T03:26:59.212 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-06T03:26:59.466 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.x 2024-06-06T03:26:59.467 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Failed to reset failed state of unit ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-06-06T03:26:59.615 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-77c25d68-23b4-11ef-bc9b-c7b262605968.target.wants/ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@.service. 2024-06-06T03:26:59.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:26:59 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1635238525' entity='client.admin' 2024-06-06T03:27:00.125 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:00 smithi040 podman[32434]: 2024-06-06 03:27:00.070792372 +0000 UTC m=+0.326935546 container start 941bccf24d8aa9c112faa180156dc093987316da6f59d3f88d9814def7570238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-06T03:27:00.125 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:00 smithi040 bash[32434]: 941bccf24d8aa9c112faa180156dc093987316da6f59d3f88d9814def7570238 2024-06-06T03:27:00.139 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T03:27:00.139 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout disabled 2024-06-06T03:27:00.148 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T03:27:00.148 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2024-06-06T03:27:00.148 INFO:teuthology.orchestra.run.smithi040.stdout:firewalld.service is not enabled 2024-06-06T03:27:00.148 INFO:teuthology.orchestra.run.smithi040.stdout:Not possible to enable service . firewalld.service is not available 2024-06-06T03:27:00.156 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T03:27:00.156 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout disabled 2024-06-06T03:27:00.164 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T03:27:00.164 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2024-06-06T03:27:00.164 INFO:teuthology.orchestra.run.smithi040.stdout:firewalld.service is not enabled 2024-06-06T03:27:00.164 INFO:teuthology.orchestra.run.smithi040.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-06T03:27:00.164 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mgr to start... 2024-06-06T03:27:00.164 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mgr... 2024-06-06T03:27:00.393 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:00 smithi040 systemd[1]: Started Ceph mgr.x for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:27:00.393 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:00.229+0000 7fb8e497c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T03:27:00.673 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:00.427+0000 7fb8e497c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:27:00.673 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:00.535+0000 7fb8e497c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T03:27:00.957 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:00 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2321279038' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T03:27:00.957 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:00.867+0000 7fb8e497c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T03:27:01.222 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:00.957+0000 7fb8e497c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T03:27:01.225 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-06-06T03:27:01.227 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-06-06T03:27:01.227 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:27:01.227 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T03:27:01.227 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T03:27:01.227 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T03:27:01.228 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T03:27:01.228 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:01.228 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T03:27:01.228 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T03:27:01.228 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 0 2024-06-06T03:27:01.228 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-06-06T03:27:01.228 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T03:27:01.228 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T03:27:01.228 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-06-06T03:27:01.228 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-06-06T03:27:01.228 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T03:27:01.228 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:01.228 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T03:27:01.228 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T03:27:01.229 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:01.229 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T03:27:01.229 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:01.229 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T03:27:01.229 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T03:27:01.236 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T03:27:01.236 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T03:27:01.236 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T03:27:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T03:27:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T03:27:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T03:27:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T03:27:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T03:27:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T03:27:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T03:27:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T03:27:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T03:27:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T03:27:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T03:27:01.237 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:01.238 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T03:27:01.238 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T03:27:01.238 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:01.238 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T03:27:01.238 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-06T03:27:01.238 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T03:27:01.238 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T03:27:01.238 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T03:27:01.238 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T03:27:01.238 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T03:27:01.238 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-06-06T03:27:01.238 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T03:27:01.238 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:01.239 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T03:27:01.239 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:01.239 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T03:26:53.165952+0000", 2024-06-06T03:27:01.239 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T03:27:01.239 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:01.239 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T03:27:01.239 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-06-06T03:27:01.239 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.591+0000 7fb22ccf4700 1 Processor -- start 2024-06-06T03:27:01.239 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.591+0000 7fb22ccf4700 1 -- start start 2024-06-06T03:27:01.239 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.591+0000 7fb22ccf4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb228108a50 0x7fb22810ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:01.239 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.591+0000 7fb22ccf4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb22807be40 con 0x7fb228108a50 2024-06-06T03:27:01.239 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.591+0000 7fb22659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb228108a50 0x7fb22810ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:01.239 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.591+0000 7fb22659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb228108a50 0x7fb22810ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40178/0 (socket says 172.21.15.40:40178) 2024-06-06T03:27:01.239 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.591+0000 7fb22659c700 1 -- 172.21.15.40:0/3195842638 learned_addr learned my addr 172.21.15.40:0/3195842638 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:01.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.592+0000 7fb22659c700 1 -- 172.21.15.40:0/3195842638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22810b380 con 0x7fb228108a50 2024-06-06T03:27:01.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.592+0000 7fb22659c700 1 --2- 172.21.15.40:0/3195842638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb228108a50 0x7fb22810ae40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb2180098a0 tx=0x7fb218009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8253b9eb04595028 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:01.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.592+0000 7fb22559a700 1 -- 172.21.15.40:0/3195842638 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb218004160 con 0x7fb228108a50 2024-06-06T03:27:01.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.592+0000 7fb22559a700 1 -- 172.21.15.40:0/3195842638 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb218004300 con 0x7fb228108a50 2024-06-06T03:27:01.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.592+0000 7fb22559a700 1 -- 172.21.15.40:0/3195842638 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb218011b50 con 0x7fb228108a50 2024-06-06T03:27:01.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.593+0000 7fb22ccf4700 1 -- 172.21.15.40:0/3195842638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb228108a50 msgr2=0x7fb22810ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:01.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.593+0000 7fb22ccf4700 1 --2- 172.21.15.40:0/3195842638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb228108a50 0x7fb22810ae40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb2180098a0 tx=0x7fb218009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:01.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.593+0000 7fb22ccf4700 1 -- 172.21.15.40:0/3195842638 shutdown_connections 2024-06-06T03:27:01.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.593+0000 7fb22ccf4700 1 --2- 172.21.15.40:0/3195842638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb228108a50 0x7fb22810ae40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:01.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.593+0000 7fb22ccf4700 1 -- 172.21.15.40:0/3195842638 >> 172.21.15.40:0/3195842638 conn(0x7fb228102440 msgr2=0x7fb228104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:01.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.593+0000 7fb22ccf4700 1 -- 172.21.15.40:0/3195842638 shutdown_connections 2024-06-06T03:27:01.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.593+0000 7fb22ccf4700 1 -- 172.21.15.40:0/3195842638 wait complete. 2024-06-06T03:27:01.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.594+0000 7fb22ccf4700 1 Processor -- start 2024-06-06T03:27:01.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.594+0000 7fb22ccf4700 1 -- start start 2024-06-06T03:27:01.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.594+0000 7fb22ccf4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb228108a50 0x7fb22819bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:01.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.594+0000 7fb22ccf4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb22810c2c0 con 0x7fb228108a50 2024-06-06T03:27:01.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.595+0000 7fb22659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb228108a50 0x7fb22819bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:01.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.595+0000 7fb22659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb228108a50 0x7fb22819bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40182/0 (socket says 172.21.15.40:40182) 2024-06-06T03:27:01.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.595+0000 7fb22659c700 1 -- 172.21.15.40:0/2321279038 learned_addr learned my addr 172.21.15.40:0/2321279038 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:01.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.595+0000 7fb22659c700 1 -- 172.21.15.40:0/2321279038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22819c440 con 0x7fb228108a50 2024-06-06T03:27:01.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.595+0000 7fb22659c700 1 --2- 172.21.15.40:0/2321279038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb228108a50 0x7fb22819bf00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb218011b60 tx=0x7fb218011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:01.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.595+0000 7fb2177fe700 1 -- 172.21.15.40:0/2321279038 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb218003970 con 0x7fb228108a50 2024-06-06T03:27:01.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.595+0000 7fb22ccf4700 1 -- 172.21.15.40:0/2321279038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22819c6d0 con 0x7fb228108a50 2024-06-06T03:27:01.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.595+0000 7fb2177fe700 1 -- 172.21.15.40:0/2321279038 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb218003b10 con 0x7fb228108a50 2024-06-06T03:27:01.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.595+0000 7fb2177fe700 1 -- 172.21.15.40:0/2321279038 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb218024610 con 0x7fb228108a50 2024-06-06T03:27:01.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.595+0000 7fb22ccf4700 1 -- 172.21.15.40:0/2321279038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22819f3c0 con 0x7fb228108a50 2024-06-06T03:27:01.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.595+0000 7fb2177fe700 1 -- 172.21.15.40:0/2321279038 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb218024ca0 con 0x7fb228108a50 2024-06-06T03:27:01.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.596+0000 7fb2177fe700 1 -- 172.21.15.40:0/2321279038 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb218011e00 con 0x7fb228108a50 2024-06-06T03:27:01.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.596+0000 7fb22ccf4700 1 -- 172.21.15.40:0/2321279038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb228063af0 con 0x7fb228108a50 2024-06-06T03:27:01.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.598+0000 7fb2177fe700 1 -- 172.21.15.40:0/2321279038 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb218043dd0 con 0x7fb228108a50 2024-06-06T03:27:01.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.673+0000 7fb22ccf4700 1 -- 172.21.15.40:0/2321279038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb22819f590 con 0x7fb228108a50 2024-06-06T03:27:01.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.673+0000 7fb2177fe700 1 -- 172.21.15.40:0/2321279038 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb218032030 con 0x7fb228108a50 2024-06-06T03:27:01.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.674+0000 7fb22ccf4700 1 -- 172.21.15.40:0/2321279038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb228108a50 msgr2=0x7fb22819bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:01.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.674+0000 7fb22ccf4700 1 --2- 172.21.15.40:0/2321279038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb228108a50 0x7fb22819bf00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb218011b60 tx=0x7fb218011c40 comp rx=0 tx=0).stop 2024-06-06T03:27:01.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.674+0000 7fb22ccf4700 1 -- 172.21.15.40:0/2321279038 shutdown_connections 2024-06-06T03:27:01.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.674+0000 7fb22ccf4700 1 --2- 172.21.15.40:0/2321279038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb228108a50 0x7fb22819bf00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:01.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.674+0000 7fb22ccf4700 1 -- 172.21.15.40:0/2321279038 >> 172.21.15.40:0/2321279038 conn(0x7fb228102440 msgr2=0x7fb228102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:01.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.674+0000 7fb22ccf4700 1 -- 172.21.15.40:0/2321279038 shutdown_connections 2024-06-06T03:27:01.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:00.674+0000 7fb22ccf4700 1 -- 172.21.15.40:0/2321279038 wait complete. 2024-06-06T03:27:01.242 INFO:teuthology.orchestra.run.smithi040.stdout:mgr not available, waiting (1/15)... 2024-06-06T03:27:02.571 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:02 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:02.221+0000 7fb8e497c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T03:27:02.572 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:02 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:02.396+0000 7fb8e497c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T03:27:02.572 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:02 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:02.571+0000 7fb8e497c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T03:27:03.175 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:02 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:02.870+0000 7fb8e497c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T03:27:03.175 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:02 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:02.966+0000 7fb8e497c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T03:27:03.175 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:03 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:03.175+0000 7fb8e497c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T03:27:03.490 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:03 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:03.384+0000 7fb8e497c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:27:04.026 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:03 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1697700793' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T03:27:04.277 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-06-06T03:27:04.277 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-06-06T03:27:04.277 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:27:04.277 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T03:27:04.277 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T03:27:04.277 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T03:27:04.277 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T03:27:04.278 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:04.278 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T03:27:04.278 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T03:27:04.278 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 0 2024-06-06T03:27:04.278 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-06-06T03:27:04.278 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T03:27:04.278 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T03:27:04.278 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-06-06T03:27:04.278 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-06-06T03:27:04.278 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T03:27:04.278 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:04.279 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T03:27:04.279 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T03:27:04.279 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:04.279 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T03:27:04.279 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:04.279 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T03:27:04.279 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T03:27:04.279 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T03:27:04.279 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T03:27:04.279 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T03:27:04.279 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T03:27:04.279 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:04.279 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T03:27:04.279 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T03:27:04.280 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T03:27:04.280 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T03:27:04.280 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T03:27:04.280 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T03:27:04.280 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T03:27:04.280 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T03:27:04.280 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T03:27:04.280 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:04.280 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T03:27:04.280 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:04.280 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T03:27:04.280 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T03:27:04.280 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:04.281 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T03:27:04.281 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-06T03:27:04.281 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T03:27:04.281 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T03:27:04.281 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T03:27:04.281 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T03:27:04.281 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T03:27:04.281 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-06-06T03:27:04.281 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T03:27:04.281 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:04.281 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T03:27:04.282 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:04.282 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T03:26:53.165952+0000", 2024-06-06T03:27:04.282 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T03:27:04.282 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:04.282 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T03:27:04.282 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-06-06T03:27:04.282 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.686+0000 7f445fdd1700 1 Processor -- start 2024-06-06T03:27:04.282 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.686+0000 7f445fdd1700 1 -- start start 2024-06-06T03:27:04.282 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.686+0000 7f445fdd1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4458106e50 0x7f4458107270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:04.282 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.686+0000 7f445fdd1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f44581077b0 con 0x7f4458106e50 2024-06-06T03:27:04.282 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.686+0000 7f445db6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4458106e50 0x7f4458107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:04.282 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.686+0000 7f445db6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4458106e50 0x7f4458107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40186/0 (socket says 172.21.15.40:40186) 2024-06-06T03:27:04.283 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.686+0000 7f445db6d700 1 -- 172.21.15.40:0/3737894242 learned_addr learned my addr 172.21.15.40:0/3737894242 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:04.283 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.686+0000 7f445db6d700 1 -- 172.21.15.40:0/3737894242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4458108040 con 0x7f4458106e50 2024-06-06T03:27:04.283 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.687+0000 7f445db6d700 1 --2- 172.21.15.40:0/3737894242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4458106e50 0x7f4458107270 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f44540098a0 tx=0x7f4454009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b3b4def981b6a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:04.283 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.687+0000 7f445cb6b700 1 -- 172.21.15.40:0/3737894242 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4454004160 con 0x7f4458106e50 2024-06-06T03:27:04.283 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.687+0000 7f445cb6b700 1 -- 172.21.15.40:0/3737894242 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4454004300 con 0x7f4458106e50 2024-06-06T03:27:04.283 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.687+0000 7f445cb6b700 1 -- 172.21.15.40:0/3737894242 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4454011b50 con 0x7f4458106e50 2024-06-06T03:27:04.283 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.687+0000 7f445fdd1700 1 -- 172.21.15.40:0/3737894242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4458106e50 msgr2=0x7f4458107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:04.283 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.687+0000 7f445fdd1700 1 --2- 172.21.15.40:0/3737894242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4458106e50 0x7f4458107270 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f44540098a0 tx=0x7f4454009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:04.283 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.687+0000 7f445fdd1700 1 -- 172.21.15.40:0/3737894242 shutdown_connections 2024-06-06T03:27:04.283 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.687+0000 7f445fdd1700 1 --2- 172.21.15.40:0/3737894242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4458106e50 0x7f4458107270 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:04.283 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.687+0000 7f445fdd1700 1 -- 172.21.15.40:0/3737894242 >> 172.21.15.40:0/3737894242 conn(0x7f4458102440 msgr2=0x7f4458104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:04.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.687+0000 7f445fdd1700 1 -- 172.21.15.40:0/3737894242 shutdown_connections 2024-06-06T03:27:04.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.688+0000 7f445fdd1700 1 -- 172.21.15.40:0/3737894242 wait complete. 2024-06-06T03:27:04.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.688+0000 7f445fdd1700 1 Processor -- start 2024-06-06T03:27:04.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.688+0000 7f445fdd1700 1 -- start start 2024-06-06T03:27:04.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.688+0000 7f445fdd1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4458106e50 0x7f445819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:04.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.688+0000 7f445fdd1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4458107930 con 0x7f4458106e50 2024-06-06T03:27:04.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.688+0000 7f445db6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4458106e50 0x7f445819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:04.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.688+0000 7f445db6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4458106e50 0x7f445819ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40192/0 (socket says 172.21.15.40:40192) 2024-06-06T03:27:04.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.688+0000 7f445db6d700 1 -- 172.21.15.40:0/1697700793 learned_addr learned my addr 172.21.15.40:0/1697700793 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:04.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.689+0000 7f445db6d700 1 -- 172.21.15.40:0/1697700793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445819b3a0 con 0x7f4458106e50 2024-06-06T03:27:04.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.689+0000 7f445db6d700 1 --2- 172.21.15.40:0/1697700793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4458106e50 0x7f445819ae60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4454011670 tx=0x7f4454004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:04.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.689+0000 7f444effd700 1 -- 172.21.15.40:0/1697700793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4454003c60 con 0x7f4458106e50 2024-06-06T03:27:04.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.689+0000 7f445fdd1700 1 -- 172.21.15.40:0/1697700793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445819b630 con 0x7f4458106e50 2024-06-06T03:27:04.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.689+0000 7f444effd700 1 -- 172.21.15.40:0/1697700793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4454020070 con 0x7f4458106e50 2024-06-06T03:27:04.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.689+0000 7f444effd700 1 -- 172.21.15.40:0/1697700793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f445401b810 con 0x7f4458106e50 2024-06-06T03:27:04.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.689+0000 7f445fdd1700 1 -- 172.21.15.40:0/1697700793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445807afa0 con 0x7f4458106e50 2024-06-06T03:27:04.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.689+0000 7f444effd700 1 -- 172.21.15.40:0/1697700793 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f445401bad0 con 0x7f4458106e50 2024-06-06T03:27:04.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.689+0000 7f444effd700 1 -- 172.21.15.40:0/1697700793 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4454011e00 con 0x7f4458106e50 2024-06-06T03:27:04.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.690+0000 7f445fdd1700 1 -- 172.21.15.40:0/1697700793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4458063af0 con 0x7f4458106e50 2024-06-06T03:27:04.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.692+0000 7f444effd700 1 -- 172.21.15.40:0/1697700793 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f445401cdb0 con 0x7f4458106e50 2024-06-06T03:27:04.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.766+0000 7f445fdd1700 1 -- 172.21.15.40:0/1697700793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f445807b5a0 con 0x7f4458106e50 2024-06-06T03:27:04.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.767+0000 7f444effd700 1 -- 172.21.15.40:0/1697700793 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4454025bc0 con 0x7f4458106e50 2024-06-06T03:27:04.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.768+0000 7f445fdd1700 1 -- 172.21.15.40:0/1697700793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4458106e50 msgr2=0x7f445819ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:04.286 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.768+0000 7f445fdd1700 1 --2- 172.21.15.40:0/1697700793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4458106e50 0x7f445819ae60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4454011670 tx=0x7f4454004260 comp rx=0 tx=0).stop 2024-06-06T03:27:04.286 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.768+0000 7f445fdd1700 1 -- 172.21.15.40:0/1697700793 shutdown_connections 2024-06-06T03:27:04.286 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.768+0000 7f445fdd1700 1 --2- 172.21.15.40:0/1697700793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4458106e50 0x7f445819ae60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:04.286 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.768+0000 7f445fdd1700 1 -- 172.21.15.40:0/1697700793 >> 172.21.15.40:0/1697700793 conn(0x7f4458102440 msgr2=0x7f4458102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:04.286 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.768+0000 7f445fdd1700 1 -- 172.21.15.40:0/1697700793 shutdown_connections 2024-06-06T03:27:04.286 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:03.768+0000 7f445fdd1700 1 -- 172.21.15.40:0/1697700793 wait complete. 2024-06-06T03:27:04.286 INFO:teuthology.orchestra.run.smithi040.stdout:mgr not available, waiting (2/15)... 2024-06-06T03:27:04.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:04 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:04.026+0000 7fb8e497c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T03:27:04.778 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:04 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:04.470+0000 7fb8e497c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T03:27:04.778 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:04 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:04.570+0000 7fb8e497c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T03:27:04.779 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:04 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:04.778+0000 7fb8e497c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T03:27:05.580 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:05 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:05.323+0000 7fb8e497c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T03:27:05.580 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:05 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:05.419+0000 7fb8e497c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T03:27:06.031 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:05 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:05.673+0000 7fb8e497c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T03:27:06.031 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:05 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:05.813+0000 7fb8e497c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T03:27:06.031 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:05 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:05.911+0000 7fb8e497c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T03:27:06.031 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:06 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:06.031+0000 7fb8e497c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T03:27:06.615 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:06 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:06.346+0000 7fb8e497c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T03:27:06.986 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:06 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/782246211' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T03:27:07.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:06 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:06.985+0000 7fb8e497c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T03:27:07.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:07 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:07.209+0000 7fb8e497c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T03:27:07.347 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-06-06T03:27:07.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-06-06T03:27:07.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:27:07.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T03:27:07.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T03:27:07.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T03:27:07.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T03:27:07.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:07.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T03:27:07.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T03:27:07.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 0 2024-06-06T03:27:07.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-06-06T03:27:07.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T03:27:07.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T03:27:07.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-06-06T03:27:07.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-06-06T03:27:07.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T03:27:07.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:07.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T03:27:07.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T03:27:07.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:07.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T03:27:07.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:07.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T03:27:07.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T03:27:07.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T03:27:07.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T03:27:07.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T03:27:07.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T03:27:07.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:07.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T03:27:07.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T03:27:07.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T03:27:07.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T03:27:07.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T03:27:07.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T03:27:07.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T03:27:07.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T03:27:07.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T03:27:07.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:07.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T03:27:07.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T03:27:07.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:07.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T03:26:53.165952+0000", 2024-06-06T03:27:07.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T03:27:07.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:07.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T03:27:07.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-06-06T03:27:07.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.717+0000 7f7f4f5cc700 1 Processor -- start 2024-06-06T03:27:07.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.717+0000 7f7f4f5cc700 1 -- start start 2024-06-06T03:27:07.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.717+0000 7f7f4f5cc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f48107f80 0x7f7f481083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:07.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.717+0000 7f7f4f5cc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7f481088e0 con 0x7f7f48107f80 2024-06-06T03:27:07.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.718+0000 7f7f4d368700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f48107f80 0x7f7f481083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:07.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.718+0000 7f7f4d368700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f48107f80 0x7f7f481083a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40202/0 (socket says 172.21.15.40:40202) 2024-06-06T03:27:07.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.718+0000 7f7f4d368700 1 -- 172.21.15.40:0/1809988674 learned_addr learned my addr 172.21.15.40:0/1809988674 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:07.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.718+0000 7f7f4d368700 1 -- 172.21.15.40:0/1809988674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f48109090 con 0x7f7f48107f80 2024-06-06T03:27:07.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.718+0000 7f7f4d368700 1 --2- 172.21.15.40:0/1809988674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f48107f80 0x7f7f481083a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f440098a0 tx=0x7f7f44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a965933bf23009 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.718+0000 7f7f3bfff700 1 -- 172.21.15.40:0/1809988674 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f44004160 con 0x7f7f48107f80 2024-06-06T03:27:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.718+0000 7f7f3bfff700 1 -- 172.21.15.40:0/1809988674 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7f44004300 con 0x7f7f48107f80 2024-06-06T03:27:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.718+0000 7f7f3bfff700 1 -- 172.21.15.40:0/1809988674 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f44011b50 con 0x7f7f48107f80 2024-06-06T03:27:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.719+0000 7f7f4f5cc700 1 -- 172.21.15.40:0/1809988674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f48107f80 msgr2=0x7f7f481083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.719+0000 7f7f4f5cc700 1 --2- 172.21.15.40:0/1809988674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f48107f80 0x7f7f481083a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f440098a0 tx=0x7f7f44009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.719+0000 7f7f4f5cc700 1 -- 172.21.15.40:0/1809988674 shutdown_connections 2024-06-06T03:27:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.719+0000 7f7f4f5cc700 1 --2- 172.21.15.40:0/1809988674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f48107f80 0x7f7f481083a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.719+0000 7f7f4f5cc700 1 -- 172.21.15.40:0/1809988674 >> 172.21.15.40:0/1809988674 conn(0x7f7f48103500 msgr2=0x7f7f48105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.719+0000 7f7f4f5cc700 1 -- 172.21.15.40:0/1809988674 shutdown_connections 2024-06-06T03:27:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.719+0000 7f7f4f5cc700 1 -- 172.21.15.40:0/1809988674 wait complete. 2024-06-06T03:27:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.720+0000 7f7f4f5cc700 1 Processor -- start 2024-06-06T03:27:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.720+0000 7f7f4f5cc700 1 -- start start 2024-06-06T03:27:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.720+0000 7f7f4f5cc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f48107f80 0x7f7f4819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.720+0000 7f7f4f5cc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810c350 con 0x7f7f48107f80 2024-06-06T03:27:07.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.720+0000 7f7f4d368700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f48107f80 0x7f7f4819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:07.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.720+0000 7f7f4d368700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f48107f80 0x7f7f4819be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40218/0 (socket says 172.21.15.40:40218) 2024-06-06T03:27:07.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.720+0000 7f7f4d368700 1 -- 172.21.15.40:0/782246211 learned_addr learned my addr 172.21.15.40:0/782246211 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:07.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.721+0000 7f7f4d368700 1 -- 172.21.15.40:0/782246211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4819c3a0 con 0x7f7f48107f80 2024-06-06T03:27:07.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.721+0000 7f7f4d368700 1 --2- 172.21.15.40:0/782246211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f48107f80 0x7f7f4819be60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f44011670 tx=0x7f7f44011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:07.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.721+0000 7f7f3a7fc700 1 -- 172.21.15.40:0/782246211 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f440039c0 con 0x7f7f48107f80 2024-06-06T03:27:07.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.721+0000 7f7f3a7fc700 1 -- 172.21.15.40:0/782246211 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7f44011e50 con 0x7f7f48107f80 2024-06-06T03:27:07.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.721+0000 7f7f4f5cc700 1 -- 172.21.15.40:0/782246211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4819c630 con 0x7f7f48107f80 2024-06-06T03:27:07.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.721+0000 7f7f3a7fc700 1 -- 172.21.15.40:0/782246211 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f4401a5c0 con 0x7f7f48107f80 2024-06-06T03:27:07.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.721+0000 7f7f4f5cc700 1 -- 172.21.15.40:0/782246211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4819f320 con 0x7f7f48107f80 2024-06-06T03:27:07.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.721+0000 7f7f3a7fc700 1 -- 172.21.15.40:0/782246211 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7f4401ac50 con 0x7f7f48107f80 2024-06-06T03:27:07.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.721+0000 7f7f3a7fc700 1 -- 172.21.15.40:0/782246211 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7f44019700 con 0x7f7f48107f80 2024-06-06T03:27:07.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.722+0000 7f7f4f5cc700 1 -- 172.21.15.40:0/782246211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f48063af0 con 0x7f7f48107f80 2024-06-06T03:27:07.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.726+0000 7f7f3a7fc700 1 -- 172.21.15.40:0/782246211 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7f44003b60 con 0x7f7f48107f80 2024-06-06T03:27:07.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.800+0000 7f7f4f5cc700 1 -- 172.21.15.40:0/782246211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7f4819f4f0 con 0x7f7f48107f80 2024-06-06T03:27:07.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.800+0000 7f7f3a7fc700 1 -- 172.21.15.40:0/782246211 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7f440199a0 con 0x7f7f48107f80 2024-06-06T03:27:07.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.802+0000 7f7f4f5cc700 1 -- 172.21.15.40:0/782246211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f48107f80 msgr2=0x7f7f4819be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:07.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.802+0000 7f7f4f5cc700 1 --2- 172.21.15.40:0/782246211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f48107f80 0x7f7f4819be60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f44011670 tx=0x7f7f44011b60 comp rx=0 tx=0).stop 2024-06-06T03:27:07.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.802+0000 7f7f4f5cc700 1 -- 172.21.15.40:0/782246211 shutdown_connections 2024-06-06T03:27:07.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.802+0000 7f7f4f5cc700 1 --2- 172.21.15.40:0/782246211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f48107f80 0x7f7f4819be60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:07.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.802+0000 7f7f4f5cc700 1 -- 172.21.15.40:0/782246211 >> 172.21.15.40:0/782246211 conn(0x7f7f48103500 msgr2=0x7f7f48104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:07.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.802+0000 7f7f4f5cc700 1 -- 172.21.15.40:0/782246211 shutdown_connections 2024-06-06T03:27:07.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:06.802+0000 7f7f4f5cc700 1 -- 172.21.15.40:0/782246211 wait complete. 2024-06-06T03:27:07.355 INFO:teuthology.orchestra.run.smithi040.stdout:mgr not available, waiting (3/15)... 2024-06-06T03:27:08.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:07 smithi040 ceph-mon[32220]: Activating manager daemon x 2024-06-06T03:27:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:07 smithi040 ceph-mon[32220]: mgrmap e2: x(active, starting, since 0.00301253s) 2024-06-06T03:27:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:07 smithi040 ceph-mon[32220]: from='mgr.14100 172.21.15.40:0/3151852835' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T03:27:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:07 smithi040 ceph-mon[32220]: from='mgr.14100 172.21.15.40:0/3151852835' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T03:27:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:07 smithi040 ceph-mon[32220]: from='mgr.14100 172.21.15.40:0/3151852835' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T03:27:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:07 smithi040 ceph-mon[32220]: from='mgr.14100 172.21.15.40:0/3151852835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:27:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:07 smithi040 ceph-mon[32220]: from='mgr.14100 172.21.15.40:0/3151852835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T03:27:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:07 smithi040 ceph-mon[32220]: Manager daemon x is now available 2024-06-06T03:27:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:07 smithi040 ceph-mon[32220]: from='mgr.14100 172.21.15.40:0/3151852835' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T03:27:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:07 smithi040 ceph-mon[32220]: from='mgr.14100 172.21.15.40:0/3151852835' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T03:27:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:07 smithi040 ceph-mon[32220]: from='mgr.14100 172.21.15.40:0/3151852835' entity='mgr.x' 2024-06-06T03:27:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:07 smithi040 ceph-mon[32220]: from='mgr.14100 172.21.15.40:0/3151852835' entity='mgr.x' 2024-06-06T03:27:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:07 smithi040 ceph-mon[32220]: from='mgr.14100 172.21.15.40:0/3151852835' entity='mgr.x' 2024-06-06T03:27:09.472 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:09 smithi040 ceph-mon[32220]: mgrmap e3: x(active, since 1.00726s) 2024-06-06T03:27:10.449 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:10 smithi040 ceph-mon[32220]: mgrmap e4: x(active, since 2s) 2024-06-06T03:27:10.449 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:10 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2058301210' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T03:27:10.730 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-06-06T03:27:10.730 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-06-06T03:27:10.730 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:27:10.730 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T03:27:10.730 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T03:27:10.730 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T03:27:10.730 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T03:27:10.730 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:10.730 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T03:27:10.730 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T03:27:10.730 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 0 2024-06-06T03:27:10.730 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-06-06T03:27:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T03:27:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T03:27:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-06-06T03:27:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-06-06T03:27:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T03:27:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T03:27:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T03:27:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T03:27:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T03:27:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T03:27:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T03:27:10.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T03:27:10.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T03:27:10.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T03:27:10.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:10.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T03:27:10.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T03:27:10.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T03:27:10.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T03:27:10.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T03:27:10.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T03:27:10.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T03:27:10.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T03:27:10.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T03:27:10.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:10.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T03:27:10.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:10.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T03:27:10.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T03:27:10.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:10.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T03:27:10.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T03:27:10.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T03:27:10.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T03:27:10.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T03:27:10.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T03:27:10.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T03:27:10.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-06-06T03:27:10.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T03:27:10.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:10.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T03:27:10.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T03:27:10.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T03:26:53.165952+0000", 2024-06-06T03:27:10.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T03:27:10.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-06-06T03:27:10.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T03:27:10.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-06-06T03:27:10.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.793+0000 7f9e31c74700 1 Processor -- start 2024-06-06T03:27:10.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.793+0000 7f9e31c74700 1 -- start start 2024-06-06T03:27:10.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.793+0000 7f9e31c74700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e2c108ac0 0x7f9e2c10aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:10.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.793+0000 7f9e31c74700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c07c630 con 0x7f9e2c108ac0 2024-06-06T03:27:10.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.793+0000 7f9e2b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e2c108ac0 0x7f9e2c10aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:10.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.794+0000 7f9e2b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e2c108ac0 0x7f9e2c10aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48842/0 (socket says 172.21.15.40:48842) 2024-06-06T03:27:10.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.794+0000 7f9e2b7fe700 1 -- 172.21.15.40:0/2784361041 learned_addr learned my addr 172.21.15.40:0/2784361041 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:10.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.794+0000 7f9e2b7fe700 1 -- 172.21.15.40:0/2784361041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c10b3f0 con 0x7f9e2c108ac0 2024-06-06T03:27:10.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.794+0000 7f9e2b7fe700 1 --2- 172.21.15.40:0/2784361041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e2c108ac0 0x7f9e2c10aeb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0098a0 tx=0x7f9e1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e16727eaf157cd2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:10.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.794+0000 7f9e2affd700 1 -- 172.21.15.40:0/2784361041 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e1c004160 con 0x7f9e2c108ac0 2024-06-06T03:27:10.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.794+0000 7f9e2affd700 1 -- 172.21.15.40:0/2784361041 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9e1c004300 con 0x7f9e2c108ac0 2024-06-06T03:27:10.736 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.794+0000 7f9e2affd700 1 -- 172.21.15.40:0/2784361041 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e1c011b50 con 0x7f9e2c108ac0 2024-06-06T03:27:10.736 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.795+0000 7f9e31c74700 1 -- 172.21.15.40:0/2784361041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e2c108ac0 msgr2=0x7f9e2c10aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:10.736 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.795+0000 7f9e31c74700 1 --2- 172.21.15.40:0/2784361041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e2c108ac0 0x7f9e2c10aeb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0098a0 tx=0x7f9e1c009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:10.736 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.795+0000 7f9e31c74700 1 -- 172.21.15.40:0/2784361041 shutdown_connections 2024-06-06T03:27:10.736 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.795+0000 7f9e31c74700 1 --2- 172.21.15.40:0/2784361041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e2c108ac0 0x7f9e2c10aeb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:10.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.795+0000 7f9e31c74700 1 -- 172.21.15.40:0/2784361041 >> 172.21.15.40:0/2784361041 conn(0x7f9e2c102470 msgr2=0x7f9e2c104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:10.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.795+0000 7f9e31c74700 1 -- 172.21.15.40:0/2784361041 shutdown_connections 2024-06-06T03:27:10.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.795+0000 7f9e31c74700 1 -- 172.21.15.40:0/2784361041 wait complete. 2024-06-06T03:27:10.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.795+0000 7f9e31c74700 1 Processor -- start 2024-06-06T03:27:10.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.796+0000 7f9e31c74700 1 -- start start 2024-06-06T03:27:10.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.796+0000 7f9e31c74700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e2c108ac0 0x7f9e2c19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:10.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.796+0000 7f9e31c74700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c10c270 con 0x7f9e2c108ac0 2024-06-06T03:27:10.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.796+0000 7f9e2b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e2c108ac0 0x7f9e2c19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:10.739 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.796+0000 7f9e2b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e2c108ac0 0x7f9e2c19be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48858/0 (socket says 172.21.15.40:48858) 2024-06-06T03:27:10.739 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.796+0000 7f9e2b7fe700 1 -- 172.21.15.40:0/2058301210 learned_addr learned my addr 172.21.15.40:0/2058301210 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:10.739 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.796+0000 7f9e2b7fe700 1 -- 172.21.15.40:0/2058301210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c19c3d0 con 0x7f9e2c108ac0 2024-06-06T03:27:10.739 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.797+0000 7f9e2b7fe700 1 --2- 172.21.15.40:0/2058301210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e2c108ac0 0x7f9e2c19be90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c011b60 tx=0x7f9e1c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:10.739 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.797+0000 7f9e297fa700 1 -- 172.21.15.40:0/2058301210 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e1c003970 con 0x7f9e2c108ac0 2024-06-06T03:27:10.739 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.797+0000 7f9e297fa700 1 -- 172.21.15.40:0/2058301210 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9e1c003b10 con 0x7f9e2c108ac0 2024-06-06T03:27:10.739 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.797+0000 7f9e31c74700 1 -- 172.21.15.40:0/2058301210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2c19c660 con 0x7f9e2c108ac0 2024-06-06T03:27:10.739 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.797+0000 7f9e297fa700 1 -- 172.21.15.40:0/2058301210 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e1c024610 con 0x7f9e2c108ac0 2024-06-06T03:27:10.739 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.797+0000 7f9e31c74700 1 -- 172.21.15.40:0/2058301210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e2c19f350 con 0x7f9e2c108ac0 2024-06-06T03:27:10.739 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.798+0000 7f9e297fa700 1 -- 172.21.15.40:0/2058301210 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f9e1c0248c0 con 0x7f9e2c108ac0 2024-06-06T03:27:10.739 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.798+0000 7f9e31c74700 1 -- 172.21.15.40:0/2058301210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e2c063af0 con 0x7f9e2c108ac0 2024-06-06T03:27:10.739 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.798+0000 7f9e297fa700 1 --2- 172.21.15.40:0/2058301210 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f9e18039050 0x7f9e1803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:10.739 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.798+0000 7f9e297fa700 1 -- 172.21.15.40:0/2058301210 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9e1c011e00 con 0x7f9e2c108ac0 2024-06-06T03:27:10.739 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.798+0000 7f9e22ffd700 1 --2- 172.21.15.40:0/2058301210 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f9e18039050 0x7f9e1803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:10.740 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.799+0000 7f9e22ffd700 1 --2- 172.21.15.40:0/2058301210 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f9e18039050 0x7f9e1803b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9e14006fd0 tx=0x7f9e14006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:10.740 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:09.803+0000 7f9e297fa700 1 -- 172.21.15.40:0/2058301210 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9e1c0172b0 con 0x7f9e2c108ac0 2024-06-06T03:27:10.740 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:10.093+0000 7f9e31c74700 1 -- 172.21.15.40:0/2058301210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9e2c106d90 con 0x7f9e2c108ac0 2024-06-06T03:27:10.740 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:10.093+0000 7f9e297fa700 1 -- 172.21.15.40:0/2058301210 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9e1c00ba30 con 0x7f9e2c108ac0 2024-06-06T03:27:10.740 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:10.096+0000 7f9e31c74700 1 -- 172.21.15.40:0/2058301210 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f9e18039050 msgr2=0x7f9e1803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:10.740 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:10.096+0000 7f9e31c74700 1 --2- 172.21.15.40:0/2058301210 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f9e18039050 0x7f9e1803b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9e14006fd0 tx=0x7f9e14006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:10.740 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:10.096+0000 7f9e31c74700 1 -- 172.21.15.40:0/2058301210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e2c108ac0 msgr2=0x7f9e2c19be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:10.740 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:10.096+0000 7f9e31c74700 1 --2- 172.21.15.40:0/2058301210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e2c108ac0 0x7f9e2c19be90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c011b60 tx=0x7f9e1c011c40 comp rx=0 tx=0).stop 2024-06-06T03:27:10.740 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:10.096+0000 7f9e31c74700 1 -- 172.21.15.40:0/2058301210 shutdown_connections 2024-06-06T03:27:10.740 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:10.096+0000 7f9e31c74700 1 --2- 172.21.15.40:0/2058301210 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f9e18039050 0x7f9e1803b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:10.740 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:10.096+0000 7f9e31c74700 1 --2- 172.21.15.40:0/2058301210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e2c108ac0 0x7f9e2c19be90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:10.740 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:10.096+0000 7f9e31c74700 1 -- 172.21.15.40:0/2058301210 >> 172.21.15.40:0/2058301210 conn(0x7f9e2c102470 msgr2=0x7f9e2c103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:10.740 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:10.097+0000 7f9e31c74700 1 -- 172.21.15.40:0/2058301210 shutdown_connections 2024-06-06T03:27:10.740 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:10.097+0000 7f9e31c74700 1 -- 172.21.15.40:0/2058301210 wait complete. 2024-06-06T03:27:10.741 INFO:teuthology.orchestra.run.smithi040.stdout:mgr is available 2024-06-06T03:27:11.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:11 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2284630151' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-06T03:27:11.932 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-06-06T03:27:11.932 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [global] 2024-06-06T03:27:11.932 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout fsid = 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:27:11.932 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-06T03:27:11.932 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-06T03:27:11.932 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-06T03:27:11.932 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-06T03:27:11.932 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [osd] 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.133+0000 7fc85c652700 1 Processor -- start 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.133+0000 7fc85c652700 1 -- start start 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.134+0000 7fc85c652700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc854108aa0 0x7fc85410ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.134+0000 7fc85c652700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc85407bda0 con 0x7fc854108aa0 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.134+0000 7fc85a3ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc854108aa0 0x7fc85410ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:11.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.134+0000 7fc85a3ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc854108aa0 0x7fc85410ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48866/0 (socket says 172.21.15.40:48866) 2024-06-06T03:27:11.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.134+0000 7fc85a3ee700 1 -- 172.21.15.40:0/430915572 learned_addr learned my addr 172.21.15.40:0/430915572 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:11.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.134+0000 7fc85a3ee700 1 -- 172.21.15.40:0/430915572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85410b3d0 con 0x7fc854108aa0 2024-06-06T03:27:11.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.135+0000 7fc85a3ee700 1 --2- 172.21.15.40:0/430915572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc854108aa0 0x7fc85410ae90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc8500098a0 tx=0x7fc850009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f7298561b3b155 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:11.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.135+0000 7fc8593ec700 1 -- 172.21.15.40:0/430915572 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc850004160 con 0x7fc854108aa0 2024-06-06T03:27:11.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.135+0000 7fc8593ec700 1 -- 172.21.15.40:0/430915572 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc850004300 con 0x7fc854108aa0 2024-06-06T03:27:11.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.135+0000 7fc8593ec700 1 -- 172.21.15.40:0/430915572 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc850011b50 con 0x7fc854108aa0 2024-06-06T03:27:11.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.135+0000 7fc85c652700 1 -- 172.21.15.40:0/430915572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc854108aa0 msgr2=0x7fc85410ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:11.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.135+0000 7fc85c652700 1 --2- 172.21.15.40:0/430915572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc854108aa0 0x7fc85410ae90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc8500098a0 tx=0x7fc850009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:11.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.136+0000 7fc85c652700 1 -- 172.21.15.40:0/430915572 shutdown_connections 2024-06-06T03:27:11.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.136+0000 7fc85c652700 1 --2- 172.21.15.40:0/430915572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc854108aa0 0x7fc85410ae90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:11.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.136+0000 7fc85c652700 1 -- 172.21.15.40:0/430915572 >> 172.21.15.40:0/430915572 conn(0x7fc8541023f0 msgr2=0x7fc854104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:11.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.136+0000 7fc85c652700 1 -- 172.21.15.40:0/430915572 shutdown_connections 2024-06-06T03:27:11.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.136+0000 7fc85c652700 1 -- 172.21.15.40:0/430915572 wait complete. 2024-06-06T03:27:11.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.136+0000 7fc85c652700 1 Processor -- start 2024-06-06T03:27:11.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.136+0000 7fc85c652700 1 -- start start 2024-06-06T03:27:11.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.136+0000 7fc85c652700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc854108aa0 0x7fc854108320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:11.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.136+0000 7fc85c652700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc85410c250 con 0x7fc854108aa0 2024-06-06T03:27:11.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.137+0000 7fc85a3ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc854108aa0 0x7fc854108320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:11.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.137+0000 7fc85a3ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc854108aa0 0x7fc854108320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48880/0 (socket says 172.21.15.40:48880) 2024-06-06T03:27:11.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.137+0000 7fc85a3ee700 1 -- 172.21.15.40:0/2284630151 learned_addr learned my addr 172.21.15.40:0/2284630151 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:11.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.137+0000 7fc85a3ee700 1 -- 172.21.15.40:0/2284630151 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc854108860 con 0x7fc854108aa0 2024-06-06T03:27:11.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.137+0000 7fc85a3ee700 1 --2- 172.21.15.40:0/2284630151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc854108aa0 0x7fc854108320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc850009fb0 tx=0x7fc850011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:11.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.137+0000 7fc8477fe700 1 -- 172.21.15.40:0/2284630151 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8500038b0 con 0x7fc854108aa0 2024-06-06T03:27:11.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.137+0000 7fc8477fe700 1 -- 172.21.15.40:0/2284630151 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc850011e50 con 0x7fc854108aa0 2024-06-06T03:27:11.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.137+0000 7fc85c652700 1 -- 172.21.15.40:0/2284630151 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc854106a50 con 0x7fc854108aa0 2024-06-06T03:27:11.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.137+0000 7fc8477fe700 1 -- 172.21.15.40:0/2284630151 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc850019420 con 0x7fc854108aa0 2024-06-06T03:27:11.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.137+0000 7fc85c652700 1 -- 172.21.15.40:0/2284630151 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc854106f90 con 0x7fc854108aa0 2024-06-06T03:27:11.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.138+0000 7fc8477fe700 1 -- 172.21.15.40:0/2284630151 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fc850019720 con 0x7fc854108aa0 2024-06-06T03:27:11.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.138+0000 7fc8477fe700 1 --2- 172.21.15.40:0/2284630151 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7fc840038fb0 0x7fc84003b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:11.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.138+0000 7fc85c652700 1 -- 172.21.15.40:0/2284630151 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc854063af0 con 0x7fc854108aa0 2024-06-06T03:27:11.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.138+0000 7fc8477fe700 1 -- 172.21.15.40:0/2284630151 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc85004d410 con 0x7fc854108aa0 2024-06-06T03:27:11.937 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.138+0000 7fc859bed700 1 --2- 172.21.15.40:0/2284630151 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7fc840038fb0 0x7fc84003b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:11.937 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.139+0000 7fc859bed700 1 --2- 172.21.15.40:0/2284630151 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7fc840038fb0 0x7fc84003b470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc848006fd0 tx=0x7fc848006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:11.937 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.144+0000 7fc8477fe700 1 -- 172.21.15.40:0/2284630151 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc85004ce30 con 0x7fc854108aa0 2024-06-06T03:27:11.937 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.352+0000 7fc85c652700 1 -- 172.21.15.40:0/2284630151 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc854107210 con 0x7fc854108aa0 2024-06-06T03:27:11.937 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.353+0000 7fc8477fe700 1 -- 172.21.15.40:0/2284630151 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fc850003ac0 con 0x7fc854108aa0 2024-06-06T03:27:11.937 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.356+0000 7fc85c652700 1 -- 172.21.15.40:0/2284630151 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7fc840038fb0 msgr2=0x7fc84003b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:11.937 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.356+0000 7fc85c652700 1 --2- 172.21.15.40:0/2284630151 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7fc840038fb0 0x7fc84003b470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc848006fd0 tx=0x7fc848006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:11.937 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.356+0000 7fc85c652700 1 -- 172.21.15.40:0/2284630151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc854108aa0 msgr2=0x7fc854108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:11.937 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.356+0000 7fc85c652700 1 --2- 172.21.15.40:0/2284630151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc854108aa0 0x7fc854108320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc850009fb0 tx=0x7fc850011bf0 comp rx=0 tx=0).stop 2024-06-06T03:27:11.938 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.356+0000 7fc85c652700 1 -- 172.21.15.40:0/2284630151 shutdown_connections 2024-06-06T03:27:11.938 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.356+0000 7fc85c652700 1 --2- 172.21.15.40:0/2284630151 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7fc840038fb0 0x7fc84003b470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:11.938 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.356+0000 7fc85c652700 1 --2- 172.21.15.40:0/2284630151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc854108aa0 0x7fc854108320 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:11.938 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.356+0000 7fc85c652700 1 -- 172.21.15.40:0/2284630151 >> 172.21.15.40:0/2284630151 conn(0x7fc8541023f0 msgr2=0x7fc854102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:11.938 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.356+0000 7fc85c652700 1 -- 172.21.15.40:0/2284630151 shutdown_connections 2024-06-06T03:27:11.938 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:11.356+0000 7fc85c652700 1 -- 172.21.15.40:0/2284630151 wait complete. 2024-06-06T03:27:11.938 INFO:teuthology.orchestra.run.smithi040.stdout:Enabling cephadm module... 2024-06-06T03:27:12.970 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:12 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1531245816' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-06T03:27:12.971 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:12 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: ignoring --setuser ceph since I am not root 2024-06-06T03:27:12.971 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:12 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: ignoring --setgroup ceph since I am not root 2024-06-06T03:27:12.971 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:12 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:12.827+0000 7f5b596f7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T03:27:13.263 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:13 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:13.020+0000 7f5b596f7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:27:13.263 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:13 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:13.127+0000 7f5b596f7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T03:27:13.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.327+0000 7f8fefab8700 1 Processor -- start 2024-06-06T03:27:13.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.327+0000 7f8fefab8700 1 -- start start 2024-06-06T03:27:13.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.328+0000 7f8fefab8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fe807e690 0x7f8fe807cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:13.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.328+0000 7f8fefab8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe807d080 con 0x7f8fe807e690 2024-06-06T03:27:13.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.328+0000 7f8fed854700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fe807e690 0x7f8fe807cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:13.271 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.328+0000 7f8fed854700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fe807e690 0x7f8fe807cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48890/0 (socket says 172.21.15.40:48890) 2024-06-06T03:27:13.271 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.328+0000 7f8fed854700 1 -- 172.21.15.40:0/1957452340 learned_addr learned my addr 172.21.15.40:0/1957452340 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:13.271 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.328+0000 7f8fed854700 1 -- 172.21.15.40:0/1957452340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe807d200 con 0x7f8fe807e690 2024-06-06T03:27:13.271 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.328+0000 7f8fed854700 1 --2- 172.21.15.40:0/1957452340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fe807e690 0x7f8fe807cb40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40098a0 tx=0x7f8fe4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783ce689eb950578 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:13.271 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.328+0000 7f8fec852700 1 -- 172.21.15.40:0/1957452340 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fe4004160 con 0x7f8fe807e690 2024-06-06T03:27:13.271 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.329+0000 7f8fec852700 1 -- 172.21.15.40:0/1957452340 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8fe4004300 con 0x7f8fe807e690 2024-06-06T03:27:13.271 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.329+0000 7f8fec852700 1 -- 172.21.15.40:0/1957452340 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fe4011b50 con 0x7f8fe807e690 2024-06-06T03:27:13.271 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.329+0000 7f8fefab8700 1 -- 172.21.15.40:0/1957452340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fe807e690 msgr2=0x7f8fe807cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:13.271 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.329+0000 7f8fefab8700 1 --2- 172.21.15.40:0/1957452340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fe807e690 0x7f8fe807cb40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40098a0 tx=0x7f8fe4009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:13.271 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.329+0000 7f8fefab8700 1 -- 172.21.15.40:0/1957452340 shutdown_connections 2024-06-06T03:27:13.271 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.329+0000 7f8fefab8700 1 --2- 172.21.15.40:0/1957452340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fe807e690 0x7f8fe807cb40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:13.272 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.329+0000 7f8fefab8700 1 -- 172.21.15.40:0/1957452340 >> 172.21.15.40:0/1957452340 conn(0x7f8fe8103500 msgr2=0x7f8fe8105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:13.272 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.329+0000 7f8fefab8700 1 -- 172.21.15.40:0/1957452340 shutdown_connections 2024-06-06T03:27:13.272 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.329+0000 7f8fefab8700 1 -- 172.21.15.40:0/1957452340 wait complete. 2024-06-06T03:27:13.272 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.330+0000 7f8fefab8700 1 Processor -- start 2024-06-06T03:27:13.272 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.330+0000 7f8fefab8700 1 -- start start 2024-06-06T03:27:13.272 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.330+0000 7f8fefab8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fe807e690 0x7f8fe819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:13.272 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.330+0000 7f8fefab8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe810e420 con 0x7f8fe807e690 2024-06-06T03:27:13.272 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.330+0000 7f8fed854700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fe807e690 0x7f8fe819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:13.272 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.330+0000 7f8fed854700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fe807e690 0x7f8fe819e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48906/0 (socket says 172.21.15.40:48906) 2024-06-06T03:27:13.272 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.330+0000 7f8fed854700 1 -- 172.21.15.40:0/1531245816 learned_addr learned my addr 172.21.15.40:0/1531245816 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:13.272 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.330+0000 7f8fed854700 1 -- 172.21.15.40:0/1531245816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe819e590 con 0x7f8fe807e690 2024-06-06T03:27:13.272 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.331+0000 7f8fed854700 1 --2- 172.21.15.40:0/1531245816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fe807e690 0x7f8fe819e050 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4011670 tx=0x7f8fe4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:13.272 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.331+0000 7f8fdeffd700 1 -- 172.21.15.40:0/1531245816 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fe40039c0 con 0x7f8fe807e690 2024-06-06T03:27:13.273 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.331+0000 7f8fdeffd700 1 -- 172.21.15.40:0/1531245816 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8fe4011e50 con 0x7f8fe807e690 2024-06-06T03:27:13.273 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.331+0000 7f8fefab8700 1 -- 172.21.15.40:0/1531245816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe819e820 con 0x7f8fe807e690 2024-06-06T03:27:13.273 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.331+0000 7f8fdeffd700 1 -- 172.21.15.40:0/1531245816 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fe401a5c0 con 0x7f8fe807e690 2024-06-06T03:27:13.273 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.331+0000 7f8fefab8700 1 -- 172.21.15.40:0/1531245816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe81a1510 con 0x7f8fe807e690 2024-06-06T03:27:13.273 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.332+0000 7f8fdeffd700 1 -- 172.21.15.40:0/1531245816 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f8fe4024430 con 0x7f8fe807e690 2024-06-06T03:27:13.273 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.332+0000 7f8fefab8700 1 -- 172.21.15.40:0/1531245816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe8063af0 con 0x7f8fe807e690 2024-06-06T03:27:13.273 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.332+0000 7f8fdeffd700 1 --2- 172.21.15.40:0/1531245816 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f8fd4038fb0 0x7f8fd403b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:13.273 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.332+0000 7f8fdeffd700 1 -- 172.21.15.40:0/1531245816 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8fe404d3e0 con 0x7f8fe807e690 2024-06-06T03:27:13.273 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.332+0000 7f8fed053700 1 --2- 172.21.15.40:0/1531245816 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f8fd4038fb0 0x7f8fd403b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:13.273 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.333+0000 7f8fed053700 1 --2- 172.21.15.40:0/1531245816 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f8fd4038fb0 0x7f8fd403b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8006fd0 tx=0x7f8fd8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:13.273 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.337+0000 7f8fdeffd700 1 -- 172.21.15.40:0/1531245816 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8fe4003b60 con 0x7f8fe807e690 2024-06-06T03:27:13.273 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.602+0000 7f8fefab8700 1 -- 172.21.15.40:0/1531245816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f8fe81a16e0 con 0x7f8fe807e690 2024-06-06T03:27:13.273 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.653+0000 7f8fdeffd700 1 -- 172.21.15.40:0/1531245816 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f8fe81a16e0 con 0x7f8fe807e690 2024-06-06T03:27:13.274 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.655+0000 7f8fdeffd700 1 -- 172.21.15.40:0/1531245816 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f8fe402d3f0 con 0x7f8fe807e690 2024-06-06T03:27:13.274 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.656+0000 7f8fefab8700 1 -- 172.21.15.40:0/1531245816 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f8fd4038fb0 msgr2=0x7f8fd403b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:13.274 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.656+0000 7f8fefab8700 1 --2- 172.21.15.40:0/1531245816 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f8fd4038fb0 0x7f8fd403b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8006fd0 tx=0x7f8fd8006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:13.274 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.656+0000 7f8fefab8700 1 -- 172.21.15.40:0/1531245816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fe807e690 msgr2=0x7f8fe819e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:13.274 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.656+0000 7f8fefab8700 1 --2- 172.21.15.40:0/1531245816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fe807e690 0x7f8fe819e050 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4011670 tx=0x7f8fe4011b60 comp rx=0 tx=0).stop 2024-06-06T03:27:13.274 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.656+0000 7f8fefab8700 1 -- 172.21.15.40:0/1531245816 shutdown_connections 2024-06-06T03:27:13.274 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.656+0000 7f8fefab8700 1 --2- 172.21.15.40:0/1531245816 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f8fd4038fb0 0x7f8fd403b470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:13.274 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.656+0000 7f8fefab8700 1 --2- 172.21.15.40:0/1531245816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fe807e690 0x7f8fe819e050 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:13.274 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.656+0000 7f8fefab8700 1 -- 172.21.15.40:0/1531245816 >> 172.21.15.40:0/1531245816 conn(0x7f8fe8103500 msgr2=0x7f8fe81040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:13.274 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.656+0000 7f8fefab8700 1 -- 172.21.15.40:0/1531245816 shutdown_connections 2024-06-06T03:27:13.274 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:12.656+0000 7f8fefab8700 1 -- 172.21.15.40:0/1531245816 wait complete. 2024-06-06T03:27:13.546 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:13 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:13.454+0000 7f5b596f7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T03:27:13.546 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:13 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:13.546+0000 7f5b596f7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T03:27:13.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:13 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1531245816' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-06T03:27:13.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:13 smithi040 ceph-mon[32220]: mgrmap e5: x(active, since 5s) 2024-06-06T03:27:14.530 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-06-06T03:27:14.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-06T03:27:14.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T03:27:14.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-06T03:27:14.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-06T03:27:14.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-06-06T03:27:14.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.654+0000 7f80a0c9f700 1 Processor -- start 2024-06-06T03:27:14.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.654+0000 7f80a0c9f700 1 -- start start 2024-06-06T03:27:14.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.654+0000 7f80a0c9f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c108a10 0x7f809c10ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:14.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.654+0000 7f80a0c9f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f809c07be80 con 0x7f809c108a10 2024-06-06T03:27:14.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.654+0000 7f809a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c108a10 0x7f809c10ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:14.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.654+0000 7f809a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c108a10 0x7f809c10ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48934/0 (socket says 172.21.15.40:48934) 2024-06-06T03:27:14.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.654+0000 7f809a59c700 1 -- 172.21.15.40:0/120948984 learned_addr learned my addr 172.21.15.40:0/120948984 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:14.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.654+0000 7f809a59c700 1 -- 172.21.15.40:0/120948984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809c10b340 con 0x7f809c108a10 2024-06-06T03:27:14.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.655+0000 7f809a59c700 1 --2- 172.21.15.40:0/120948984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c108a10 0x7f809c10ae00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f808c0098a0 tx=0x7f808c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea60651f13a7cda server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:14.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.655+0000 7f809959a700 1 -- 172.21.15.40:0/120948984 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f808c004160 con 0x7f809c108a10 2024-06-06T03:27:14.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.655+0000 7f809959a700 1 -- 172.21.15.40:0/120948984 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f808c004300 con 0x7f809c108a10 2024-06-06T03:27:14.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.655+0000 7f809959a700 1 -- 172.21.15.40:0/120948984 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f808c011b50 con 0x7f809c108a10 2024-06-06T03:27:14.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.656+0000 7f80a0c9f700 1 -- 172.21.15.40:0/120948984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c108a10 msgr2=0x7f809c10ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:14.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.656+0000 7f80a0c9f700 1 --2- 172.21.15.40:0/120948984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c108a10 0x7f809c10ae00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f808c0098a0 tx=0x7f808c009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:14.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.656+0000 7f80a0c9f700 1 -- 172.21.15.40:0/120948984 shutdown_connections 2024-06-06T03:27:14.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.656+0000 7f80a0c9f700 1 --2- 172.21.15.40:0/120948984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c108a10 0x7f809c10ae00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:14.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.656+0000 7f80a0c9f700 1 -- 172.21.15.40:0/120948984 >> 172.21.15.40:0/120948984 conn(0x7f809c102440 msgr2=0x7f809c104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:14.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.656+0000 7f80a0c9f700 1 -- 172.21.15.40:0/120948984 shutdown_connections 2024-06-06T03:27:14.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.656+0000 7f80a0c9f700 1 -- 172.21.15.40:0/120948984 wait complete. 2024-06-06T03:27:14.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.657+0000 7f80a0c9f700 1 Processor -- start 2024-06-06T03:27:14.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.657+0000 7f80a0c9f700 1 -- start start 2024-06-06T03:27:14.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.657+0000 7f80a0c9f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c108a10 0x7f809c19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:14.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.657+0000 7f80a0c9f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f809c10c290 con 0x7f809c108a10 2024-06-06T03:27:14.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.657+0000 7f809a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c108a10 0x7f809c19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:14.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.657+0000 7f809a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c108a10 0x7f809c19bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48944/0 (socket says 172.21.15.40:48944) 2024-06-06T03:27:14.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.657+0000 7f809a59c700 1 -- 172.21.15.40:0/577408099 learned_addr learned my addr 172.21.15.40:0/577408099 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:14.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.657+0000 7f809a59c700 1 -- 172.21.15.40:0/577408099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809c19c450 con 0x7f809c108a10 2024-06-06T03:27:14.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.658+0000 7f809a59c700 1 --2- 172.21.15.40:0/577408099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c108a10 0x7f809c19bf10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f808c009fb0 tx=0x7f808c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:14.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.660+0000 7f808b7fe700 1 -- 172.21.15.40:0/577408099 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f808c0038b0 con 0x7f809c108a10 2024-06-06T03:27:14.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.660+0000 7f808b7fe700 1 -- 172.21.15.40:0/577408099 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f808c011e50 con 0x7f809c108a10 2024-06-06T03:27:14.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.660+0000 7f80a0c9f700 1 -- 172.21.15.40:0/577408099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809c19c6e0 con 0x7f809c108a10 2024-06-06T03:27:14.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.660+0000 7f808b7fe700 1 -- 172.21.15.40:0/577408099 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f808c019420 con 0x7f809c108a10 2024-06-06T03:27:14.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.660+0000 7f80a0c9f700 1 -- 172.21.15.40:0/577408099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809c19f3d0 con 0x7f809c108a10 2024-06-06T03:27:14.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.661+0000 7f808b7fe700 1 -- 172.21.15.40:0/577408099 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f808c0196d0 con 0x7f809c108a10 2024-06-06T03:27:14.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.661+0000 7f80a0c9f700 1 -- 172.21.15.40:0/577408099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f809c063af0 con 0x7f809c108a10 2024-06-06T03:27:14.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.661+0000 7f808b7fe700 1 --2- 172.21.15.40:0/577408099 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f8084039050 0x7f808403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:14.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.661+0000 7f808b7fe700 1 -- 172.21.15.40:0/577408099 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f808c04d200 con 0x7f809c108a10 2024-06-06T03:27:14.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.661+0000 7f8099d9b700 1 -- 172.21.15.40:0/577408099 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f8084039050 msgr2=0x7f808403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1920198673 2024-06-06T03:27:14.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.661+0000 7f8099d9b700 1 --2- 172.21.15.40:0/577408099 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f8084039050 0x7f808403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T03:27:14.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.666+0000 7f808b7fe700 1 -- 172.21.15.40:0/577408099 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f808c00bc70 con 0x7f809c108a10 2024-06-06T03:27:14.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.862+0000 7f8099d9b700 1 -- 172.21.15.40:0/577408099 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f8084039050 msgr2=0x7f808403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1920198673 2024-06-06T03:27:14.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.862+0000 7f8099d9b700 1 --2- 172.21.15.40:0/577408099 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f8084039050 0x7f808403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T03:27:14.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.934+0000 7f80a0c9f700 1 -- 172.21.15.40:0/577408099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f809c19f5a0 con 0x7f809c108a10 2024-06-06T03:27:14.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.934+0000 7f808b7fe700 1 -- 172.21.15.40:0/577408099 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f809c19f5a0 con 0x7f809c108a10 2024-06-06T03:27:14.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.938+0000 7f80a0c9f700 1 -- 172.21.15.40:0/577408099 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f8084039050 msgr2=0x7f808403b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:27:14.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.938+0000 7f80a0c9f700 1 --2- 172.21.15.40:0/577408099 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f8084039050 0x7f808403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:14.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.938+0000 7f80a0c9f700 1 -- 172.21.15.40:0/577408099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c108a10 msgr2=0x7f809c19bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:14.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.938+0000 7f80a0c9f700 1 --2- 172.21.15.40:0/577408099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c108a10 0x7f809c19bf10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f808c009fb0 tx=0x7f808c011bf0 comp rx=0 tx=0).stop 2024-06-06T03:27:14.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.938+0000 7f80a0c9f700 1 -- 172.21.15.40:0/577408099 shutdown_connections 2024-06-06T03:27:14.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.938+0000 7f80a0c9f700 1 --2- 172.21.15.40:0/577408099 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f8084039050 0x7f808403b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:14.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.938+0000 7f80a0c9f700 1 --2- 172.21.15.40:0/577408099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c108a10 0x7f809c19bf10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:14.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.938+0000 7f80a0c9f700 1 -- 172.21.15.40:0/577408099 >> 172.21.15.40:0/577408099 conn(0x7f809c102440 msgr2=0x7f809c102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:14.536 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.940+0000 7f80a0c9f700 1 -- 172.21.15.40:0/577408099 shutdown_connections 2024-06-06T03:27:14.536 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:13.941+0000 7f80a0c9f700 1 -- 172.21.15.40:0/577408099 wait complete. 2024-06-06T03:27:14.536 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for the mgr to restart... 2024-06-06T03:27:14.536 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mgr epoch 5... 2024-06-06T03:27:14.774 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:14 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/577408099' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-06T03:27:15.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:14 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:14.831+0000 7f5b596f7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T03:27:15.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:15 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:15.012+0000 7f5b596f7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T03:27:15.490 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:15 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:15.189+0000 7f5b596f7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T03:27:15.490 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:15 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:15.490+0000 7f5b596f7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T03:27:15.796 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:15 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:15.586+0000 7f5b596f7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T03:27:15.796 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:15 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:15.796+0000 7f5b596f7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T03:27:16.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:15 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:15.996+0000 7f5b596f7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:27:16.830 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:16 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:16.625+0000 7f5b596f7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T03:27:17.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:17 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:17.066+0000 7f5b596f7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T03:27:17.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:17 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:17.167+0000 7f5b596f7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T03:27:17.830 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:17 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:17.377+0000 7f5b596f7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T03:27:18.272 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:17 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:17.923+0000 7f5b596f7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T03:27:18.273 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:18.018+0000 7f5b596f7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T03:27:18.580 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:18.272+0000 7f5b596f7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T03:27:18.580 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:18.412+0000 7f5b596f7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T03:27:18.580 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:18.511+0000 7f5b596f7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T03:27:18.940 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:18.630+0000 7f5b596f7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T03:27:18.941 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:18.940+0000 7f5b596f7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T03:27:19.830 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:19.570+0000 7f5b596f7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T03:27:19.830 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:19.797+0000 7f5b596f7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T03:27:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: Active manager daemon x restarted 2024-06-06T03:27:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: Activating manager daemon x 2024-06-06T03:27:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: osdmap e2: 0 total, 0 up, 0 in 2024-06-06T03:27:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: mgrmap e6: x(active, starting, since 0.00367477s) 2024-06-06T03:27:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:27:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T03:27:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T03:27:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T03:27:20.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T03:27:20.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: Manager daemon x is now available 2024-06-06T03:27:20.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' 2024-06-06T03:27:20.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' 2024-06-06T03:27:20.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:27:20.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:27:20.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:19 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T03:27:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:20 smithi040 ceph-mon[32220]: Found migration_current of "None". Setting to last migration. 2024-06-06T03:27:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:20 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T03:27:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:20 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' 2024-06-06T03:27:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:20 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' 2024-06-06T03:27:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:20 smithi040 ceph-mon[32220]: mgrmap e7: x(active, since 1.0055s) 2024-06-06T03:27:21.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-06-06T03:27:21.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-06T03:27:21.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-06T03:27:21.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-06-06T03:27:21.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.979+0000 7f88fe0dd700 1 Processor -- start 2024-06-06T03:27:21.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.979+0000 7f88fe0dd700 1 -- start start 2024-06-06T03:27:21.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.979+0000 7f88fe0dd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88f8106870 0x7f88f8106c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:21.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.979+0000 7f88fe0dd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f88f807c640 con 0x7f88f8106870 2024-06-06T03:27:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.979+0000 7f88f77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88f8106870 0x7f88f8106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.979+0000 7f88f77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88f8106870 0x7f88f8106c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48952/0 (socket says 172.21.15.40:48952) 2024-06-06T03:27:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.979+0000 7f88f77fe700 1 -- 172.21.15.40:0/3275723860 learned_addr learned my addr 172.21.15.40:0/3275723860 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.979+0000 7f88f77fe700 1 -- 172.21.15.40:0/3275723860 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f81071d0 con 0x7f88f8106870 2024-06-06T03:27:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.979+0000 7f88f77fe700 1 --2- 172.21.15.40:0/3275723860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88f8106870 0x7f88f8106c90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f88e80098a0 tx=0x7f88e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f89bb4258dcdb3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.980+0000 7f88f6ffd700 1 -- 172.21.15.40:0/3275723860 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e8004160 con 0x7f88f8106870 2024-06-06T03:27:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.980+0000 7f88f6ffd700 1 -- 172.21.15.40:0/3275723860 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f88e8004300 con 0x7f88f8106870 2024-06-06T03:27:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.980+0000 7f88f6ffd700 1 -- 172.21.15.40:0/3275723860 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e8011b50 con 0x7f88f8106870 2024-06-06T03:27:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.980+0000 7f88fe0dd700 1 -- 172.21.15.40:0/3275723860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88f8106870 msgr2=0x7f88f8106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.980+0000 7f88fe0dd700 1 --2- 172.21.15.40:0/3275723860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88f8106870 0x7f88f8106c90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f88e80098a0 tx=0x7f88e8009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.980+0000 7f88fe0dd700 1 -- 172.21.15.40:0/3275723860 shutdown_connections 2024-06-06T03:27:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.980+0000 7f88fe0dd700 1 --2- 172.21.15.40:0/3275723860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88f8106870 0x7f88f8106c90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.980+0000 7f88fe0dd700 1 -- 172.21.15.40:0/3275723860 >> 172.21.15.40:0/3275723860 conn(0x7f88f8102430 msgr2=0x7f88f8104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:21.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.981+0000 7f88fe0dd700 1 -- 172.21.15.40:0/3275723860 shutdown_connections 2024-06-06T03:27:21.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.981+0000 7f88fe0dd700 1 -- 172.21.15.40:0/3275723860 wait complete. 2024-06-06T03:27:21.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.981+0000 7f88fe0dd700 1 Processor -- start 2024-06-06T03:27:21.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.981+0000 7f88fe0dd700 1 -- start start 2024-06-06T03:27:21.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.981+0000 7f88fe0dd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88f8106870 0x7f88f819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:21.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.981+0000 7f88fe0dd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f88f8108000 con 0x7f88f8106870 2024-06-06T03:27:21.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.981+0000 7f88f77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88f8106870 0x7f88f819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:21.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.982+0000 7f88f77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88f8106870 0x7f88f819e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48962/0 (socket says 172.21.15.40:48962) 2024-06-06T03:27:21.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.982+0000 7f88f77fe700 1 -- 172.21.15.40:0/790318357 learned_addr learned my addr 172.21.15.40:0/790318357 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:21.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.982+0000 7f88f77fe700 1 -- 172.21.15.40:0/790318357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f819e5b0 con 0x7f88f8106870 2024-06-06T03:27:21.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.982+0000 7f88f77fe700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88f8106870 0x7f88f819e070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009fb0 tx=0x7f88e8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:21.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.982+0000 7f88f57fa700 1 -- 172.21.15.40:0/790318357 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e80038b0 con 0x7f88f8106870 2024-06-06T03:27:21.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.982+0000 7f88f57fa700 1 -- 172.21.15.40:0/790318357 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f88e8011e50 con 0x7f88f8106870 2024-06-06T03:27:21.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.982+0000 7f88fe0dd700 1 -- 172.21.15.40:0/790318357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f819e840 con 0x7f88f8106870 2024-06-06T03:27:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.982+0000 7f88f57fa700 1 -- 172.21.15.40:0/790318357 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e8019420 con 0x7f88f8106870 2024-06-06T03:27:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.982+0000 7f88fe0dd700 1 -- 172.21.15.40:0/790318357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f81a1530 con 0x7f88f8106870 2024-06-06T03:27:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.983+0000 7f88f57fa700 1 -- 172.21.15.40:0/790318357 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f88e80196f0 con 0x7f88f8106870 2024-06-06T03:27:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.983+0000 7f88f57fa700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f88e4039050 0x7f88e403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.983+0000 7f88f57fa700 1 -- 172.21.15.40:0/790318357 --> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f88e403bc20 con 0x7f88e4039050 2024-06-06T03:27:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.983+0000 7f88f57fa700 1 -- 172.21.15.40:0/790318357 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f88e804d200 con 0x7f88f8106870 2024-06-06T03:27:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.983+0000 7f88effff700 1 -- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f88e4039050 msgr2=0x7f88e403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1920198673 2024-06-06T03:27:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:14.984+0000 7f88effff700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f88e4039050 0x7f88e403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T03:27:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:15.185+0000 7f88effff700 1 -- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f88e4039050 msgr2=0x7f88e403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1920198673 2024-06-06T03:27:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:15.185+0000 7f88effff700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f88e4039050 0x7f88e403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T03:27:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:15.586+0000 7f88effff700 1 -- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f88e4039050 msgr2=0x7f88e403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1920198673 2024-06-06T03:27:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:15.586+0000 7f88effff700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f88e4039050 0x7f88e403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-06T03:27:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:16.387+0000 7f88effff700 1 -- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f88e4039050 msgr2=0x7f88e403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1920198673 2024-06-06T03:27:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:16.387+0000 7f88effff700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f88e4039050 0x7f88e403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-06T03:27:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:17.988+0000 7f88effff700 1 -- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f88e4039050 msgr2=0x7f88e403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1920198673 2024-06-06T03:27:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:17.988+0000 7f88effff700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f88e4039050 0x7f88e403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-06T03:27:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:19.800+0000 7f88f57fa700 1 -- 172.21.15.40:0/790318357 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f88e8003a50 con 0x7f88f8106870 2024-06-06T03:27:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:19.800+0000 7f88f57fa700 1 -- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f88e4039050 msgr2=0x7f88e403b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:27:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:19.800+0000 7f88f57fa700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f88e4039050 0x7f88e403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.801+0000 7f88f57fa700 1 -- 172.21.15.40:0/790318357 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f88e8018350 con 0x7f88f8106870 2024-06-06T03:27:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.801+0000 7f88f57fa700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f88e403c980 0x7f88e403ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.801+0000 7f88f57fa700 1 -- 172.21.15.40:0/790318357 --> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f88e8001d30 con 0x7f88e403c980 2024-06-06T03:27:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.802+0000 7f88effff700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f88e403c980 0x7f88e403ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.802+0000 7f88effff700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f88e403c980 0x7f88e403ed70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f88e0003a10 tx=0x7f88e00092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.803+0000 7f88f57fa700 1 -- 172.21.15.40:0/790318357 <== mgr.14120 v2:172.21.15.40:6800/3930494734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f88e8001d30 con 0x7f88e403c980 2024-06-06T03:27:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.815+0000 7f88fe0dd700 1 -- 172.21.15.40:0/790318357 --> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f88f8063af0 con 0x7f88e403c980 2024-06-06T03:27:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.815+0000 7f88f57fa700 1 -- 172.21.15.40:0/790318357 <== mgr.14120 v2:172.21.15.40:6800/3930494734 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f88f8063af0 con 0x7f88e403c980 2024-06-06T03:27:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.815+0000 7f88fe0dd700 1 -- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f88e403c980 msgr2=0x7f88e403ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:21.409 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.815+0000 7f88fe0dd700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f88e403c980 0x7f88e403ed70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f88e0003a10 tx=0x7f88e00092b0 comp rx=0 tx=0).stop 2024-06-06T03:27:21.409 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.815+0000 7f88fe0dd700 1 -- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88f8106870 msgr2=0x7f88f819e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:21.409 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.815+0000 7f88fe0dd700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88f8106870 0x7f88f819e070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009fb0 tx=0x7f88e8011bf0 comp rx=0 tx=0).stop 2024-06-06T03:27:21.409 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.815+0000 7f88fe0dd700 1 -- 172.21.15.40:0/790318357 shutdown_connections 2024-06-06T03:27:21.409 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.815+0000 7f88fe0dd700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f88e403c980 0x7f88e403ed70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:21.409 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.815+0000 7f88fe0dd700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88f8106870 0x7f88f819e070 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:21.409 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.815+0000 7f88fe0dd700 1 --2- 172.21.15.40:0/790318357 >> [v2:172.21.15.40:6800/1920198673,v1:172.21.15.40:6801/1920198673] conn(0x7f88e4039050 0x7f88e403b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:21.409 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.815+0000 7f88fe0dd700 1 -- 172.21.15.40:0/790318357 >> 172.21.15.40:0/790318357 conn(0x7f88f8102430 msgr2=0x7f88f8194ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:21.409 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.815+0000 7f88fe0dd700 1 -- 172.21.15.40:0/790318357 shutdown_connections 2024-06-06T03:27:21.409 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:20.815+0000 7f88fe0dd700 1 -- 172.21.15.40:0/790318357 wait complete. 2024-06-06T03:27:21.409 INFO:teuthology.orchestra.run.smithi040.stdout:mgr epoch 5 is available 2024-06-06T03:27:21.409 INFO:teuthology.orchestra.run.smithi040.stdout:Setting orchestrator backend to cephadm... 2024-06-06T03:27:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:21 smithi040 ceph-mon[32220]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-06T03:27:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:21 smithi040 ceph-mon[32220]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-06T03:27:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:21 smithi040 ceph-mon[32220]: [06/Jun/2024:03:27:21] ENGINE Bus STARTING 2024-06-06T03:27:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:21 smithi040 ceph-mon[32220]: [06/Jun/2024:03:27:21] ENGINE Serving on https://172.21.15.40:7150 2024-06-06T03:27:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:21 smithi040 ceph-mon[32220]: [06/Jun/2024:03:27:21] ENGINE Serving on http://172.21.15.40:8765 2024-06-06T03:27:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:21 smithi040 ceph-mon[32220]: [06/Jun/2024:03:27:21] ENGINE Bus STARTED 2024-06-06T03:27:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:21 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:27:22.660 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.846+0000 7f52cabe4700 1 Processor -- start 2024-06-06T03:27:22.660 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.846+0000 7f52cabe4700 1 -- start start 2024-06-06T03:27:22.660 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.846+0000 7f52cabe4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c4108a70 0x7f52c410ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:22.660 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.846+0000 7f52cabe4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f52c407be20 con 0x7f52c4108a70 2024-06-06T03:27:22.660 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.847+0000 7f52c8980700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c4108a70 0x7f52c410ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.847+0000 7f52c8980700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c4108a70 0x7f52c410ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39924/0 (socket says 172.21.15.40:39924) 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.847+0000 7f52c8980700 1 -- 172.21.15.40:0/2720009536 learned_addr learned my addr 172.21.15.40:0/2720009536 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.847+0000 7f52c8980700 1 -- 172.21.15.40:0/2720009536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c410b3a0 con 0x7f52c4108a70 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.847+0000 7f52c8980700 1 --2- 172.21.15.40:0/2720009536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c4108a70 0x7f52c410ae60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f52b80098a0 tx=0x7f52b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16644f13cda9788 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.847+0000 7f52c37fe700 1 -- 172.21.15.40:0/2720009536 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b8004160 con 0x7f52c4108a70 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.847+0000 7f52c37fe700 1 -- 172.21.15.40:0/2720009536 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52b8004300 con 0x7f52c4108a70 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.847+0000 7f52c37fe700 1 -- 172.21.15.40:0/2720009536 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b8011b50 con 0x7f52c4108a70 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.848+0000 7f52cabe4700 1 -- 172.21.15.40:0/2720009536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c4108a70 msgr2=0x7f52c410ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.848+0000 7f52cabe4700 1 --2- 172.21.15.40:0/2720009536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c4108a70 0x7f52c410ae60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f52b80098a0 tx=0x7f52b8009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.848+0000 7f52cabe4700 1 -- 172.21.15.40:0/2720009536 shutdown_connections 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.848+0000 7f52cabe4700 1 --2- 172.21.15.40:0/2720009536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c4108a70 0x7f52c410ae60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.848+0000 7f52cabe4700 1 -- 172.21.15.40:0/2720009536 >> 172.21.15.40:0/2720009536 conn(0x7f52c4102440 msgr2=0x7f52c4104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.848+0000 7f52cabe4700 1 -- 172.21.15.40:0/2720009536 shutdown_connections 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.848+0000 7f52cabe4700 1 -- 172.21.15.40:0/2720009536 wait complete. 2024-06-06T03:27:22.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.849+0000 7f52cabe4700 1 Processor -- start 2024-06-06T03:27:22.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.849+0000 7f52cabe4700 1 -- start start 2024-06-06T03:27:22.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.849+0000 7f52cabe4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c4108a70 0x7f52c41082f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:22.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.849+0000 7f52cabe4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f52c410c290 con 0x7f52c4108a70 2024-06-06T03:27:22.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.849+0000 7f52c8980700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c4108a70 0x7f52c41082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:22.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.849+0000 7f52c8980700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c4108a70 0x7f52c41082f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39928/0 (socket says 172.21.15.40:39928) 2024-06-06T03:27:22.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.849+0000 7f52c8980700 1 -- 172.21.15.40:0/524776974 learned_addr learned my addr 172.21.15.40:0/524776974 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:22.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.849+0000 7f52c8980700 1 -- 172.21.15.40:0/524776974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c4108830 con 0x7f52c4108a70 2024-06-06T03:27:22.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.849+0000 7f52c8980700 1 --2- 172.21.15.40:0/524776974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c4108a70 0x7f52c41082f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009fb0 tx=0x7f52b8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:22.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.849+0000 7f52c1ffb700 1 -- 172.21.15.40:0/524776974 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b80038b0 con 0x7f52c4108a70 2024-06-06T03:27:22.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.849+0000 7f52c1ffb700 1 -- 172.21.15.40:0/524776974 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52b8011e50 con 0x7f52c4108a70 2024-06-06T03:27:22.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.849+0000 7f52cabe4700 1 -- 172.21.15.40:0/524776974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c4106a20 con 0x7f52c4108a70 2024-06-06T03:27:22.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.849+0000 7f52c1ffb700 1 -- 172.21.15.40:0/524776974 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b8019420 con 0x7f52c4108a70 2024-06-06T03:27:22.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.850+0000 7f52cabe4700 1 -- 172.21.15.40:0/524776974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c4106f60 con 0x7f52c4108a70 2024-06-06T03:27:22.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.850+0000 7f52c1ffb700 1 -- 172.21.15.40:0/524776974 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f52b8019720 con 0x7f52c4108a70 2024-06-06T03:27:22.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.851+0000 7f52cabe4700 1 -- 172.21.15.40:0/524776974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c4063af0 con 0x7f52c4108a70 2024-06-06T03:27:22.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.851+0000 7f52c1ffb700 1 --2- 172.21.15.40:0/524776974 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f52b4038ef0 0x7f52b403b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:22.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.851+0000 7f52c1ffb700 1 -- 172.21.15.40:0/524776974 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f52b804d320 con 0x7f52c4108a70 2024-06-06T03:27:22.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.851+0000 7f52c3fff700 1 --2- 172.21.15.40:0/524776974 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f52b4038ef0 0x7f52b403b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:22.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.851+0000 7f52c3fff700 1 --2- 172.21.15.40:0/524776974 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f52b4038ef0 0x7f52b403b3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52b0006fd0 tx=0x7f52b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:22.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.857+0000 7f52c1ffb700 1 -- 172.21.15.40:0/524776974 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f52b800bc70 con 0x7f52c4108a70 2024-06-06T03:27:22.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:21.861+0000 7f52c1ffb700 1 -- 172.21.15.40:0/524776974 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f52b802d7c0 con 0x7f52c4108a70 2024-06-06T03:27:22.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:22.081+0000 7f52cabe4700 1 -- 172.21.15.40:0/524776974 --> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f52c41036b0 con 0x7f52b4038ef0 2024-06-06T03:27:22.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:22.089+0000 7f52c1ffb700 1 -- 172.21.15.40:0/524776974 <== mgr.14120 v2:172.21.15.40:6800/3930494734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f52c41036b0 con 0x7f52b4038ef0 2024-06-06T03:27:22.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:22.091+0000 7f52cabe4700 1 -- 172.21.15.40:0/524776974 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f52b4038ef0 msgr2=0x7f52b403b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:22.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:22.091+0000 7f52cabe4700 1 --2- 172.21.15.40:0/524776974 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f52b4038ef0 0x7f52b403b3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52b0006fd0 tx=0x7f52b0006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:22.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:22.091+0000 7f52cabe4700 1 -- 172.21.15.40:0/524776974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c4108a70 msgr2=0x7f52c41082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:22.664 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:22.091+0000 7f52cabe4700 1 --2- 172.21.15.40:0/524776974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c4108a70 0x7f52c41082f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009fb0 tx=0x7f52b8011bf0 comp rx=0 tx=0).stop 2024-06-06T03:27:22.665 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:22.092+0000 7f52cabe4700 1 -- 172.21.15.40:0/524776974 shutdown_connections 2024-06-06T03:27:22.665 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:22.092+0000 7f52cabe4700 1 --2- 172.21.15.40:0/524776974 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f52b4038ef0 0x7f52b403b3b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:22.665 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:22.092+0000 7f52cabe4700 1 --2- 172.21.15.40:0/524776974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52c4108a70 0x7f52c41082f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:22.665 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:22.092+0000 7f52cabe4700 1 -- 172.21.15.40:0/524776974 >> 172.21.15.40:0/524776974 conn(0x7f52c4102440 msgr2=0x7f52c4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:22.665 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:22.092+0000 7f52cabe4700 1 -- 172.21.15.40:0/524776974 shutdown_connections 2024-06-06T03:27:22.665 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:22.092+0000 7f52cabe4700 1 -- 172.21.15.40:0/524776974 wait complete. 2024-06-06T03:27:22.904 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:22 smithi040 ceph-mon[32220]: mgrmap e8: x(active, since 2s) 2024-06-06T03:27:22.904 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:22 smithi040 ceph-mon[32220]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:27:22.904 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:22 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' 2024-06-06T03:27:22.904 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:22 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:27:23.908 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:23 smithi040 ceph-mon[32220]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:27:23.916 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-06T03:27:23.916 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.117+0000 7fa98017d700 1 Processor -- start 2024-06-06T03:27:23.916 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.117+0000 7fa98017d700 1 -- start start 2024-06-06T03:27:23.916 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.117+0000 7fa98017d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa978107f10 0x7fa978108350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:23.916 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.117+0000 7fa98017d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa978108890 con 0x7fa978107f10 2024-06-06T03:27:23.916 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.118+0000 7fa97df19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa978107f10 0x7fa978108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:23.916 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.118+0000 7fa97df19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa978107f10 0x7fa978108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39936/0 (socket says 172.21.15.40:39936) 2024-06-06T03:27:23.916 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.118+0000 7fa97df19700 1 -- 172.21.15.40:0/3617831286 learned_addr learned my addr 172.21.15.40:0/3617831286 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:23.917 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.118+0000 7fa97df19700 1 -- 172.21.15.40:0/3617831286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9781090f0 con 0x7fa978107f10 2024-06-06T03:27:23.917 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.118+0000 7fa97df19700 1 --2- 172.21.15.40:0/3617831286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa978107f10 0x7fa978108350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0098a0 tx=0x7fa96c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fedb4c19e7187c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:23.917 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.118+0000 7fa97cf17700 1 -- 172.21.15.40:0/3617831286 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa96c004160 con 0x7fa978107f10 2024-06-06T03:27:23.917 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.118+0000 7fa97cf17700 1 -- 172.21.15.40:0/3617831286 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa96c004300 con 0x7fa978107f10 2024-06-06T03:27:23.917 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.118+0000 7fa97cf17700 1 -- 172.21.15.40:0/3617831286 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa96c011b50 con 0x7fa978107f10 2024-06-06T03:27:23.917 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.119+0000 7fa98017d700 1 -- 172.21.15.40:0/3617831286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa978107f10 msgr2=0x7fa978108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:23.917 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.119+0000 7fa98017d700 1 --2- 172.21.15.40:0/3617831286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa978107f10 0x7fa978108350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0098a0 tx=0x7fa96c009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:23.917 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.119+0000 7fa98017d700 1 -- 172.21.15.40:0/3617831286 shutdown_connections 2024-06-06T03:27:23.917 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.119+0000 7fa98017d700 1 --2- 172.21.15.40:0/3617831286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa978107f10 0x7fa978108350 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:23.917 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.119+0000 7fa98017d700 1 -- 172.21.15.40:0/3617831286 >> 172.21.15.40:0/3617831286 conn(0x7fa978103500 msgr2=0x7fa978105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:23.917 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.119+0000 7fa98017d700 1 -- 172.21.15.40:0/3617831286 shutdown_connections 2024-06-06T03:27:23.917 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.119+0000 7fa98017d700 1 -- 172.21.15.40:0/3617831286 wait complete. 2024-06-06T03:27:23.917 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.120+0000 7fa98017d700 1 Processor -- start 2024-06-06T03:27:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.120+0000 7fa98017d700 1 -- start start 2024-06-06T03:27:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.120+0000 7fa98017d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa978107f10 0x7fa97819be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.120+0000 7fa98017d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa978108a10 con 0x7fa978107f10 2024-06-06T03:27:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.120+0000 7fa97df19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa978107f10 0x7fa97819be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.121+0000 7fa97df19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa978107f10 0x7fa97819be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39950/0 (socket says 172.21.15.40:39950) 2024-06-06T03:27:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.121+0000 7fa97df19700 1 -- 172.21.15.40:0/3184460810 learned_addr learned my addr 172.21.15.40:0/3184460810 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.121+0000 7fa97df19700 1 -- 172.21.15.40:0/3184460810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97819c380 con 0x7fa978107f10 2024-06-06T03:27:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.121+0000 7fa97df19700 1 --2- 172.21.15.40:0/3184460810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa978107f10 0x7fa97819be40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa96c011670 tx=0x7fa96c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.121+0000 7fa96affd700 1 -- 172.21.15.40:0/3184460810 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa96c0039c0 con 0x7fa978107f10 2024-06-06T03:27:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.121+0000 7fa96affd700 1 -- 172.21.15.40:0/3184460810 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa96c011e50 con 0x7fa978107f10 2024-06-06T03:27:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.121+0000 7fa98017d700 1 -- 172.21.15.40:0/3184460810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97819c610 con 0x7fa978107f10 2024-06-06T03:27:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.121+0000 7fa96affd700 1 -- 172.21.15.40:0/3184460810 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa96c01a5c0 con 0x7fa978107f10 2024-06-06T03:27:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.121+0000 7fa98017d700 1 -- 172.21.15.40:0/3184460810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97819f300 con 0x7fa978107f10 2024-06-06T03:27:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.122+0000 7fa98017d700 1 -- 172.21.15.40:0/3184460810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa978063af0 con 0x7fa978107f10 2024-06-06T03:27:23.919 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.122+0000 7fa96affd700 1 -- 172.21.15.40:0/3184460810 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa96c024430 con 0x7fa978107f10 2024-06-06T03:27:23.919 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.123+0000 7fa96affd700 1 --2- 172.21.15.40:0/3184460810 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7fa964039050 0x7fa96403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:23.919 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.123+0000 7fa96affd700 1 -- 172.21.15.40:0/3184460810 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa96c04d410 con 0x7fa978107f10 2024-06-06T03:27:23.919 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.123+0000 7fa97d718700 1 --2- 172.21.15.40:0/3184460810 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7fa964039050 0x7fa96403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:23.919 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.123+0000 7fa97d718700 1 --2- 172.21.15.40:0/3184460810 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7fa964039050 0x7fa96403b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa974006fd0 tx=0x7fa974006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:23.919 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.128+0000 7fa96affd700 1 -- 172.21.15.40:0/3184460810 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa96c02d3f0 con 0x7fa978107f10 2024-06-06T03:27:23.919 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.336+0000 7fa98017d700 1 -- 172.21.15.40:0/3184460810 --> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fa978104750 con 0x7fa964039050 2024-06-06T03:27:23.919 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.336+0000 7fa96affd700 1 -- 172.21.15.40:0/3184460810 <== mgr.14120 v2:172.21.15.40:6800/3930494734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fa978104750 con 0x7fa964039050 2024-06-06T03:27:23.919 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.339+0000 7fa98017d700 1 -- 172.21.15.40:0/3184460810 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7fa964039050 msgr2=0x7fa96403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:23.919 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.339+0000 7fa98017d700 1 --2- 172.21.15.40:0/3184460810 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7fa964039050 0x7fa96403b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa974006fd0 tx=0x7fa974006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:23.919 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.339+0000 7fa98017d700 1 -- 172.21.15.40:0/3184460810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa978107f10 msgr2=0x7fa97819be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:23.919 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.339+0000 7fa98017d700 1 --2- 172.21.15.40:0/3184460810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa978107f10 0x7fa97819be40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa96c011670 tx=0x7fa96c011b60 comp rx=0 tx=0).stop 2024-06-06T03:27:23.919 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.339+0000 7fa98017d700 1 -- 172.21.15.40:0/3184460810 shutdown_connections 2024-06-06T03:27:23.919 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.339+0000 7fa98017d700 1 --2- 172.21.15.40:0/3184460810 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7fa964039050 0x7fa96403b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:23.920 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.339+0000 7fa98017d700 1 --2- 172.21.15.40:0/3184460810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa978107f10 0x7fa97819be40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:23.920 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.339+0000 7fa98017d700 1 -- 172.21.15.40:0/3184460810 >> 172.21.15.40:0/3184460810 conn(0x7fa978103500 msgr2=0x7fa978104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:23.920 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.339+0000 7fa98017d700 1 -- 172.21.15.40:0/3184460810 shutdown_connections 2024-06-06T03:27:23.920 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:23.339+0000 7fa98017d700 1 -- 172.21.15.40:0/3184460810 wait complete. 2024-06-06T03:27:23.920 INFO:teuthology.orchestra.run.smithi040.stdout:Generating ssh key... 2024-06-06T03:27:24.810 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: Generating public/private rsa key pair. 2024-06-06T03:27:24.810 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: Your identification has been saved in /tmp/tmpbij48dw7/key. 2024-06-06T03:27:24.810 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: Your public key has been saved in /tmp/tmpbij48dw7/key.pub. 2024-06-06T03:27:24.811 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: The key fingerprint is: 2024-06-06T03:27:24.811 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: SHA256:OGWbnL9EjsF4v8cp0+UKWx55fJxFzRbcfgmzl7eCjyg ceph-77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:27:24.811 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: The key's randomart image is: 2024-06-06T03:27:24.811 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: +---[RSA 3072]----+ 2024-06-06T03:27:24.811 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: | ...| 2024-06-06T03:27:24.811 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: | o o+| 2024-06-06T03:27:24.811 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: | o +.B| 2024-06-06T03:27:24.811 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: | B + . *+| 2024-06-06T03:27:24.811 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: | + S . . . =| 2024-06-06T03:27:24.811 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: | o B . +.oo| 2024-06-06T03:27:24.811 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: | . Bo*++.o| 2024-06-06T03:27:24.811 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: | E ooO=+.. | 2024-06-06T03:27:24.811 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: | . ++o. | 2024-06-06T03:27:24.812 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: +----[SHA256]-----+ 2024-06-06T03:27:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-mon[32220]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:27:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-mon[32220]: Generating ssh key... 2024-06-06T03:27:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' 2024-06-06T03:27:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:24 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' 2024-06-06T03:27:25.347 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.343+0000 7f2de3f9e700 1 Processor -- start 2024-06-06T03:27:25.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.343+0000 7f2de3f9e700 1 -- start start 2024-06-06T03:27:25.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.343+0000 7f2de3f9e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ddc107f80 0x7f2ddc1083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:25.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.343+0000 7f2de3f9e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc108970 con 0x7f2ddc107f80 2024-06-06T03:27:25.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.343+0000 7f2de1d3a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ddc107f80 0x7f2ddc1083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:25.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.343+0000 7f2de1d3a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ddc107f80 0x7f2ddc1083a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39966/0 (socket says 172.21.15.40:39966) 2024-06-06T03:27:25.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.343+0000 7f2de1d3a700 1 -- 172.21.15.40:0/3676755181 learned_addr learned my addr 172.21.15.40:0/3676755181 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:25.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.344+0000 7f2de1d3a700 1 -- 172.21.15.40:0/3676755181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ddc109180 con 0x7f2ddc107f80 2024-06-06T03:27:25.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.344+0000 7f2de1d3a700 1 --2- 172.21.15.40:0/3676755181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ddc107f80 0x7f2ddc1083a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80098a0 tx=0x7f2dd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc153e245cf405e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:25.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.344+0000 7f2de0d38700 1 -- 172.21.15.40:0/3676755181 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dd8004160 con 0x7f2ddc107f80 2024-06-06T03:27:25.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.344+0000 7f2de0d38700 1 -- 172.21.15.40:0/3676755181 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2dd8004300 con 0x7f2ddc107f80 2024-06-06T03:27:25.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.344+0000 7f2de0d38700 1 -- 172.21.15.40:0/3676755181 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dd8011b50 con 0x7f2ddc107f80 2024-06-06T03:27:25.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.345+0000 7f2de3f9e700 1 -- 172.21.15.40:0/3676755181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ddc107f80 msgr2=0x7f2ddc1083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:25.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.345+0000 7f2de3f9e700 1 --2- 172.21.15.40:0/3676755181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ddc107f80 0x7f2ddc1083a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80098a0 tx=0x7f2dd8009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:25.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.345+0000 7f2de3f9e700 1 -- 172.21.15.40:0/3676755181 shutdown_connections 2024-06-06T03:27:25.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.345+0000 7f2de3f9e700 1 --2- 172.21.15.40:0/3676755181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ddc107f80 0x7f2ddc1083a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:25.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.345+0000 7f2de3f9e700 1 -- 172.21.15.40:0/3676755181 >> 172.21.15.40:0/3676755181 conn(0x7f2ddc103580 msgr2=0x7f2ddc1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:25.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.345+0000 7f2de3f9e700 1 -- 172.21.15.40:0/3676755181 shutdown_connections 2024-06-06T03:27:25.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.345+0000 7f2de3f9e700 1 -- 172.21.15.40:0/3676755181 wait complete. 2024-06-06T03:27:25.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.346+0000 7f2de3f9e700 1 Processor -- start 2024-06-06T03:27:25.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.346+0000 7f2de3f9e700 1 -- start start 2024-06-06T03:27:25.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.346+0000 7f2de3f9e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ddc107f80 0x7f2ddc07e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:25.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.346+0000 7f2de3f9e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc10c3b0 con 0x7f2ddc107f80 2024-06-06T03:27:25.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.346+0000 7f2de1d3a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ddc107f80 0x7f2ddc07e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:25.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.346+0000 7f2de1d3a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ddc107f80 0x7f2ddc07e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39978/0 (socket says 172.21.15.40:39978) 2024-06-06T03:27:25.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.346+0000 7f2de1d3a700 1 -- 172.21.15.40:0/1943358340 learned_addr learned my addr 172.21.15.40:0/1943358340 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:25.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.346+0000 7f2de1d3a700 1 -- 172.21.15.40:0/1943358340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ddc07e910 con 0x7f2ddc107f80 2024-06-06T03:27:25.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.346+0000 7f2de1d3a700 1 --2- 172.21.15.40:0/1943358340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ddc107f80 0x7f2ddc07e3d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8011850 tx=0x7f2dd8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:25.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.347+0000 7f2dd2ffd700 1 -- 172.21.15.40:0/1943358340 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dd80039e0 con 0x7f2ddc107f80 2024-06-06T03:27:25.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.347+0000 7f2dd2ffd700 1 -- 172.21.15.40:0/1943358340 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2dd8011e50 con 0x7f2ddc107f80 2024-06-06T03:27:25.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.347+0000 7f2de3f9e700 1 -- 172.21.15.40:0/1943358340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ddc07cb00 con 0x7f2ddc107f80 2024-06-06T03:27:25.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.347+0000 7f2dd2ffd700 1 -- 172.21.15.40:0/1943358340 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dd80245a0 con 0x7f2ddc107f80 2024-06-06T03:27:25.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.347+0000 7f2de3f9e700 1 -- 172.21.15.40:0/1943358340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ddc07d040 con 0x7f2ddc107f80 2024-06-06T03:27:25.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.348+0000 7f2dd2ffd700 1 -- 172.21.15.40:0/1943358340 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2dd8022020 con 0x7f2ddc107f80 2024-06-06T03:27:25.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.348+0000 7f2de3f9e700 1 -- 172.21.15.40:0/1943358340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ddc10c530 con 0x7f2ddc107f80 2024-06-06T03:27:25.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.348+0000 7f2dd2ffd700 1 --2- 172.21.15.40:0/1943358340 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f2dc8039050 0x7f2dc803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:25.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.348+0000 7f2dd2ffd700 1 -- 172.21.15.40:0/1943358340 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2dd804cf30 con 0x7f2ddc107f80 2024-06-06T03:27:25.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.348+0000 7f2de1539700 1 --2- 172.21.15.40:0/1943358340 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f2dc8039050 0x7f2dc803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:25.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.348+0000 7f2de1539700 1 --2- 172.21.15.40:0/1943358340 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f2dc8039050 0x7f2dc803b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc006fd0 tx=0x7f2dcc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:25.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.353+0000 7f2dd2ffd700 1 -- 172.21.15.40:0/1943358340 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2dd801a950 con 0x7f2ddc107f80 2024-06-06T03:27:25.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.559+0000 7f2de3f9e700 1 -- 172.21.15.40:0/1943358340 --> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f2ddc07d880 con 0x7f2dc8039050 2024-06-06T03:27:25.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.809+0000 7f2dd2ffd700 1 -- 172.21.15.40:0/1943358340 <== mgr.14120 v2:172.21.15.40:6800/3930494734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2ddc07d880 con 0x7f2dc8039050 2024-06-06T03:27:25.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.811+0000 7f2de3f9e700 1 -- 172.21.15.40:0/1943358340 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f2dc8039050 msgr2=0x7f2dc803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:25.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.812+0000 7f2de3f9e700 1 --2- 172.21.15.40:0/1943358340 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f2dc8039050 0x7f2dc803b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc006fd0 tx=0x7f2dcc006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:25.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.812+0000 7f2de3f9e700 1 -- 172.21.15.40:0/1943358340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ddc107f80 msgr2=0x7f2ddc07e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:25.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.812+0000 7f2de3f9e700 1 --2- 172.21.15.40:0/1943358340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ddc107f80 0x7f2ddc07e3d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8011850 tx=0x7f2dd8011c10 comp rx=0 tx=0).stop 2024-06-06T03:27:25.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.812+0000 7f2de3f9e700 1 -- 172.21.15.40:0/1943358340 shutdown_connections 2024-06-06T03:27:25.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.812+0000 7f2de3f9e700 1 --2- 172.21.15.40:0/1943358340 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f2dc8039050 0x7f2dc803b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:25.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.812+0000 7f2de3f9e700 1 --2- 172.21.15.40:0/1943358340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ddc107f80 0x7f2ddc07e3d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:25.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.812+0000 7f2de3f9e700 1 -- 172.21.15.40:0/1943358340 >> 172.21.15.40:0/1943358340 conn(0x7f2ddc103580 msgr2=0x7f2ddc1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:25.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.812+0000 7f2de3f9e700 1 -- 172.21.15.40:0/1943358340 shutdown_connections 2024-06-06T03:27:25.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:24.812+0000 7f2de3f9e700 1 -- 172.21.15.40:0/1943358340 wait complete. 2024-06-06T03:27:26.555 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjInNI2ZvFZ4i16r6bn9u8f5UhAAdNkZROqZ0tGeUzNC4WqwzX3iMvIuRl9FVQSns66EM6XIxe3wCXWsuANkz4h89qYcX+xq/SNCUZaAe4F2T3Dfqff3fuereto7pHD6mO7+WduqXIGzyHoPXsTa/NL0oTodsnWdHa/pWATJu8WRI9CRyiT2q+Bmv30EMLxjqZP7HHHvj8/RwOg6cTB7lZq4JZZho6QHZUEHvT9I0EkmRRBw7Co2VTSqpJ1Zi9IWHrLnlnJvpat8NBbDS3qa1twZTfPxU1eBlkwHehkaoEFs79aH5nGewx7swSRbATFAQqoEI45UJf679D8KLv4vKBHdKQWLlI1DfW1Darv3wMAQLV2qXddtNqxvq6K4Z1S0vAinlFREhI3sAQTJUp+1N8VMt0vlsPIKJnPFIfrK1MDtuo9ZO4CnwrIIT9BZhFeTaUgBN6NL8TPECAAdVP4+vjPkqF2e4ZZo/2HCn9bJ2FycqE+kXNpKbN6NYp7ykUcTM= ceph-77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:27:26.648 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.771+0000 7f518b16b700 1 Processor -- start 2024-06-06T03:27:26.648 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.771+0000 7f518b16b700 1 -- start start 2024-06-06T03:27:26.649 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.772+0000 7f518b16b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5184107f00 0x7f5184108340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:26.649 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.772+0000 7f518b16b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5184108880 con 0x7f5184107f00 2024-06-06T03:27:26.649 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.772+0000 7f5188f07700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5184107f00 0x7f5184108340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:26.649 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.772+0000 7f5188f07700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5184107f00 0x7f5184108340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39984/0 (socket says 172.21.15.40:39984) 2024-06-06T03:27:26.649 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.772+0000 7f5188f07700 1 -- 172.21.15.40:0/1719831400 learned_addr learned my addr 172.21.15.40:0/1719831400 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:26.649 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.772+0000 7f5188f07700 1 -- 172.21.15.40:0/1719831400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51841090e0 con 0x7f5184107f00 2024-06-06T03:27:26.649 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.772+0000 7f5188f07700 1 --2- 172.21.15.40:0/1719831400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5184107f00 0x7f5184108340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f51780098a0 tx=0x7f5178009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4f3d8211640a7d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:26.649 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.772+0000 7f51837fe700 1 -- 172.21.15.40:0/1719831400 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5178004160 con 0x7f5184107f00 2024-06-06T03:27:26.649 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.772+0000 7f51837fe700 1 -- 172.21.15.40:0/1719831400 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5178004300 con 0x7f5184107f00 2024-06-06T03:27:26.649 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.772+0000 7f51837fe700 1 -- 172.21.15.40:0/1719831400 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5178011b50 con 0x7f5184107f00 2024-06-06T03:27:26.649 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.773+0000 7f518b16b700 1 -- 172.21.15.40:0/1719831400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5184107f00 msgr2=0x7f5184108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:26.649 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.773+0000 7f518b16b700 1 --2- 172.21.15.40:0/1719831400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5184107f00 0x7f5184108340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f51780098a0 tx=0x7f5178009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:26.650 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.773+0000 7f518b16b700 1 -- 172.21.15.40:0/1719831400 shutdown_connections 2024-06-06T03:27:26.650 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.773+0000 7f518b16b700 1 --2- 172.21.15.40:0/1719831400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5184107f00 0x7f5184108340 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:26.650 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.773+0000 7f518b16b700 1 -- 172.21.15.40:0/1719831400 >> 172.21.15.40:0/1719831400 conn(0x7f51841034f0 msgr2=0x7f5184105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:26.650 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.773+0000 7f518b16b700 1 -- 172.21.15.40:0/1719831400 shutdown_connections 2024-06-06T03:27:26.650 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.773+0000 7f518b16b700 1 -- 172.21.15.40:0/1719831400 wait complete. 2024-06-06T03:27:26.650 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.774+0000 7f518b16b700 1 Processor -- start 2024-06-06T03:27:26.650 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.774+0000 7f518b16b700 1 -- start start 2024-06-06T03:27:26.650 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.774+0000 7f518b16b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5184107f00 0x7f518419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:26.650 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.774+0000 7f518b16b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5184108a00 con 0x7f5184107f00 2024-06-06T03:27:26.650 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.774+0000 7f5188f07700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5184107f00 0x7f518419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:26.650 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.774+0000 7f5188f07700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5184107f00 0x7f518419be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39986/0 (socket says 172.21.15.40:39986) 2024-06-06T03:27:26.650 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.774+0000 7f5188f07700 1 -- 172.21.15.40:0/1828193859 learned_addr learned my addr 172.21.15.40:0/1828193859 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:26.650 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.774+0000 7f5188f07700 1 -- 172.21.15.40:0/1828193859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518419c350 con 0x7f5184107f00 2024-06-06T03:27:26.651 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.774+0000 7f5188f07700 1 --2- 172.21.15.40:0/1828193859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5184107f00 0x7f518419be10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5178011670 tx=0x7f5178011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:26.651 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.775+0000 7f5181ffb700 1 -- 172.21.15.40:0/1828193859 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51780039c0 con 0x7f5184107f00 2024-06-06T03:27:26.651 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.775+0000 7f5181ffb700 1 -- 172.21.15.40:0/1828193859 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5178011e50 con 0x7f5184107f00 2024-06-06T03:27:26.651 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.775+0000 7f518b16b700 1 -- 172.21.15.40:0/1828193859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518419c5e0 con 0x7f5184107f00 2024-06-06T03:27:26.651 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.775+0000 7f5181ffb700 1 -- 172.21.15.40:0/1828193859 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f517801a5c0 con 0x7f5184107f00 2024-06-06T03:27:26.651 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.775+0000 7f518b16b700 1 -- 172.21.15.40:0/1828193859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518419f2d0 con 0x7f5184107f00 2024-06-06T03:27:26.651 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.776+0000 7f5181ffb700 1 -- 172.21.15.40:0/1828193859 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f517801a870 con 0x7f5184107f00 2024-06-06T03:27:26.651 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.776+0000 7f518b16b700 1 -- 172.21.15.40:0/1828193859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5184063af0 con 0x7f5184107f00 2024-06-06T03:27:26.651 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.776+0000 7f5181ffb700 1 --2- 172.21.15.40:0/1828193859 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5174039050 0x7f517403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:26.651 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.776+0000 7f5181ffb700 1 -- 172.21.15.40:0/1828193859 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f517804d460 con 0x7f5184107f00 2024-06-06T03:27:26.651 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.776+0000 7f5183fff700 1 --2- 172.21.15.40:0/1828193859 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5174039050 0x7f517403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:26.651 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.777+0000 7f5183fff700 1 --2- 172.21.15.40:0/1828193859 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5174039050 0x7f517403b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5170006fd0 tx=0x7f5170006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:26.652 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.781+0000 7f5181ffb700 1 -- 172.21.15.40:0/1828193859 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5178019c70 con 0x7f5184107f00 2024-06-06T03:27:26.652 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.989+0000 7f518b16b700 1 -- 172.21.15.40:0/1828193859 --> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f5184104720 con 0x7f5174039050 2024-06-06T03:27:26.652 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.990+0000 7f5181ffb700 1 -- 172.21.15.40:0/1828193859 <== mgr.14120 v2:172.21.15.40:6800/3930494734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f5184104720 con 0x7f5174039050 2024-06-06T03:27:26.652 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.992+0000 7f518b16b700 1 -- 172.21.15.40:0/1828193859 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5174039050 msgr2=0x7f517403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:26.652 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.992+0000 7f518b16b700 1 --2- 172.21.15.40:0/1828193859 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5174039050 0x7f517403b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5170006fd0 tx=0x7f5170006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:26.652 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.992+0000 7f518b16b700 1 -- 172.21.15.40:0/1828193859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5184107f00 msgr2=0x7f518419be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:26.652 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.992+0000 7f518b16b700 1 --2- 172.21.15.40:0/1828193859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5184107f00 0x7f518419be10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5178011670 tx=0x7f5178011b60 comp rx=0 tx=0).stop 2024-06-06T03:27:26.652 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.992+0000 7f518b16b700 1 -- 172.21.15.40:0/1828193859 shutdown_connections 2024-06-06T03:27:26.652 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.992+0000 7f518b16b700 1 --2- 172.21.15.40:0/1828193859 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5174039050 0x7f517403b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:26.652 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.992+0000 7f518b16b700 1 --2- 172.21.15.40:0/1828193859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5184107f00 0x7f518419be10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:26.652 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.992+0000 7f518b16b700 1 -- 172.21.15.40:0/1828193859 >> 172.21.15.40:0/1828193859 conn(0x7f51841034f0 msgr2=0x7f5184104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:26.652 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.992+0000 7f518b16b700 1 -- 172.21.15.40:0/1828193859 shutdown_connections 2024-06-06T03:27:26.652 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:25.992+0000 7f518b16b700 1 -- 172.21.15.40:0/1828193859 wait complete. 2024-06-06T03:27:26.653 INFO:teuthology.orchestra.run.smithi040.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-06T03:27:26.653 INFO:teuthology.orchestra.run.smithi040.stdout:Adding key to root@localhost authorized_keys... 2024-06-06T03:27:26.653 INFO:teuthology.orchestra.run.smithi040.stdout:Adding host smithi040... 2024-06-06T03:27:26.815 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:26 smithi040 ceph-mon[32220]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:27:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:27 smithi040 ceph-mon[32220]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi040", "addr": "172.21.15.40", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:27:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:28 smithi040 ceph-mon[32220]: Deploying cephadm binary to smithi040 2024-06-06T03:27:32.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Added host 'smithi040' with addr '172.21.15.40' 2024-06-06T03:27:32.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.025+0000 7f8d24a86700 1 Processor -- start 2024-06-06T03:27:32.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.025+0000 7f8d24a86700 1 -- start start 2024-06-06T03:27:32.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.025+0000 7f8d24a86700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d20106e70 0x7f8d20107290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:32.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.025+0000 7f8d24a86700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8d201077d0 con 0x7f8d20106e70 2024-06-06T03:27:32.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.025+0000 7f8d1e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d20106e70 0x7f8d20107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:32.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.025+0000 7f8d1e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d20106e70 0x7f8d20107290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40000/0 (socket says 172.21.15.40:40000) 2024-06-06T03:27:32.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.025+0000 7f8d1e59c700 1 -- 172.21.15.40:0/1714355641 learned_addr learned my addr 172.21.15.40:0/1714355641 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:32.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.025+0000 7f8d1e59c700 1 -- 172.21.15.40:0/1714355641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d20107fd0 con 0x7f8d20106e70 2024-06-06T03:27:32.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.026+0000 7f8d1e59c700 1 --2- 172.21.15.40:0/1714355641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d20106e70 0x7f8d20107290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8d100098a0 tx=0x7f8d10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20b7c3ed7e9f1a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:32.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.026+0000 7f8d1d59a700 1 -- 172.21.15.40:0/1714355641 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d10004160 con 0x7f8d20106e70 2024-06-06T03:27:32.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.026+0000 7f8d1d59a700 1 -- 172.21.15.40:0/1714355641 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d10004300 con 0x7f8d20106e70 2024-06-06T03:27:32.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.026+0000 7f8d1d59a700 1 -- 172.21.15.40:0/1714355641 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d10011b50 con 0x7f8d20106e70 2024-06-06T03:27:32.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.026+0000 7f8d24a86700 1 -- 172.21.15.40:0/1714355641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d20106e70 msgr2=0x7f8d20107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:32.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.026+0000 7f8d24a86700 1 --2- 172.21.15.40:0/1714355641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d20106e70 0x7f8d20107290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8d100098a0 tx=0x7f8d10009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:32.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.027+0000 7f8d24a86700 1 -- 172.21.15.40:0/1714355641 shutdown_connections 2024-06-06T03:27:32.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.027+0000 7f8d24a86700 1 --2- 172.21.15.40:0/1714355641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d20106e70 0x7f8d20107290 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:32.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.027+0000 7f8d24a86700 1 -- 172.21.15.40:0/1714355641 >> 172.21.15.40:0/1714355641 conn(0x7f8d20102470 msgr2=0x7f8d20104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:32.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.027+0000 7f8d24a86700 1 -- 172.21.15.40:0/1714355641 shutdown_connections 2024-06-06T03:27:32.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.027+0000 7f8d24a86700 1 -- 172.21.15.40:0/1714355641 wait complete. 2024-06-06T03:27:32.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.027+0000 7f8d24a86700 1 Processor -- start 2024-06-06T03:27:32.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.028+0000 7f8d24a86700 1 -- start start 2024-06-06T03:27:32.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.028+0000 7f8d24a86700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d20106e70 0x7f8d2007c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:32.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.028+0000 7f8d24a86700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2010b290 con 0x7f8d20106e70 2024-06-06T03:27:32.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.028+0000 7f8d1e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d20106e70 0x7f8d2007c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:32.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.028+0000 7f8d1e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d20106e70 0x7f8d2007c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40006/0 (socket says 172.21.15.40:40006) 2024-06-06T03:27:32.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.028+0000 7f8d1e59c700 1 -- 172.21.15.40:0/4120214731 learned_addr learned my addr 172.21.15.40:0/4120214731 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:32.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.028+0000 7f8d1e59c700 1 -- 172.21.15.40:0/4120214731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2019c980 con 0x7f8d20106e70 2024-06-06T03:27:32.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.028+0000 7f8d1e59c700 1 --2- 172.21.15.40:0/4120214731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d20106e70 0x7f8d2007c310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009fb0 tx=0x7f8d10011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:32.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.029+0000 7f8d0f7fe700 1 -- 172.21.15.40:0/4120214731 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d100038b0 con 0x7f8d20106e70 2024-06-06T03:27:32.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.029+0000 7f8d0f7fe700 1 -- 172.21.15.40:0/4120214731 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d10011e50 con 0x7f8d20106e70 2024-06-06T03:27:32.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.029+0000 7f8d24a86700 1 -- 172.21.15.40:0/4120214731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d201a0f90 con 0x7f8d20106e70 2024-06-06T03:27:32.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.029+0000 7f8d0f7fe700 1 -- 172.21.15.40:0/4120214731 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d10019420 con 0x7f8d20106e70 2024-06-06T03:27:32.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.029+0000 7f8d24a86700 1 -- 172.21.15.40:0/4120214731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d201a13b0 con 0x7f8d20106e70 2024-06-06T03:27:32.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.029+0000 7f8d0f7fe700 1 -- 172.21.15.40:0/4120214731 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8d10019720 con 0x7f8d20106e70 2024-06-06T03:27:32.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.030+0000 7f8d0f7fe700 1 --2- 172.21.15.40:0/4120214731 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f8d08039000 0x7f8d0803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:32.248 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.030+0000 7f8d0f7fe700 1 -- 172.21.15.40:0/4120214731 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8d1004d350 con 0x7f8d20106e70 2024-06-06T03:27:32.248 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.030+0000 7f8d1dd9b700 1 --2- 172.21.15.40:0/4120214731 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f8d08039000 0x7f8d0803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:32.248 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.030+0000 7f8d24a86700 1 -- 172.21.15.40:0/4120214731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d20063af0 con 0x7f8d20106e70 2024-06-06T03:27:32.248 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.030+0000 7f8d1dd9b700 1 --2- 172.21.15.40:0/4120214731 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f8d08039000 0x7f8d0803b4c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8d14006fd0 tx=0x7f8d14006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:32.248 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.035+0000 7f8d0f7fe700 1 -- 172.21.15.40:0/4120214731 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8d10023c90 con 0x7f8d20106e70 2024-06-06T03:27:32.248 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:27.246+0000 7f8d24a86700 1 -- 172.21.15.40:0/4120214731 --> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi040", "addr": "172.21.15.40", "target": ["mon-mgr", ""]}) -- 0x7f8d20103700 con 0x7f8d08039000 2024-06-06T03:27:32.248 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:31.511+0000 7f8d0f7fe700 1 -- 172.21.15.40:0/4120214731 <== mgr.14120 v2:172.21.15.40:6800/3930494734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f8d20103700 con 0x7f8d08039000 2024-06-06T03:27:32.248 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:31.514+0000 7f8d24a86700 1 -- 172.21.15.40:0/4120214731 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f8d08039000 msgr2=0x7f8d0803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:32.248 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:31.514+0000 7f8d24a86700 1 --2- 172.21.15.40:0/4120214731 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f8d08039000 0x7f8d0803b4c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8d14006fd0 tx=0x7f8d14006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:32.248 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:31.514+0000 7f8d24a86700 1 -- 172.21.15.40:0/4120214731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d20106e70 msgr2=0x7f8d2007c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:32.248 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:31.514+0000 7f8d24a86700 1 --2- 172.21.15.40:0/4120214731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d20106e70 0x7f8d2007c310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009fb0 tx=0x7f8d10011bf0 comp rx=0 tx=0).stop 2024-06-06T03:27:32.248 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:31.514+0000 7f8d24a86700 1 -- 172.21.15.40:0/4120214731 shutdown_connections 2024-06-06T03:27:32.248 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:31.514+0000 7f8d24a86700 1 --2- 172.21.15.40:0/4120214731 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f8d08039000 0x7f8d0803b4c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:32.248 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:31.514+0000 7f8d24a86700 1 --2- 172.21.15.40:0/4120214731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d20106e70 0x7f8d2007c310 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:32.249 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:31.514+0000 7f8d24a86700 1 -- 172.21.15.40:0/4120214731 >> 172.21.15.40:0/4120214731 conn(0x7f8d20102470 msgr2=0x7f8d20102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:32.249 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:31.514+0000 7f8d24a86700 1 -- 172.21.15.40:0/4120214731 shutdown_connections 2024-06-06T03:27:32.249 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:31.514+0000 7f8d24a86700 1 -- 172.21.15.40:0/4120214731 wait complete. 2024-06-06T03:27:32.249 INFO:teuthology.orchestra.run.smithi040.stdout:Deploying unmanaged mon service... 2024-06-06T03:27:32.608 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:32 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' 2024-06-06T03:27:32.608 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:32 smithi040 ceph-mon[32220]: Added host smithi040 2024-06-06T03:27:32.608 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:32 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:27:33.637 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-06T03:27:33.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.647+0000 7ff718e2b700 1 Processor -- start 2024-06-06T03:27:33.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.647+0000 7ff718e2b700 1 -- start start 2024-06-06T03:27:33.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.647+0000 7ff718e2b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7141089c0 0x7ff71410adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:33.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.647+0000 7ff718e2b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff71407be80 con 0x7ff7141089c0 2024-06-06T03:27:33.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.647+0000 7ff71259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7141089c0 0x7ff71410adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:33.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.647+0000 7ff71259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7141089c0 0x7ff71410adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41600/0 (socket says 172.21.15.40:41600) 2024-06-06T03:27:33.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.648+0000 7ff71259c700 1 -- 172.21.15.40:0/770143672 learned_addr learned my addr 172.21.15.40:0/770143672 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:33.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.648+0000 7ff71259c700 1 -- 172.21.15.40:0/770143672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71410b2f0 con 0x7ff7141089c0 2024-06-06T03:27:33.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.648+0000 7ff71259c700 1 --2- 172.21.15.40:0/770143672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7141089c0 0x7ff71410adb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff7040098a0 tx=0x7ff704009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cae0d619b2acc77 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:33.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.648+0000 7ff71159a700 1 -- 172.21.15.40:0/770143672 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff704004160 con 0x7ff7141089c0 2024-06-06T03:27:33.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.648+0000 7ff71159a700 1 -- 172.21.15.40:0/770143672 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff704004300 con 0x7ff7141089c0 2024-06-06T03:27:33.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.648+0000 7ff71159a700 1 -- 172.21.15.40:0/770143672 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff704011b50 con 0x7ff7141089c0 2024-06-06T03:27:33.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.649+0000 7ff718e2b700 1 -- 172.21.15.40:0/770143672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7141089c0 msgr2=0x7ff71410adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:33.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.649+0000 7ff718e2b700 1 --2- 172.21.15.40:0/770143672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7141089c0 0x7ff71410adb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff7040098a0 tx=0x7ff704009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:33.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.649+0000 7ff718e2b700 1 -- 172.21.15.40:0/770143672 shutdown_connections 2024-06-06T03:27:33.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.649+0000 7ff718e2b700 1 --2- 172.21.15.40:0/770143672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7141089c0 0x7ff71410adb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:33.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.649+0000 7ff718e2b700 1 -- 172.21.15.40:0/770143672 >> 172.21.15.40:0/770143672 conn(0x7ff7141023f0 msgr2=0x7ff714104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:33.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.649+0000 7ff718e2b700 1 -- 172.21.15.40:0/770143672 shutdown_connections 2024-06-06T03:27:33.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.649+0000 7ff718e2b700 1 -- 172.21.15.40:0/770143672 wait complete. 2024-06-06T03:27:33.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.650+0000 7ff718e2b700 1 Processor -- start 2024-06-06T03:27:33.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.650+0000 7ff718e2b700 1 -- start start 2024-06-06T03:27:33.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.650+0000 7ff718e2b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7141089c0 0x7ff71419bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:33.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.650+0000 7ff718e2b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff71410c240 con 0x7ff7141089c0 2024-06-06T03:27:33.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.650+0000 7ff71259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7141089c0 0x7ff71419bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:33.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.650+0000 7ff71259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7141089c0 0x7ff71419bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41602/0 (socket says 172.21.15.40:41602) 2024-06-06T03:27:33.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.650+0000 7ff71259c700 1 -- 172.21.15.40:0/4046705429 learned_addr learned my addr 172.21.15.40:0/4046705429 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:33.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.650+0000 7ff71259c700 1 -- 172.21.15.40:0/4046705429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71419c400 con 0x7ff7141089c0 2024-06-06T03:27:33.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.651+0000 7ff71259c700 1 --2- 172.21.15.40:0/4046705429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7141089c0 0x7ff71419bec0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff704009fb0 tx=0x7ff704011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:33.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.651+0000 7ff7037fe700 1 -- 172.21.15.40:0/4046705429 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7040038b0 con 0x7ff7141089c0 2024-06-06T03:27:33.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.651+0000 7ff7037fe700 1 -- 172.21.15.40:0/4046705429 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff704011e50 con 0x7ff7141089c0 2024-06-06T03:27:33.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.651+0000 7ff718e2b700 1 -- 172.21.15.40:0/4046705429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71419c690 con 0x7ff7141089c0 2024-06-06T03:27:33.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.651+0000 7ff7037fe700 1 -- 172.21.15.40:0/4046705429 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff704019420 con 0x7ff7141089c0 2024-06-06T03:27:33.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.651+0000 7ff718e2b700 1 -- 172.21.15.40:0/4046705429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71419f380 con 0x7ff7141089c0 2024-06-06T03:27:33.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.652+0000 7ff7037fe700 1 -- 172.21.15.40:0/4046705429 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff704019720 con 0x7ff7141089c0 2024-06-06T03:27:33.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.652+0000 7ff7037fe700 1 --2- 172.21.15.40:0/4046705429 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7ff6fc039000 0x7ff6fc03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:33.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.652+0000 7ff7037fe700 1 -- 172.21.15.40:0/4046705429 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff70404d310 con 0x7ff7141089c0 2024-06-06T03:27:33.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.652+0000 7ff711d9b700 1 --2- 172.21.15.40:0/4046705429 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7ff6fc039000 0x7ff6fc03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:33.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.652+0000 7ff718e2b700 1 -- 172.21.15.40:0/4046705429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff714063af0 con 0x7ff7141089c0 2024-06-06T03:27:33.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.653+0000 7ff711d9b700 1 --2- 172.21.15.40:0/4046705429 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7ff6fc039000 0x7ff6fc03b4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff708006fd0 tx=0x7ff708006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:33.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.658+0000 7ff7037fe700 1 -- 172.21.15.40:0/4046705429 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff70400bc70 con 0x7ff7141089c0 2024-06-06T03:27:33.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.888+0000 7ff718e2b700 1 -- 172.21.15.40:0/4046705429 --> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff7141036b0 con 0x7ff6fc039000 2024-06-06T03:27:33.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.895+0000 7ff7037fe700 1 -- 172.21.15.40:0/4046705429 <== mgr.14120 v2:172.21.15.40:6800/3930494734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff7141036b0 con 0x7ff6fc039000 2024-06-06T03:27:33.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.897+0000 7ff718e2b700 1 -- 172.21.15.40:0/4046705429 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7ff6fc039000 msgr2=0x7ff6fc03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:33.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.897+0000 7ff718e2b700 1 --2- 172.21.15.40:0/4046705429 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7ff6fc039000 0x7ff6fc03b4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff708006fd0 tx=0x7ff708006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:33.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.897+0000 7ff718e2b700 1 -- 172.21.15.40:0/4046705429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7141089c0 msgr2=0x7ff71419bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:33.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.897+0000 7ff718e2b700 1 --2- 172.21.15.40:0/4046705429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7141089c0 0x7ff71419bec0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff704009fb0 tx=0x7ff704011bf0 comp rx=0 tx=0).stop 2024-06-06T03:27:33.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.897+0000 7ff718e2b700 1 -- 172.21.15.40:0/4046705429 shutdown_connections 2024-06-06T03:27:33.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.897+0000 7ff718e2b700 1 --2- 172.21.15.40:0/4046705429 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7ff6fc039000 0x7ff6fc03b4c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:33.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.897+0000 7ff718e2b700 1 --2- 172.21.15.40:0/4046705429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7141089c0 0x7ff71419bec0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:33.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.897+0000 7ff718e2b700 1 -- 172.21.15.40:0/4046705429 >> 172.21.15.40:0/4046705429 conn(0x7ff7141023f0 msgr2=0x7ff714102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:33.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.897+0000 7ff718e2b700 1 -- 172.21.15.40:0/4046705429 shutdown_connections 2024-06-06T03:27:33.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:32.897+0000 7ff718e2b700 1 -- 172.21.15.40:0/4046705429 wait complete. 2024-06-06T03:27:33.642 INFO:teuthology.orchestra.run.smithi040.stdout:Deploying unmanaged mgr service... 2024-06-06T03:27:34.241 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:33 smithi040 ceph-mon[32220]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:27:34.242 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:33 smithi040 ceph-mon[32220]: Saving service mon spec with placement count:5 2024-06-06T03:27:34.242 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:33 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' 2024-06-06T03:27:35.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:35 smithi040 ceph-mon[32220]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:27:35.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:35 smithi040 ceph-mon[32220]: Saving service mgr spec with placement count:2 2024-06-06T03:27:35.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:35 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' 2024-06-06T03:27:35.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:35 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' 2024-06-06T03:27:35.608 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-06T03:27:35.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.022+0000 7fbfd4987700 1 Processor -- start 2024-06-06T03:27:35.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.022+0000 7fbfd4987700 1 -- start start 2024-06-06T03:27:35.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.022+0000 7fbfd4987700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfd0107f00 0x7fbfd0108340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:35.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.022+0000 7fbfd4987700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0108880 con 0x7fbfd0107f00 2024-06-06T03:27:35.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.023+0000 7fbfce59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfd0107f00 0x7fbfd0108340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:35.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.023+0000 7fbfce59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfd0107f00 0x7fbfd0108340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41612/0 (socket says 172.21.15.40:41612) 2024-06-06T03:27:35.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.023+0000 7fbfce59c700 1 -- 172.21.15.40:0/3707003864 learned_addr learned my addr 172.21.15.40:0/3707003864 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:35.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.023+0000 7fbfce59c700 1 -- 172.21.15.40:0/3707003864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd01090e0 con 0x7fbfd0107f00 2024-06-06T03:27:35.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.023+0000 7fbfce59c700 1 --2- 172.21.15.40:0/3707003864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfd0107f00 0x7fbfd0108340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009e80 tx=0x7fbfc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edea27ff98432f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:35.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.023+0000 7fbfcd59a700 1 -- 172.21.15.40:0/3707003864 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfc0017070 con 0x7fbfd0107f00 2024-06-06T03:27:35.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.023+0000 7fbfcd59a700 1 -- 172.21.15.40:0/3707003864 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbfc000b8c0 con 0x7fbfd0107f00 2024-06-06T03:27:35.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.023+0000 7fbfcd59a700 1 -- 172.21.15.40:0/3707003864 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfc00045c0 con 0x7fbfd0107f00 2024-06-06T03:27:35.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.024+0000 7fbfd4987700 1 -- 172.21.15.40:0/3707003864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfd0107f00 msgr2=0x7fbfd0108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:35.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.024+0000 7fbfd4987700 1 --2- 172.21.15.40:0/3707003864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfd0107f00 0x7fbfd0108340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009e80 tx=0x7fbfc000b330 comp rx=0 tx=0).stop 2024-06-06T03:27:35.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.024+0000 7fbfd4987700 1 -- 172.21.15.40:0/3707003864 shutdown_connections 2024-06-06T03:27:35.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.024+0000 7fbfd4987700 1 --2- 172.21.15.40:0/3707003864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfd0107f00 0x7fbfd0108340 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:35.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.024+0000 7fbfd4987700 1 -- 172.21.15.40:0/3707003864 >> 172.21.15.40:0/3707003864 conn(0x7fbfd01034f0 msgr2=0x7fbfd0105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:35.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.024+0000 7fbfd4987700 1 -- 172.21.15.40:0/3707003864 shutdown_connections 2024-06-06T03:27:35.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.024+0000 7fbfd4987700 1 -- 172.21.15.40:0/3707003864 wait complete. 2024-06-06T03:27:35.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.025+0000 7fbfd4987700 1 Processor -- start 2024-06-06T03:27:35.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.025+0000 7fbfd4987700 1 -- start start 2024-06-06T03:27:35.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.025+0000 7fbfd4987700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfd0107f00 0x7fbfd007e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:35.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.025+0000 7fbfd4987700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0108a00 con 0x7fbfd0107f00 2024-06-06T03:27:35.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.025+0000 7fbfce59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfd0107f00 0x7fbfd007e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:35.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.025+0000 7fbfce59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfd0107f00 0x7fbfd007e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41622/0 (socket says 172.21.15.40:41622) 2024-06-06T03:27:35.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.025+0000 7fbfce59c700 1 -- 172.21.15.40:0/3799193154 learned_addr learned my addr 172.21.15.40:0/3799193154 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:35.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.025+0000 7fbfce59c700 1 -- 172.21.15.40:0/3799193154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd007e910 con 0x7fbfd0107f00 2024-06-06T03:27:35.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.025+0000 7fbfce59c700 1 --2- 172.21.15.40:0/3799193154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfd0107f00 0x7fbfd007e3d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00072a0 tx=0x7fbfc000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:35.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.026+0000 7fbfc77fe700 1 -- 172.21.15.40:0/3799193154 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfc0020020 con 0x7fbfd0107f00 2024-06-06T03:27:35.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.026+0000 7fbfd4987700 1 -- 172.21.15.40:0/3799193154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd007cb00 con 0x7fbfd0107f00 2024-06-06T03:27:35.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.026+0000 7fbfc77fe700 1 -- 172.21.15.40:0/3799193154 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbfc0003b50 con 0x7fbfd0107f00 2024-06-06T03:27:35.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.026+0000 7fbfc77fe700 1 -- 172.21.15.40:0/3799193154 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfc0017040 con 0x7fbfd0107f00 2024-06-06T03:27:35.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.026+0000 7fbfd4987700 1 -- 172.21.15.40:0/3799193154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd007d040 con 0x7fbfd0107f00 2024-06-06T03:27:35.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.026+0000 7fbfc77fe700 1 -- 172.21.15.40:0/3799193154 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbfc0023020 con 0x7fbfd0107f00 2024-06-06T03:27:35.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.027+0000 7fbfd4987700 1 -- 172.21.15.40:0/3799193154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd0063af0 con 0x7fbfd0107f00 2024-06-06T03:27:35.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.027+0000 7fbfc77fe700 1 --2- 172.21.15.40:0/3799193154 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7fbfbc039000 0x7fbfbc03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:35.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.027+0000 7fbfc77fe700 1 -- 172.21.15.40:0/3799193154 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbfc000f050 con 0x7fbfd0107f00 2024-06-06T03:27:35.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.027+0000 7fbfcdd9b700 1 --2- 172.21.15.40:0/3799193154 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7fbfbc039000 0x7fbfbc03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:35.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.027+0000 7fbfcdd9b700 1 --2- 172.21.15.40:0/3799193154 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7fbfbc039000 0x7fbfbc03b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8006fd0 tx=0x7fbfb8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:35.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.032+0000 7fbfc77fe700 1 -- 172.21.15.40:0/3799193154 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbfc002a020 con 0x7fbfd0107f00 2024-06-06T03:27:35.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.260+0000 7fbfd4987700 1 -- 172.21.15.40:0/3799193154 --> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbfd007d8a0 con 0x7fbfbc039000 2024-06-06T03:27:35.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.267+0000 7fbfc77fe700 1 -- 172.21.15.40:0/3799193154 <== mgr.14120 v2:172.21.15.40:6800/3930494734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbfd007d8a0 con 0x7fbfbc039000 2024-06-06T03:27:35.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.269+0000 7fbfd4987700 1 -- 172.21.15.40:0/3799193154 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7fbfbc039000 msgr2=0x7fbfbc03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:35.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.269+0000 7fbfd4987700 1 --2- 172.21.15.40:0/3799193154 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7fbfbc039000 0x7fbfbc03b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8006fd0 tx=0x7fbfb8006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:35.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.269+0000 7fbfd4987700 1 -- 172.21.15.40:0/3799193154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfd0107f00 msgr2=0x7fbfd007e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:35.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.269+0000 7fbfd4987700 1 --2- 172.21.15.40:0/3799193154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfd0107f00 0x7fbfd007e3d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00072a0 tx=0x7fbfc000bd50 comp rx=0 tx=0).stop 2024-06-06T03:27:35.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.269+0000 7fbfd4987700 1 -- 172.21.15.40:0/3799193154 shutdown_connections 2024-06-06T03:27:35.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.269+0000 7fbfd4987700 1 --2- 172.21.15.40:0/3799193154 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7fbfbc039000 0x7fbfbc03b4c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:35.613 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.269+0000 7fbfd4987700 1 --2- 172.21.15.40:0/3799193154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfd0107f00 0x7fbfd007e3d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:35.613 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.269+0000 7fbfd4987700 1 -- 172.21.15.40:0/3799193154 >> 172.21.15.40:0/3799193154 conn(0x7fbfd01034f0 msgr2=0x7fbfd0103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:35.613 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.269+0000 7fbfd4987700 1 -- 172.21.15.40:0/3799193154 shutdown_connections 2024-06-06T03:27:35.613 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:34.269+0000 7fbfd4987700 1 -- 172.21.15.40:0/3799193154 wait complete. 2024-06-06T03:27:37.077 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:36 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' 2024-06-06T03:27:37.077 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:36 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1634763450' entity='client.admin' 2024-06-06T03:27:37.084 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.081+0000 7f343dbc3700 1 Processor -- start 2024-06-06T03:27:37.084 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.081+0000 7f343dbc3700 1 -- start start 2024-06-06T03:27:37.084 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.082+0000 7f343dbc3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3438108a30 0x7f343810ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:37.084 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.082+0000 7f343dbc3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f343807be60 con 0x7f3438108a30 2024-06-06T03:27:37.084 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.082+0000 7f34377fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3438108a30 0x7f343810ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.082+0000 7f34377fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3438108a30 0x7f343810ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41632/0 (socket says 172.21.15.40:41632) 2024-06-06T03:27:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.082+0000 7f34377fe700 1 -- 172.21.15.40:0/1661627284 learned_addr learned my addr 172.21.15.40:0/1661627284 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.082+0000 7f34377fe700 1 -- 172.21.15.40:0/1661627284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343810b360 con 0x7f3438108a30 2024-06-06T03:27:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.082+0000 7f34377fe700 1 --2- 172.21.15.40:0/1661627284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3438108a30 0x7f343810ae20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f34280098a0 tx=0x7f3428009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690a904444e233a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.082+0000 7f34367fc700 1 -- 172.21.15.40:0/1661627284 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3428004160 con 0x7f3438108a30 2024-06-06T03:27:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.083+0000 7f34367fc700 1 -- 172.21.15.40:0/1661627284 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3428004300 con 0x7f3438108a30 2024-06-06T03:27:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.083+0000 7f34367fc700 1 -- 172.21.15.40:0/1661627284 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3428011b20 con 0x7f3438108a30 2024-06-06T03:27:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.083+0000 7f343dbc3700 1 -- 172.21.15.40:0/1661627284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3438108a30 msgr2=0x7f343810ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.083+0000 7f343dbc3700 1 --2- 172.21.15.40:0/1661627284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3438108a30 0x7f343810ae20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f34280098a0 tx=0x7f3428009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.083+0000 7f343dbc3700 1 -- 172.21.15.40:0/1661627284 shutdown_connections 2024-06-06T03:27:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.083+0000 7f343dbc3700 1 --2- 172.21.15.40:0/1661627284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3438108a30 0x7f343810ae20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.083+0000 7f343dbc3700 1 -- 172.21.15.40:0/1661627284 >> 172.21.15.40:0/1661627284 conn(0x7f3438102440 msgr2=0x7f3438104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.083+0000 7f343dbc3700 1 -- 172.21.15.40:0/1661627284 shutdown_connections 2024-06-06T03:27:37.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.083+0000 7f343dbc3700 1 -- 172.21.15.40:0/1661627284 wait complete. 2024-06-06T03:27:37.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.084+0000 7f343dbc3700 1 Processor -- start 2024-06-06T03:27:37.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.084+0000 7f343dbc3700 1 -- start start 2024-06-06T03:27:37.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.084+0000 7f343dbc3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3438108a30 0x7f343819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:37.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.084+0000 7f343dbc3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f343810c360 con 0x7f3438108a30 2024-06-06T03:27:37.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.085+0000 7f34377fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3438108a30 0x7f343819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:37.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.085+0000 7f34377fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3438108a30 0x7f343819be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41648/0 (socket says 172.21.15.40:41648) 2024-06-06T03:27:37.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.085+0000 7f34377fe700 1 -- 172.21.15.40:0/1634763450 learned_addr learned my addr 172.21.15.40:0/1634763450 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:37.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.085+0000 7f34377fe700 1 -- 172.21.15.40:0/1634763450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343819c3a0 con 0x7f3438108a30 2024-06-06T03:27:37.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.085+0000 7f34377fe700 1 --2- 172.21.15.40:0/1634763450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3438108a30 0x7f343819be60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f34280116a0 tx=0x7f3428011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:37.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.085+0000 7f3434ff9700 1 -- 172.21.15.40:0/1634763450 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3428003a40 con 0x7f3438108a30 2024-06-06T03:27:37.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.085+0000 7f3434ff9700 1 -- 172.21.15.40:0/1634763450 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3428003be0 con 0x7f3438108a30 2024-06-06T03:27:37.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.085+0000 7f343dbc3700 1 -- 172.21.15.40:0/1634763450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343819c690 con 0x7f3438108a30 2024-06-06T03:27:37.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.085+0000 7f3434ff9700 1 -- 172.21.15.40:0/1634763450 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f342801a720 con 0x7f3438108a30 2024-06-06T03:27:37.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.086+0000 7f343dbc3700 1 -- 172.21.15.40:0/1634763450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343819f320 con 0x7f3438108a30 2024-06-06T03:27:37.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.086+0000 7f343dbc3700 1 -- 172.21.15.40:0/1634763450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3438063af0 con 0x7f3438108a30 2024-06-06T03:27:37.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.087+0000 7f3434ff9700 1 -- 172.21.15.40:0/1634763450 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f342801aa20 con 0x7f3438108a30 2024-06-06T03:27:37.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.087+0000 7f3434ff9700 1 --2- 172.21.15.40:0/1634763450 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f3420039000 0x7f342003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:37.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.087+0000 7f3434ff9700 1 -- 172.21.15.40:0/1634763450 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f342804d6a0 con 0x7f3438108a30 2024-06-06T03:27:37.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.087+0000 7f3436ffd700 1 --2- 172.21.15.40:0/1634763450 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f3420039000 0x7f342003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:37.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.087+0000 7f3436ffd700 1 --2- 172.21.15.40:0/1634763450 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f3420039000 0x7f342003b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f342c006fd0 tx=0x7f342c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:37.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.096+0000 7f3434ff9700 1 -- 172.21.15.40:0/1634763450 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3428011e20 con 0x7f3438108a30 2024-06-06T03:27:37.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.313+0000 7f343dbc3700 1 -- 172.21.15.40:0/1634763450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f343819f950 con 0x7f3438108a30 2024-06-06T03:27:37.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.315+0000 7f3434ff9700 1 -- 172.21.15.40:0/1634763450 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f342801adc0 con 0x7f3438108a30 2024-06-06T03:27:37.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.318+0000 7f343dbc3700 1 -- 172.21.15.40:0/1634763450 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f3420039000 msgr2=0x7f342003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:37.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.318+0000 7f343dbc3700 1 --2- 172.21.15.40:0/1634763450 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f3420039000 0x7f342003b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f342c006fd0 tx=0x7f342c006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:37.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.318+0000 7f343dbc3700 1 -- 172.21.15.40:0/1634763450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3438108a30 msgr2=0x7f343819be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:37.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.318+0000 7f343dbc3700 1 --2- 172.21.15.40:0/1634763450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3438108a30 0x7f343819be60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f34280116a0 tx=0x7f3428011ba0 comp rx=0 tx=0).stop 2024-06-06T03:27:37.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.318+0000 7f343dbc3700 1 -- 172.21.15.40:0/1634763450 shutdown_connections 2024-06-06T03:27:37.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.318+0000 7f343dbc3700 1 --2- 172.21.15.40:0/1634763450 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f3420039000 0x7f342003b4c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:37.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.318+0000 7f343dbc3700 1 --2- 172.21.15.40:0/1634763450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3438108a30 0x7f343819be60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:37.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.318+0000 7f343dbc3700 1 -- 172.21.15.40:0/1634763450 >> 172.21.15.40:0/1634763450 conn(0x7f3438102440 msgr2=0x7f3438102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:37.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.319+0000 7f343dbc3700 1 -- 172.21.15.40:0/1634763450 shutdown_connections 2024-06-06T03:27:37.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:36.319+0000 7f343dbc3700 1 -- 172.21.15.40:0/1634763450 wait complete. 2024-06-06T03:27:38.668 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.529+0000 7f1bf6171700 1 Processor -- start 2024-06-06T03:27:38.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.529+0000 7f1bf6171700 1 -- start start 2024-06-06T03:27:38.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.529+0000 7f1bf6171700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bf0107f80 0x7f1bf01083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:38.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.529+0000 7f1bf6171700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf0108970 con 0x7f1bf0107f80 2024-06-06T03:27:38.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.529+0000 7f1bef7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bf0107f80 0x7f1bf01083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:38.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.530+0000 7f1bef7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bf0107f80 0x7f1bf01083a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41660/0 (socket says 172.21.15.40:41660) 2024-06-06T03:27:38.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.530+0000 7f1bef7fe700 1 -- 172.21.15.40:0/2140580273 learned_addr learned my addr 172.21.15.40:0/2140580273 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:38.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.530+0000 7f1bef7fe700 1 -- 172.21.15.40:0/2140580273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf0109180 con 0x7f1bf0107f80 2024-06-06T03:27:38.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.530+0000 7f1bef7fe700 1 --2- 172.21.15.40:0/2140580273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bf0107f80 0x7f1bf01083a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009e80 tx=0x7f1be000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57363bfc7660ec8b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:38.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.530+0000 7f1bee7fc700 1 -- 172.21.15.40:0/2140580273 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be0017070 con 0x7f1bf0107f80 2024-06-06T03:27:38.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.530+0000 7f1bee7fc700 1 -- 172.21.15.40:0/2140580273 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be000b8c0 con 0x7f1bf0107f80 2024-06-06T03:27:38.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.530+0000 7f1bee7fc700 1 -- 172.21.15.40:0/2140580273 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be00045c0 con 0x7f1bf0107f80 2024-06-06T03:27:38.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.531+0000 7f1bf6171700 1 -- 172.21.15.40:0/2140580273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bf0107f80 msgr2=0x7f1bf01083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:38.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.531+0000 7f1bf6171700 1 --2- 172.21.15.40:0/2140580273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bf0107f80 0x7f1bf01083a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009e80 tx=0x7f1be000b330 comp rx=0 tx=0).stop 2024-06-06T03:27:38.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.531+0000 7f1bf6171700 1 -- 172.21.15.40:0/2140580273 shutdown_connections 2024-06-06T03:27:38.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.531+0000 7f1bf6171700 1 --2- 172.21.15.40:0/2140580273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bf0107f80 0x7f1bf01083a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:38.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.531+0000 7f1bf6171700 1 -- 172.21.15.40:0/2140580273 >> 172.21.15.40:0/2140580273 conn(0x7f1bf0103580 msgr2=0x7f1bf01059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:38.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.531+0000 7f1bf6171700 1 -- 172.21.15.40:0/2140580273 shutdown_connections 2024-06-06T03:27:38.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.531+0000 7f1bf6171700 1 -- 172.21.15.40:0/2140580273 wait complete. 2024-06-06T03:27:38.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.531+0000 7f1bf6171700 1 Processor -- start 2024-06-06T03:27:38.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.532+0000 7f1bf6171700 1 -- start start 2024-06-06T03:27:38.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.532+0000 7f1bf6171700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bf0107f80 0x7f1bf007e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:38.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.532+0000 7f1bf6171700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf010c530 con 0x7f1bf0107f80 2024-06-06T03:27:38.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.532+0000 7f1bef7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bf0107f80 0x7f1bf007e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:38.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.532+0000 7f1bef7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bf0107f80 0x7f1bf007e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41672/0 (socket says 172.21.15.40:41672) 2024-06-06T03:27:38.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.532+0000 7f1bef7fe700 1 -- 172.21.15.40:0/1235361294 learned_addr learned my addr 172.21.15.40:0/1235361294 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:38.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.532+0000 7f1bef7fe700 1 -- 172.21.15.40:0/1235361294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf007e910 con 0x7f1bf0107f80 2024-06-06T03:27:38.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.532+0000 7f1bef7fe700 1 --2- 172.21.15.40:0/1235361294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bf0107f80 0x7f1bf007e3d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1be0003fa0 tx=0x7f1be0004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:38.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.533+0000 7f1becff9700 1 -- 172.21.15.40:0/1235361294 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be0020020 con 0x7f1bf0107f80 2024-06-06T03:27:38.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.533+0000 7f1bf6171700 1 -- 172.21.15.40:0/1235361294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf007cb00 con 0x7f1bf0107f80 2024-06-06T03:27:38.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.533+0000 7f1becff9700 1 -- 172.21.15.40:0/1235361294 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be000bba0 con 0x7f1bf0107f80 2024-06-06T03:27:38.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.533+0000 7f1becff9700 1 -- 172.21.15.40:0/1235361294 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be0017040 con 0x7f1bf0107f80 2024-06-06T03:27:38.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.533+0000 7f1bf6171700 1 -- 172.21.15.40:0/1235361294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf007d040 con 0x7f1bf0107f80 2024-06-06T03:27:38.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.534+0000 7f1becff9700 1 -- 172.21.15.40:0/1235361294 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1be0023020 con 0x7f1bf0107f80 2024-06-06T03:27:38.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.534+0000 7f1bf6171700 1 -- 172.21.15.40:0/1235361294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf010c6b0 con 0x7f1bf0107f80 2024-06-06T03:27:38.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.534+0000 7f1becff9700 1 --2- 172.21.15.40:0/1235361294 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f1bdc039000 0x7f1bdc03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:38.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.534+0000 7f1becff9700 1 -- 172.21.15.40:0/1235361294 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1be000f050 con 0x7f1bf0107f80 2024-06-06T03:27:38.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.534+0000 7f1beeffd700 1 --2- 172.21.15.40:0/1235361294 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f1bdc039000 0x7f1bdc03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:38.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.534+0000 7f1beeffd700 1 --2- 172.21.15.40:0/1235361294 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f1bdc039000 0x7f1bdc03b4c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8006fd0 tx=0x7f1bd8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:38.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.539+0000 7f1becff9700 1 -- 172.21.15.40:0/1235361294 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1be002a020 con 0x7f1bf0107f80 2024-06-06T03:27:38.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.745+0000 7f1bf6171700 1 -- 172.21.15.40:0/1235361294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f1bf0063af0 con 0x7f1bf0107f80 2024-06-06T03:27:38.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.748+0000 7f1becff9700 1 -- 172.21.15.40:0/1235361294 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f1be002d350 con 0x7f1bf0107f80 2024-06-06T03:27:38.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.750+0000 7f1bf6171700 1 -- 172.21.15.40:0/1235361294 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f1bdc039000 msgr2=0x7f1bdc03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:38.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.750+0000 7f1bf6171700 1 --2- 172.21.15.40:0/1235361294 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f1bdc039000 0x7f1bdc03b4c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8006fd0 tx=0x7f1bd8006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:38.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.750+0000 7f1bf6171700 1 -- 172.21.15.40:0/1235361294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bf0107f80 msgr2=0x7f1bf007e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:38.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.750+0000 7f1bf6171700 1 --2- 172.21.15.40:0/1235361294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bf0107f80 0x7f1bf007e3d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1be0003fa0 tx=0x7f1be0004080 comp rx=0 tx=0).stop 2024-06-06T03:27:38.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.750+0000 7f1bf6171700 1 -- 172.21.15.40:0/1235361294 shutdown_connections 2024-06-06T03:27:38.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.750+0000 7f1bf6171700 1 --2- 172.21.15.40:0/1235361294 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f1bdc039000 0x7f1bdc03b4c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:38.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.750+0000 7f1bf6171700 1 --2- 172.21.15.40:0/1235361294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bf0107f80 0x7f1bf007e3d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:38.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.750+0000 7f1bf6171700 1 -- 172.21.15.40:0/1235361294 >> 172.21.15.40:0/1235361294 conn(0x7f1bf0103580 msgr2=0x7f1bf0104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:38.673 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.750+0000 7f1bf6171700 1 -- 172.21.15.40:0/1235361294 shutdown_connections 2024-06-06T03:27:38.673 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:37.750+0000 7f1bf6171700 1 -- 172.21.15.40:0/1235361294 wait complete. 2024-06-06T03:27:38.673 INFO:teuthology.orchestra.run.smithi040.stdout:Enabling the dashboard module... 2024-06-06T03:27:38.996 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:38 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1235361294' entity='client.admin' 2024-06-06T03:27:38.997 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:38 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' 2024-06-06T03:27:40.569 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:40 smithi040 ceph-mon[32220]: from='mgr.14120 172.21.15.40:0/793599031' entity='mgr.x' 2024-06-06T03:27:40.569 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:40 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2584591101' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-06T03:27:40.569 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:40 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: ignoring --setuser ceph since I am not root 2024-06-06T03:27:40.569 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:40 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: ignoring --setgroup ceph since I am not root 2024-06-06T03:27:40.569 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:40 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:40.570+0000 7f4a37b0d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T03:27:40.830 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:40 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:40.763+0000 7f4a37b0d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:27:41.105 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:40 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:40.870+0000 7f4a37b0d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T03:27:41.389 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:41 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:41.197+0000 7f4a37b0d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T03:27:41.389 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:41 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:41.287+0000 7f4a37b0d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T03:27:41.482 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.106+0000 7f5c856f9700 1 Processor -- start 2024-06-06T03:27:41.482 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.106+0000 7f5c856f9700 1 -- start start 2024-06-06T03:27:41.482 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.106+0000 7f5c856f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c80108450 0x7f5c80108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:41.482 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.106+0000 7f5c856f9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5c80108e40 con 0x7f5c80108450 2024-06-06T03:27:41.482 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.107+0000 7f5c7effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c80108450 0x7f5c80108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:41.482 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.107+0000 7f5c7effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c80108450 0x7f5c80108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46542/0 (socket says 172.21.15.40:46542) 2024-06-06T03:27:41.482 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.107+0000 7f5c7effd700 1 -- 172.21.15.40:0/3066467929 learned_addr learned my addr 172.21.15.40:0/3066467929 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:41.483 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.107+0000 7f5c7effd700 1 -- 172.21.15.40:0/3066467929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c801070b0 con 0x7f5c80108450 2024-06-06T03:27:41.483 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.107+0000 7f5c7effd700 1 --2- 172.21.15.40:0/3066467929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c80108450 0x7f5c80108870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5c700098a0 tx=0x7f5c70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece3026723dd2be5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:41.483 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.107+0000 7f5c7dffb700 1 -- 172.21.15.40:0/3066467929 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c70004160 con 0x7f5c80108450 2024-06-06T03:27:41.483 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.107+0000 7f5c7dffb700 1 -- 172.21.15.40:0/3066467929 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c70004300 con 0x7f5c80108450 2024-06-06T03:27:41.483 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.107+0000 7f5c7dffb700 1 -- 172.21.15.40:0/3066467929 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c70011b20 con 0x7f5c80108450 2024-06-06T03:27:41.483 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.108+0000 7f5c856f9700 1 -- 172.21.15.40:0/3066467929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c80108450 msgr2=0x7f5c80108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:41.483 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.108+0000 7f5c856f9700 1 --2- 172.21.15.40:0/3066467929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c80108450 0x7f5c80108870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5c700098a0 tx=0x7f5c70009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:41.483 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.108+0000 7f5c856f9700 1 -- 172.21.15.40:0/3066467929 shutdown_connections 2024-06-06T03:27:41.483 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.108+0000 7f5c856f9700 1 --2- 172.21.15.40:0/3066467929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c80108450 0x7f5c80108870 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:41.483 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.108+0000 7f5c856f9700 1 -- 172.21.15.40:0/3066467929 >> 172.21.15.40:0/3066467929 conn(0x7f5c80101bc0 msgr2=0x7f5c80104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:41.483 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.108+0000 7f5c856f9700 1 -- 172.21.15.40:0/3066467929 shutdown_connections 2024-06-06T03:27:41.483 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.108+0000 7f5c856f9700 1 -- 172.21.15.40:0/3066467929 wait complete. 2024-06-06T03:27:41.483 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.109+0000 7f5c856f9700 1 Processor -- start 2024-06-06T03:27:41.484 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.109+0000 7f5c856f9700 1 -- start start 2024-06-06T03:27:41.484 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.109+0000 7f5c856f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c80108450 0x7f5c8019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:41.484 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.109+0000 7f5c856f9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8010b470 con 0x7f5c80108450 2024-06-06T03:27:41.484 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.109+0000 7f5c7effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c80108450 0x7f5c8019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:41.484 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.109+0000 7f5c7effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c80108450 0x7f5c8019bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46554/0 (socket says 172.21.15.40:46554) 2024-06-06T03:27:41.484 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.109+0000 7f5c7effd700 1 -- 172.21.15.40:0/2584591101 learned_addr learned my addr 172.21.15.40:0/2584591101 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:41.484 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.110+0000 7f5c7effd700 1 -- 172.21.15.40:0/2584591101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8019c3e0 con 0x7f5c80108450 2024-06-06T03:27:41.484 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.110+0000 7f5c7effd700 1 --2- 172.21.15.40:0/2584591101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c80108450 0x7f5c8019bea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5c700116a0 tx=0x7f5c70011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:41.484 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.110+0000 7f5c67fff700 1 -- 172.21.15.40:0/2584591101 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c70003a40 con 0x7f5c80108450 2024-06-06T03:27:41.484 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.110+0000 7f5c67fff700 1 -- 172.21.15.40:0/2584591101 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c70003be0 con 0x7f5c80108450 2024-06-06T03:27:41.484 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.110+0000 7f5c856f9700 1 -- 172.21.15.40:0/2584591101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c8019c670 con 0x7f5c80108450 2024-06-06T03:27:41.484 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.110+0000 7f5c67fff700 1 -- 172.21.15.40:0/2584591101 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c7001a750 con 0x7f5c80108450 2024-06-06T03:27:41.484 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.110+0000 7f5c856f9700 1 -- 172.21.15.40:0/2584591101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c8019cad0 con 0x7f5c80108450 2024-06-06T03:27:41.485 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.111+0000 7f5c67fff700 1 -- 172.21.15.40:0/2584591101 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5c7001aa50 con 0x7f5c80108450 2024-06-06T03:27:41.485 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.111+0000 7f5c856f9700 1 -- 172.21.15.40:0/2584591101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c80063af0 con 0x7f5c80108450 2024-06-06T03:27:41.485 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.111+0000 7f5c67fff700 1 --2- 172.21.15.40:0/2584591101 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5c68039000 0x7f5c6803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:41.485 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.111+0000 7f5c67fff700 1 -- 172.21.15.40:0/2584591101 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5c7004d6e0 con 0x7f5c80108450 2024-06-06T03:27:41.485 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.111+0000 7f5c7e7fc700 1 --2- 172.21.15.40:0/2584591101 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5c68039000 0x7f5c6803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:41.485 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.112+0000 7f5c7e7fc700 1 --2- 172.21.15.40:0/2584591101 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5c68039000 0x7f5c6803b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5c74006fd0 tx=0x7f5c74006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:41.485 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.116+0000 7f5c67fff700 1 -- 172.21.15.40:0/2584591101 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5c70011e20 con 0x7f5c80108450 2024-06-06T03:27:41.485 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:39.390+0000 7f5c856f9700 1 -- 172.21.15.40:0/2584591101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f5c8002e010 con 0x7f5c80108450 2024-06-06T03:27:41.485 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:40.385+0000 7f5c67fff700 1 -- 172.21.15.40:0/2584591101 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f5c7001adf0 con 0x7f5c80108450 2024-06-06T03:27:41.485 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:40.386+0000 7f5c67fff700 1 -- 172.21.15.40:0/2584591101 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f5c7002d910 con 0x7f5c80108450 2024-06-06T03:27:41.485 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:40.388+0000 7f5c856f9700 1 -- 172.21.15.40:0/2584591101 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5c68039000 msgr2=0x7f5c6803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:41.485 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:40.388+0000 7f5c856f9700 1 --2- 172.21.15.40:0/2584591101 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5c68039000 0x7f5c6803b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5c74006fd0 tx=0x7f5c74006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:41.486 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:40.388+0000 7f5c856f9700 1 -- 172.21.15.40:0/2584591101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c80108450 msgr2=0x7f5c8019bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:41.486 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:40.388+0000 7f5c856f9700 1 --2- 172.21.15.40:0/2584591101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c80108450 0x7f5c8019bea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5c700116a0 tx=0x7f5c70011ba0 comp rx=0 tx=0).stop 2024-06-06T03:27:41.486 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:40.388+0000 7f5c856f9700 1 -- 172.21.15.40:0/2584591101 shutdown_connections 2024-06-06T03:27:41.486 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:40.388+0000 7f5c856f9700 1 --2- 172.21.15.40:0/2584591101 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5c68039000 0x7f5c6803b4c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:41.486 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:40.388+0000 7f5c856f9700 1 --2- 172.21.15.40:0/2584591101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c80108450 0x7f5c8019bea0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:41.486 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:40.388+0000 7f5c856f9700 1 -- 172.21.15.40:0/2584591101 >> 172.21.15.40:0/2584591101 conn(0x7f5c80101bc0 msgr2=0x7f5c80102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:41.486 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:40.388+0000 7f5c856f9700 1 -- 172.21.15.40:0/2584591101 shutdown_connections 2024-06-06T03:27:41.486 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:40.388+0000 7f5c856f9700 1 -- 172.21.15.40:0/2584591101 wait complete. 2024-06-06T03:27:41.720 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:41 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2584591101' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-06T03:27:41.720 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:41 smithi040 ceph-mon[32220]: mgrmap e9: x(active, since 20s) 2024-06-06T03:27:42.739 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:42 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2178005702' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-06T03:27:42.739 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:42 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:42.563+0000 7f4a37b0d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T03:27:42.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-06-06T03:27:42.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-06T03:27:42.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T03:27:42.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-06T03:27:42.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-06T03:27:42.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-06-06T03:27:42.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.923+0000 7f5fed6ec700 1 Processor -- start 2024-06-06T03:27:42.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.923+0000 7f5fed6ec700 1 -- start start 2024-06-06T03:27:42.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.924+0000 7f5fed6ec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe8106620 0x7f5fe8106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:42.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.924+0000 7f5fed6ec700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe8107010 con 0x7f5fe8106620 2024-06-06T03:27:42.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.924+0000 7f5fe6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe8106620 0x7f5fe8106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:42.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.924+0000 7f5fe6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe8106620 0x7f5fe8106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46574/0 (socket says 172.21.15.40:46574) 2024-06-06T03:27:42.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.924+0000 7f5fe6ffd700 1 -- 172.21.15.40:0/3384153068 learned_addr learned my addr 172.21.15.40:0/3384153068 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:42.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.924+0000 7f5fe6ffd700 1 -- 172.21.15.40:0/3384153068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe8107820 con 0x7f5fe8106620 2024-06-06T03:27:42.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.925+0000 7f5fe6ffd700 1 --2- 172.21.15.40:0/3384153068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe8106620 0x7f5fe8106a40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80098a0 tx=0x7f5fd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f036910f5c0d5d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:42.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.925+0000 7f5fe5ffb700 1 -- 172.21.15.40:0/3384153068 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fd8004160 con 0x7f5fe8106620 2024-06-06T03:27:42.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.925+0000 7f5fe5ffb700 1 -- 172.21.15.40:0/3384153068 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd8004300 con 0x7f5fe8106620 2024-06-06T03:27:42.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.925+0000 7f5fe5ffb700 1 -- 172.21.15.40:0/3384153068 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fd8011b20 con 0x7f5fe8106620 2024-06-06T03:27:42.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.925+0000 7f5fed6ec700 1 -- 172.21.15.40:0/3384153068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe8106620 msgr2=0x7f5fe8106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:42.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.925+0000 7f5fed6ec700 1 --2- 172.21.15.40:0/3384153068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe8106620 0x7f5fe8106a40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80098a0 tx=0x7f5fd8009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:42.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.926+0000 7f5fed6ec700 1 -- 172.21.15.40:0/3384153068 shutdown_connections 2024-06-06T03:27:42.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.926+0000 7f5fed6ec700 1 --2- 172.21.15.40:0/3384153068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe8106620 0x7f5fe8106a40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:42.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.926+0000 7f5fed6ec700 1 -- 172.21.15.40:0/3384153068 >> 172.21.15.40:0/3384153068 conn(0x7f5fe8101bc0 msgr2=0x7f5fe8104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:42.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.926+0000 7f5fed6ec700 1 -- 172.21.15.40:0/3384153068 shutdown_connections 2024-06-06T03:27:42.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.926+0000 7f5fed6ec700 1 -- 172.21.15.40:0/3384153068 wait complete. 2024-06-06T03:27:42.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.926+0000 7f5fed6ec700 1 Processor -- start 2024-06-06T03:27:42.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.926+0000 7f5fed6ec700 1 -- start start 2024-06-06T03:27:42.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.927+0000 7f5fed6ec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe8106620 0x7f5fe819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:42.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.927+0000 7f5fed6ec700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe810abd0 con 0x7f5fe8106620 2024-06-06T03:27:42.757 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.927+0000 7f5fe6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe8106620 0x7f5fe819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:42.757 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.927+0000 7f5fe6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe8106620 0x7f5fe819ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46576/0 (socket says 172.21.15.40:46576) 2024-06-06T03:27:42.757 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.927+0000 7f5fe6ffd700 1 -- 172.21.15.40:0/2178005702 learned_addr learned my addr 172.21.15.40:0/2178005702 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:42.757 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.927+0000 7f5fe6ffd700 1 -- 172.21.15.40:0/2178005702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe819b360 con 0x7f5fe8106620 2024-06-06T03:27:42.757 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.927+0000 7f5fe6ffd700 1 --2- 172.21.15.40:0/2178005702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe8106620 0x7f5fe819ae20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8000c00 tx=0x7f5fd8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:42.757 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.927+0000 7f5fdffff700 1 -- 172.21.15.40:0/2178005702 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fd8003b60 con 0x7f5fe8106620 2024-06-06T03:27:42.757 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.927+0000 7f5fdffff700 1 -- 172.21.15.40:0/2178005702 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd801a910 con 0x7f5fe8106620 2024-06-06T03:27:42.757 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.927+0000 7f5fed6ec700 1 -- 172.21.15.40:0/2178005702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe819b5f0 con 0x7f5fe8106620 2024-06-06T03:27:42.757 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.928+0000 7f5fdffff700 1 -- 172.21.15.40:0/2178005702 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fd8024930 con 0x7f5fe8106620 2024-06-06T03:27:42.757 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.928+0000 7f5fed6ec700 1 -- 172.21.15.40:0/2178005702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe819ba50 con 0x7f5fe8106620 2024-06-06T03:27:42.757 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.928+0000 7f5fdffff700 1 -- 172.21.15.40:0/2178005702 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f5fd8022020 con 0x7f5fe8106620 2024-06-06T03:27:42.757 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.928+0000 7f5fed6ec700 1 -- 172.21.15.40:0/2178005702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe8063af0 con 0x7f5fe8106620 2024-06-06T03:27:42.757 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.929+0000 7f5fdffff700 1 --2- 172.21.15.40:0/2178005702 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5fd403b2b0 0x7f5fd403d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:42.758 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.929+0000 7f5fdffff700 1 -- 172.21.15.40:0/2178005702 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5fd804d2a0 con 0x7f5fe8106620 2024-06-06T03:27:42.758 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.929+0000 7f5fe67fc700 1 -- 172.21.15.40:0/2178005702 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5fd403b2b0 msgr2=0x7f5fd403d770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/3930494734 2024-06-06T03:27:42.758 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.929+0000 7f5fe67fc700 1 --2- 172.21.15.40:0/2178005702 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5fd403b2b0 0x7f5fd403d770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T03:27:42.758 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:41.934+0000 7f5fdffff700 1 -- 172.21.15.40:0/2178005702 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5fd800b5a0 con 0x7f5fe8106620 2024-06-06T03:27:42.758 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:42.130+0000 7f5fe67fc700 1 -- 172.21.15.40:0/2178005702 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5fd403b2b0 msgr2=0x7f5fd403d770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/3930494734 2024-06-06T03:27:42.758 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:42.130+0000 7f5fe67fc700 1 --2- 172.21.15.40:0/2178005702 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5fd403b2b0 0x7f5fd403d770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T03:27:42.758 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:42.198+0000 7f5fed6ec700 1 -- 172.21.15.40:0/2178005702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f5fe8102df0 con 0x7f5fe8106620 2024-06-06T03:27:42.758 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:42.199+0000 7f5fdffff700 1 -- 172.21.15.40:0/2178005702 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f5fd8021d80 con 0x7f5fe8106620 2024-06-06T03:27:42.758 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:42.201+0000 7f5fed6ec700 1 -- 172.21.15.40:0/2178005702 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5fd403b2b0 msgr2=0x7f5fd403d770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:27:42.758 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:42.201+0000 7f5fed6ec700 1 --2- 172.21.15.40:0/2178005702 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5fd403b2b0 0x7f5fd403d770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:42.758 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:42.201+0000 7f5fed6ec700 1 -- 172.21.15.40:0/2178005702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe8106620 msgr2=0x7f5fe819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:42.758 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:42.201+0000 7f5fed6ec700 1 --2- 172.21.15.40:0/2178005702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe8106620 0x7f5fe819ae20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8000c00 tx=0x7f5fd8011c10 comp rx=0 tx=0).stop 2024-06-06T03:27:42.759 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:42.201+0000 7f5fed6ec700 1 -- 172.21.15.40:0/2178005702 shutdown_connections 2024-06-06T03:27:42.759 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:42.201+0000 7f5fed6ec700 1 --2- 172.21.15.40:0/2178005702 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f5fd403b2b0 0x7f5fd403d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:42.759 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:42.201+0000 7f5fed6ec700 1 --2- 172.21.15.40:0/2178005702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe8106620 0x7f5fe819ae20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:42.759 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:42.201+0000 7f5fed6ec700 1 -- 172.21.15.40:0/2178005702 >> 172.21.15.40:0/2178005702 conn(0x7f5fe8101bc0 msgr2=0x7f5fe8103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:42.759 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:42.201+0000 7f5fed6ec700 1 -- 172.21.15.40:0/2178005702 shutdown_connections 2024-06-06T03:27:42.759 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:42.201+0000 7f5fed6ec700 1 -- 172.21.15.40:0/2178005702 wait complete. 2024-06-06T03:27:42.759 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for the mgr to restart... 2024-06-06T03:27:42.759 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mgr epoch 9... 2024-06-06T03:27:43.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:42 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:42.740+0000 7f4a37b0d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T03:27:43.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:42 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:42.921+0000 7f4a37b0d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T03:27:43.407 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:43 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:43.233+0000 7f4a37b0d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T03:27:43.407 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:43 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:43.329+0000 7f4a37b0d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T03:27:43.742 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:43 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:43.541+0000 7f4a37b0d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T03:27:44.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:43 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:43.743+0000 7f4a37b0d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:27:44.812 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:44 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:44.372+0000 7f4a37b0d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T03:27:45.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:44 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:44.813+0000 7f4a37b0d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T03:27:45.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:44 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:44.914+0000 7f4a37b0d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T03:27:45.580 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:45 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:45.124+0000 7f4a37b0d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T03:27:46.046 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:45 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:45.692+0000 7f4a37b0d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T03:27:46.046 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:45 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:45.789+0000 7f4a37b0d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T03:27:46.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:46 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:46.047+0000 7f4a37b0d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T03:27:46.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:46 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:46.187+0000 7f4a37b0d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T03:27:46.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:46 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:46.287+0000 7f4a37b0d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T03:27:46.718 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:46 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:46.409+0000 7f4a37b0d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T03:27:46.719 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:46 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:46.719+0000 7f4a37b0d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T03:27:47.631 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:47 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:47.349+0000 7f4a37b0d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T03:27:47.631 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:27:47 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:27:47.577+0000 7f4a37b0d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T03:27:48.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:47 smithi040 ceph-mon[32220]: Active manager daemon x restarted 2024-06-06T03:27:48.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:47 smithi040 ceph-mon[32220]: Activating manager daemon x 2024-06-06T03:27:48.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:47 smithi040 ceph-mon[32220]: osdmap e3: 0 total, 0 up, 0 in 2024-06-06T03:27:48.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:47 smithi040 ceph-mon[32220]: mgrmap e10: x(active, starting, since 0.0033073s) 2024-06-06T03:27:48.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:27:48.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T03:27:48.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T03:27:48.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T03:27:48.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T03:27:48.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:47 smithi040 ceph-mon[32220]: Manager daemon x is now available 2024-06-06T03:27:48.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:27:48.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T03:27:48.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T03:27:48.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:48 smithi040 ceph-mon[32220]: mgrmap e11: x(active, since 1.00638s) 2024-06-06T03:27:49.157 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-06-06T03:27:49.158 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-06T03:27:49.158 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-06T03:27:49.158 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-06-06T03:27:49.158 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.202+0000 7f3538f03700 1 Processor -- start 2024-06-06T03:27:49.158 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.202+0000 7f3538f03700 1 -- start start 2024-06-06T03:27:49.158 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.202+0000 7f3538f03700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35341089e0 0x7f353410add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:49.158 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.202+0000 7f3538f03700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f353407be60 con 0x7f35341089e0 2024-06-06T03:27:49.158 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.202+0000 7f353259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35341089e0 0x7f353410add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:49.159 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.202+0000 7f353259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35341089e0 0x7f353410add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46590/0 (socket says 172.21.15.40:46590) 2024-06-06T03:27:49.159 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.202+0000 7f353259c700 1 -- 172.21.15.40:0/2752009137 learned_addr learned my addr 172.21.15.40:0/2752009137 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:49.159 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.202+0000 7f353259c700 1 -- 172.21.15.40:0/2752009137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353410b310 con 0x7f35341089e0 2024-06-06T03:27:49.159 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.203+0000 7f353259c700 1 --2- 172.21.15.40:0/2752009137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35341089e0 0x7f353410add0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f35240098a0 tx=0x7f3524009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b2df0b79d91abf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:49.159 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.203+0000 7f353159a700 1 -- 172.21.15.40:0/2752009137 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3524004160 con 0x7f35341089e0 2024-06-06T03:27:49.159 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.203+0000 7f353159a700 1 -- 172.21.15.40:0/2752009137 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3524004300 con 0x7f35341089e0 2024-06-06T03:27:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.203+0000 7f353159a700 1 -- 172.21.15.40:0/2752009137 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3524011b20 con 0x7f35341089e0 2024-06-06T03:27:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.204+0000 7f3538f03700 1 -- 172.21.15.40:0/2752009137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35341089e0 msgr2=0x7f353410add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.204+0000 7f3538f03700 1 --2- 172.21.15.40:0/2752009137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35341089e0 0x7f353410add0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f35240098a0 tx=0x7f3524009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.204+0000 7f3538f03700 1 -- 172.21.15.40:0/2752009137 shutdown_connections 2024-06-06T03:27:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.204+0000 7f3538f03700 1 --2- 172.21.15.40:0/2752009137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35341089e0 0x7f353410add0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.204+0000 7f3538f03700 1 -- 172.21.15.40:0/2752009137 >> 172.21.15.40:0/2752009137 conn(0x7f35341023f0 msgr2=0x7f3534104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.204+0000 7f3538f03700 1 -- 172.21.15.40:0/2752009137 shutdown_connections 2024-06-06T03:27:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.204+0000 7f3538f03700 1 -- 172.21.15.40:0/2752009137 wait complete. 2024-06-06T03:27:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.205+0000 7f3538f03700 1 Processor -- start 2024-06-06T03:27:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.205+0000 7f3538f03700 1 -- start start 2024-06-06T03:27:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.205+0000 7f3538f03700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35341089e0 0x7f353419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.205+0000 7f3538f03700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f353410c310 con 0x7f35341089e0 2024-06-06T03:27:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.205+0000 7f353259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35341089e0 0x7f353419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.205+0000 7f353259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35341089e0 0x7f353419be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46602/0 (socket says 172.21.15.40:46602) 2024-06-06T03:27:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.205+0000 7f353259c700 1 -- 172.21.15.40:0/671046774 learned_addr learned my addr 172.21.15.40:0/671046774 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.206+0000 7f353259c700 1 -- 172.21.15.40:0/671046774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353419c350 con 0x7f35341089e0 2024-06-06T03:27:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.206+0000 7f353259c700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35341089e0 0x7f353419be10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f35240116a0 tx=0x7f3524011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.206+0000 7f35237fe700 1 -- 172.21.15.40:0/671046774 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3524003a40 con 0x7f35341089e0 2024-06-06T03:27:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.206+0000 7f3538f03700 1 -- 172.21.15.40:0/671046774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353419c640 con 0x7f35341089e0 2024-06-06T03:27:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.206+0000 7f35237fe700 1 -- 172.21.15.40:0/671046774 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3524003be0 con 0x7f35341089e0 2024-06-06T03:27:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.206+0000 7f35237fe700 1 -- 172.21.15.40:0/671046774 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f352401a720 con 0x7f35341089e0 2024-06-06T03:27:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.206+0000 7f3538f03700 1 -- 172.21.15.40:0/671046774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f353419f2d0 con 0x7f35341089e0 2024-06-06T03:27:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.207+0000 7f35237fe700 1 -- 172.21.15.40:0/671046774 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f352401a9d0 con 0x7f35341089e0 2024-06-06T03:27:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.207+0000 7f35237fe700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f351c0390b0 0x7f351c03b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.207+0000 7f3538f03700 1 -- 172.21.15.40:0/671046774 --> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3534063af0 con 0x7f351c0390b0 2024-06-06T03:27:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.207+0000 7f35237fe700 1 -- 172.21.15.40:0/671046774 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f352404d610 con 0x7f35341089e0 2024-06-06T03:27:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.207+0000 7f3531d9b700 1 -- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f351c0390b0 msgr2=0x7f351c03b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/3930494734 2024-06-06T03:27:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.207+0000 7f3531d9b700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f351c0390b0 0x7f351c03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T03:27:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.408+0000 7f3531d9b700 1 -- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f351c0390b0 msgr2=0x7f351c03b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/3930494734 2024-06-06T03:27:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.408+0000 7f3531d9b700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f351c0390b0 0x7f351c03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T03:27:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.809+0000 7f3531d9b700 1 -- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f351c0390b0 msgr2=0x7f351c03b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/3930494734 2024-06-06T03:27:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:43.809+0000 7f3531d9b700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f351c0390b0 0x7f351c03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-06T03:27:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:44.610+0000 7f3531d9b700 1 -- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f351c0390b0 msgr2=0x7f351c03b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/3930494734 2024-06-06T03:27:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:44.610+0000 7f3531d9b700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f351c0390b0 0x7f351c03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-06T03:27:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:46.211+0000 7f3531d9b700 1 -- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f351c0390b0 msgr2=0x7f351c03b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/3930494734 2024-06-06T03:27:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:46.211+0000 7f3531d9b700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f351c0390b0 0x7f351c03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-06T03:27:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:47.580+0000 7f35237fe700 1 -- 172.21.15.40:0/671046774 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f352401ad10 con 0x7f35341089e0 2024-06-06T03:27:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:47.580+0000 7f35237fe700 1 -- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f351c0390b0 msgr2=0x7f351c03b570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:27:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:47.580+0000 7f35237fe700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f351c0390b0 0x7f351c03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.582+0000 7f35237fe700 1 -- 172.21.15.40:0/671046774 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f352404e8b0 con 0x7f35341089e0 2024-06-06T03:27:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.582+0000 7f35237fe700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f351c03c980 0x7f351c03ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.583+0000 7f35237fe700 1 -- 172.21.15.40:0/671046774 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3524008ea0 con 0x7f351c03c980 2024-06-06T03:27:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.583+0000 7f3531d9b700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f351c03c980 0x7f351c03ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.584+0000 7f3531d9b700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f351c03c980 0x7f351c03ed70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3528003a10 tx=0x7f3528009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.584+0000 7f35237fe700 1 -- 172.21.15.40:0/671046774 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f3524008ea0 con 0x7f351c03c980 2024-06-06T03:27:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.598+0000 7f3538f03700 1 -- 172.21.15.40:0/671046774 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3534103640 con 0x7f351c03c980 2024-06-06T03:27:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.598+0000 7f35237fe700 1 -- 172.21.15.40:0/671046774 <== mgr.14152 v2:172.21.15.40:6800/869712339 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f3534103640 con 0x7f351c03c980 2024-06-06T03:27:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.598+0000 7f3538f03700 1 -- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f351c03c980 msgr2=0x7f351c03ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.598+0000 7f3538f03700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f351c03c980 0x7f351c03ed70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3528003a10 tx=0x7f3528009270 comp rx=0 tx=0).stop 2024-06-06T03:27:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.598+0000 7f3538f03700 1 -- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35341089e0 msgr2=0x7f353419be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.598+0000 7f3538f03700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35341089e0 0x7f353419be10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f35240116a0 tx=0x7f3524011ba0 comp rx=0 tx=0).stop 2024-06-06T03:27:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.598+0000 7f3538f03700 1 -- 172.21.15.40:0/671046774 shutdown_connections 2024-06-06T03:27:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.598+0000 7f3538f03700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f351c03c980 0x7f351c03ed70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.598+0000 7f3538f03700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35341089e0 0x7f353419be10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.598+0000 7f3538f03700 1 --2- 172.21.15.40:0/671046774 >> [v2:172.21.15.40:6800/3930494734,v1:172.21.15.40:6801/3930494734] conn(0x7f351c0390b0 0x7f351c03b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.598+0000 7f3538f03700 1 -- 172.21.15.40:0/671046774 >> 172.21.15.40:0/671046774 conn(0x7f35341023f0 msgr2=0x7f3534102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.598+0000 7f3538f03700 1 -- 172.21.15.40:0/671046774 shutdown_connections 2024-06-06T03:27:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:48.598+0000 7f3538f03700 1 -- 172.21.15.40:0/671046774 wait complete. 2024-06-06T03:27:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:mgr epoch 9 is available 2024-06-06T03:27:49.165 INFO:teuthology.orchestra.run.smithi040.stdout:Generating a dashboard self-signed certificate... 2024-06-06T03:27:49.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:49 smithi040 ceph-mon[32220]: [06/Jun/2024:03:27:48] ENGINE Bus STARTING 2024-06-06T03:27:49.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:49 smithi040 ceph-mon[32220]: [06/Jun/2024:03:27:48] ENGINE Serving on https://172.21.15.40:7150 2024-06-06T03:27:49.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:49 smithi040 ceph-mon[32220]: [06/Jun/2024:03:27:49] ENGINE Serving on http://172.21.15.40:8765 2024-06-06T03:27:49.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:49 smithi040 ceph-mon[32220]: [06/Jun/2024:03:27:49] ENGINE Bus STARTED 2024-06-06T03:27:50.554 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-06T03:27:50.554 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.566+0000 7f6aec320700 1 Processor -- start 2024-06-06T03:27:50.555 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.566+0000 7f6aec320700 1 -- start start 2024-06-06T03:27:50.555 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.566+0000 7f6aec320700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae4107f80 0x7f6ae41083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:50.555 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.566+0000 7f6aec320700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4108970 con 0x7f6ae4107f80 2024-06-06T03:27:50.555 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.567+0000 7f6aea0bc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae4107f80 0x7f6ae41083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:50.555 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.567+0000 7f6aea0bc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae4107f80 0x7f6ae41083a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55322/0 (socket says 172.21.15.40:55322) 2024-06-06T03:27:50.555 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.567+0000 7f6aea0bc700 1 -- 172.21.15.40:0/982467701 learned_addr learned my addr 172.21.15.40:0/982467701 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:50.555 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.567+0000 7f6aea0bc700 1 -- 172.21.15.40:0/982467701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae4109180 con 0x7f6ae4107f80 2024-06-06T03:27:50.555 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.567+0000 7f6aea0bc700 1 --2- 172.21.15.40:0/982467701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae4107f80 0x7f6ae41083a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009e80 tx=0x7f6ad400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01b97490667de87 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:50.555 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.567+0000 7f6ae90ba700 1 -- 172.21.15.40:0/982467701 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ad4017070 con 0x7f6ae4107f80 2024-06-06T03:27:50.555 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.567+0000 7f6ae90ba700 1 -- 172.21.15.40:0/982467701 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad400b8c0 con 0x7f6ae4107f80 2024-06-06T03:27:50.555 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.567+0000 7f6ae90ba700 1 -- 172.21.15.40:0/982467701 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ad40045c0 con 0x7f6ae4107f80 2024-06-06T03:27:50.555 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.568+0000 7f6aec320700 1 -- 172.21.15.40:0/982467701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae4107f80 msgr2=0x7f6ae41083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:50.555 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.568+0000 7f6aec320700 1 --2- 172.21.15.40:0/982467701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae4107f80 0x7f6ae41083a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009e80 tx=0x7f6ad400b330 comp rx=0 tx=0).stop 2024-06-06T03:27:50.556 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.568+0000 7f6aec320700 1 -- 172.21.15.40:0/982467701 shutdown_connections 2024-06-06T03:27:50.556 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.568+0000 7f6aec320700 1 --2- 172.21.15.40:0/982467701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae4107f80 0x7f6ae41083a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:50.556 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.568+0000 7f6aec320700 1 -- 172.21.15.40:0/982467701 >> 172.21.15.40:0/982467701 conn(0x7f6ae4103580 msgr2=0x7f6ae41059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:50.556 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.568+0000 7f6aec320700 1 -- 172.21.15.40:0/982467701 shutdown_connections 2024-06-06T03:27:50.556 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.568+0000 7f6aec320700 1 -- 172.21.15.40:0/982467701 wait complete. 2024-06-06T03:27:50.556 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.569+0000 7f6aec320700 1 Processor -- start 2024-06-06T03:27:50.556 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.569+0000 7f6aec320700 1 -- start start 2024-06-06T03:27:50.556 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.569+0000 7f6aec320700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae4107f80 0x7f6ae419bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:50.556 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.569+0000 7f6aec320700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae410c530 con 0x7f6ae4107f80 2024-06-06T03:27:50.556 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.569+0000 7f6aea0bc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae4107f80 0x7f6ae419bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:50.556 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.569+0000 7f6aea0bc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae4107f80 0x7f6ae419bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55326/0 (socket says 172.21.15.40:55326) 2024-06-06T03:27:50.556 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.569+0000 7f6aea0bc700 1 -- 172.21.15.40:0/915340723 learned_addr learned my addr 172.21.15.40:0/915340723 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:50.557 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.569+0000 7f6aea0bc700 1 -- 172.21.15.40:0/915340723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae419c400 con 0x7f6ae4107f80 2024-06-06T03:27:50.557 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.570+0000 7f6aea0bc700 1 --2- 172.21.15.40:0/915340723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae4107f80 0x7f6ae419bec0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4003fa0 tx=0x7f6ad4004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:50.557 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.570+0000 7f6ae37fe700 1 -- 172.21.15.40:0/915340723 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ad4020020 con 0x7f6ae4107f80 2024-06-06T03:27:50.557 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.570+0000 7f6ae37fe700 1 -- 172.21.15.40:0/915340723 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad400bba0 con 0x7f6ae4107f80 2024-06-06T03:27:50.557 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.570+0000 7f6aec320700 1 -- 172.21.15.40:0/915340723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae419c690 con 0x7f6ae4107f80 2024-06-06T03:27:50.557 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.570+0000 7f6ae37fe700 1 -- 172.21.15.40:0/915340723 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ad4017040 con 0x7f6ae4107f80 2024-06-06T03:27:50.557 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.570+0000 7f6aec320700 1 -- 172.21.15.40:0/915340723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae419f380 con 0x7f6ae4107f80 2024-06-06T03:27:50.557 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.571+0000 7f6ae37fe700 1 -- 172.21.15.40:0/915340723 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f6ad4023020 con 0x7f6ae4107f80 2024-06-06T03:27:50.557 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.571+0000 7f6aec320700 1 -- 172.21.15.40:0/915340723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae410c6b0 con 0x7f6ae4107f80 2024-06-06T03:27:50.557 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.571+0000 7f6ae37fe700 1 --2- 172.21.15.40:0/915340723 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6acc038f90 0x7f6acc03b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:50.557 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.571+0000 7f6ae37fe700 1 -- 172.21.15.40:0/915340723 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6ad400f050 con 0x7f6ae4107f80 2024-06-06T03:27:50.557 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.571+0000 7f6ae98bb700 1 --2- 172.21.15.40:0/915340723 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6acc038f90 0x7f6acc03b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:50.557 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.571+0000 7f6ae98bb700 1 --2- 172.21.15.40:0/915340723 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6acc038f90 0x7f6acc03b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8006fd0 tx=0x7f6ad8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:50.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.576+0000 7f6ae37fe700 1 -- 172.21.15.40:0/915340723 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6ad402a020 con 0x7f6ae4107f80 2024-06-06T03:27:50.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.637+0000 7f6ae37fe700 1 -- 172.21.15.40:0/915340723 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f6ad4012950 con 0x7f6ae4107f80 2024-06-06T03:27:50.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.790+0000 7f6aec320700 1 -- 172.21.15.40:0/915340723 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f6ae41047a0 con 0x7f6acc038f90 2024-06-06T03:27:50.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.969+0000 7f6ae37fe700 1 -- 172.21.15.40:0/915340723 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f6ae41047a0 con 0x7f6acc038f90 2024-06-06T03:27:50.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.972+0000 7f6aec320700 1 -- 172.21.15.40:0/915340723 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6acc038f90 msgr2=0x7f6acc03b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:50.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.972+0000 7f6aec320700 1 --2- 172.21.15.40:0/915340723 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6acc038f90 0x7f6acc03b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8006fd0 tx=0x7f6ad8006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:50.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.972+0000 7f6aec320700 1 -- 172.21.15.40:0/915340723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae4107f80 msgr2=0x7f6ae419bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:50.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.972+0000 7f6aec320700 1 --2- 172.21.15.40:0/915340723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae4107f80 0x7f6ae419bec0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4003fa0 tx=0x7f6ad4004080 comp rx=0 tx=0).stop 2024-06-06T03:27:50.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.973+0000 7f6aec320700 1 -- 172.21.15.40:0/915340723 shutdown_connections 2024-06-06T03:27:50.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.973+0000 7f6aec320700 1 --2- 172.21.15.40:0/915340723 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6acc038f90 0x7f6acc03b450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:50.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.973+0000 7f6aec320700 1 --2- 172.21.15.40:0/915340723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae4107f80 0x7f6ae419bec0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:50.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.973+0000 7f6aec320700 1 -- 172.21.15.40:0/915340723 >> 172.21.15.40:0/915340723 conn(0x7f6ae4103580 msgr2=0x7f6ae4104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:50.559 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.973+0000 7f6aec320700 1 -- 172.21.15.40:0/915340723 shutdown_connections 2024-06-06T03:27:50.559 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:49.973+0000 7f6aec320700 1 -- 172.21.15.40:0/915340723 wait complete. 2024-06-06T03:27:50.559 INFO:teuthology.orchestra.run.smithi040.stdout:Creating initial admin user... 2024-06-06T03:27:50.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:50 smithi040 ceph-mon[32220]: mgrmap e12: x(active, since 2s) 2024-06-06T03:27:50.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:50 smithi040 ceph-mon[32220]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:27:50.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:27:50.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:27:50.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:27:51.736 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:51 smithi040 ceph-mon[32220]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:27:51.736 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:51 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:27:51.736 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:51 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:27:52.044 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$qEyBjkKto62DN16qJDx9rOlq1gv3oJJYlujYBqMnNd8j/FjAjxgJC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717644471, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-06T03:27:52.045 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.978+0000 7fa51f9d3700 1 Processor -- start 2024-06-06T03:27:52.045 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.978+0000 7fa51f9d3700 1 -- start start 2024-06-06T03:27:52.045 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.978+0000 7fa51f9d3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa518106840 0x7fa518106c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:52.045 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.978+0000 7fa51f9d3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa51807be40 con 0x7fa518106840 2024-06-06T03:27:52.045 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.978+0000 7fa51d76f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa518106840 0x7fa518106c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:52.045 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.978+0000 7fa51d76f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa518106840 0x7fa518106c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55332/0 (socket says 172.21.15.40:55332) 2024-06-06T03:27:52.045 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.978+0000 7fa51d76f700 1 -- 172.21.15.40:0/1828187156 learned_addr learned my addr 172.21.15.40:0/1828187156 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:52.045 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.979+0000 7fa51d76f700 1 -- 172.21.15.40:0/1828187156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5181071a0 con 0x7fa518106840 2024-06-06T03:27:52.045 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.979+0000 7fa51d76f700 1 --2- 172.21.15.40:0/1828187156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa518106840 0x7fa518106c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5140098a0 tx=0x7fa514009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1ce4f61586dd34 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:52.045 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.979+0000 7fa50ffff700 1 -- 172.21.15.40:0/1828187156 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa514004160 con 0x7fa518106840 2024-06-06T03:27:52.045 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.979+0000 7fa50ffff700 1 -- 172.21.15.40:0/1828187156 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa514004300 con 0x7fa518106840 2024-06-06T03:27:52.045 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.979+0000 7fa50ffff700 1 -- 172.21.15.40:0/1828187156 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa514011b20 con 0x7fa518106840 2024-06-06T03:27:52.046 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.980+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1828187156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa518106840 msgr2=0x7fa518106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:52.046 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.980+0000 7fa51f9d3700 1 --2- 172.21.15.40:0/1828187156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa518106840 0x7fa518106c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5140098a0 tx=0x7fa514009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:52.046 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.980+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1828187156 shutdown_connections 2024-06-06T03:27:52.046 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.980+0000 7fa51f9d3700 1 --2- 172.21.15.40:0/1828187156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa518106840 0x7fa518106c60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:52.046 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.980+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1828187156 >> 172.21.15.40:0/1828187156 conn(0x7fa518102440 msgr2=0x7fa518104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:52.046 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.980+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1828187156 shutdown_connections 2024-06-06T03:27:52.046 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.980+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1828187156 wait complete. 2024-06-06T03:27:52.046 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.980+0000 7fa51f9d3700 1 Processor -- start 2024-06-06T03:27:52.046 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.981+0000 7fa51f9d3700 1 -- start start 2024-06-06T03:27:52.046 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.981+0000 7fa51f9d3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa518106840 0x7fa51819e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:52.046 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.981+0000 7fa51f9d3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa518108270 con 0x7fa518106840 2024-06-06T03:27:52.046 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.981+0000 7fa51d76f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa518106840 0x7fa51819e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:52.047 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.981+0000 7fa51d76f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa518106840 0x7fa51819e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55342/0 (socket says 172.21.15.40:55342) 2024-06-06T03:27:52.047 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.981+0000 7fa51d76f700 1 -- 172.21.15.40:0/1616978355 learned_addr learned my addr 172.21.15.40:0/1616978355 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:52.047 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.981+0000 7fa51d76f700 1 -- 172.21.15.40:0/1616978355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51819e5d0 con 0x7fa518106840 2024-06-06T03:27:52.047 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.981+0000 7fa51d76f700 1 --2- 172.21.15.40:0/1616978355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa518106840 0x7fa51819e090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa514011b60 tx=0x7fa514011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:52.047 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.981+0000 7fa50e7fc700 1 -- 172.21.15.40:0/1616978355 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa514003b60 con 0x7fa518106840 2024-06-06T03:27:52.047 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.982+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1616978355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51819e860 con 0x7fa518106840 2024-06-06T03:27:52.047 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.982+0000 7fa50e7fc700 1 -- 172.21.15.40:0/1616978355 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa51401a910 con 0x7fa518106840 2024-06-06T03:27:52.047 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.982+0000 7fa50e7fc700 1 -- 172.21.15.40:0/1616978355 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa514024930 con 0x7fa518106840 2024-06-06T03:27:52.047 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.982+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1616978355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa51819ecc0 con 0x7fa518106840 2024-06-06T03:27:52.047 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.982+0000 7fa50e7fc700 1 -- 172.21.15.40:0/1616978355 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fa514022020 con 0x7fa518106840 2024-06-06T03:27:52.047 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.983+0000 7fa50e7fc700 1 --2- 172.21.15.40:0/1616978355 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa5040390a0 0x7fa50403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:52.047 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.983+0000 7fa50e7fc700 1 -- 172.21.15.40:0/1616978355 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa51404d3b0 con 0x7fa518106840 2024-06-06T03:27:52.047 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.983+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1616978355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa518063af0 con 0x7fa518106840 2024-06-06T03:27:52.048 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.983+0000 7fa51cf6e700 1 --2- 172.21.15.40:0/1616978355 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa5040390a0 0x7fa50403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:52.048 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.983+0000 7fa51cf6e700 1 --2- 172.21.15.40:0/1616978355 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa5040390a0 0x7fa50403b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa508006fd0 tx=0x7fa508006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:52.048 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:50.988+0000 7fa50e7fc700 1 -- 172.21.15.40:0/1616978355 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa5140172b0 con 0x7fa518106840 2024-06-06T03:27:52.048 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:51.215+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1616978355 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fa518197bc0 con 0x7fa5040390a0 2024-06-06T03:27:52.048 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:51.463+0000 7fa50e7fc700 1 -- 172.21.15.40:0/1616978355 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fa518197bc0 con 0x7fa5040390a0 2024-06-06T03:27:52.048 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:51.465+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1616978355 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa5040390a0 msgr2=0x7fa50403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:52.048 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:51.465+0000 7fa51f9d3700 1 --2- 172.21.15.40:0/1616978355 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa5040390a0 0x7fa50403b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa508006fd0 tx=0x7fa508006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:52.048 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:51.465+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1616978355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa518106840 msgr2=0x7fa51819e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:52.048 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:51.465+0000 7fa51f9d3700 1 --2- 172.21.15.40:0/1616978355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa518106840 0x7fa51819e090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa514011b60 tx=0x7fa514011c40 comp rx=0 tx=0).stop 2024-06-06T03:27:52.048 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:51.465+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1616978355 shutdown_connections 2024-06-06T03:27:52.048 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:51.465+0000 7fa51f9d3700 1 --2- 172.21.15.40:0/1616978355 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa5040390a0 0x7fa50403b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:52.048 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:51.465+0000 7fa51f9d3700 1 --2- 172.21.15.40:0/1616978355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa518106840 0x7fa51819e090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:52.048 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:51.465+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1616978355 >> 172.21.15.40:0/1616978355 conn(0x7fa518102440 msgr2=0x7fa518194d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:52.049 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:51.465+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1616978355 shutdown_connections 2024-06-06T03:27:52.049 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:51.465+0000 7fa51f9d3700 1 -- 172.21.15.40:0/1616978355 wait complete. 2024-06-06T03:27:52.049 INFO:teuthology.orchestra.run.smithi040.stdout:Fetching dashboard port number... 2024-06-06T03:27:52.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:52 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2166120093' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-06T03:27:54.034 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 8443 2024-06-06T03:27:54.034 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.495+0000 7f5cf72f7700 1 Processor -- start 2024-06-06T03:27:54.034 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.495+0000 7f5cf72f7700 1 -- start start 2024-06-06T03:27:54.034 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.495+0000 7f5cf72f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cf0106e90 0x7f5cf01072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:54.034 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.495+0000 7f5cf72f7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01077f0 con 0x7f5cf0106e90 2024-06-06T03:27:54.034 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.496+0000 7f5cf5093700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cf0106e90 0x7f5cf01072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:54.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.496+0000 7f5cf5093700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cf0106e90 0x7f5cf01072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55358/0 (socket says 172.21.15.40:55358) 2024-06-06T03:27:54.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.496+0000 7f5cf5093700 1 -- 172.21.15.40:0/925596574 learned_addr learned my addr 172.21.15.40:0/925596574 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:54.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.496+0000 7f5cf5093700 1 -- 172.21.15.40:0/925596574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf0107fa0 con 0x7f5cf0106e90 2024-06-06T03:27:54.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.496+0000 7f5cf5093700 1 --2- 172.21.15.40:0/925596574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cf0106e90 0x7f5cf01072b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0098a0 tx=0x7f5cec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2ef4a700c4f05b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:54.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.496+0000 7f5ce7fff700 1 -- 172.21.15.40:0/925596574 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cec004160 con 0x7f5cf0106e90 2024-06-06T03:27:54.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.497+0000 7f5ce7fff700 1 -- 172.21.15.40:0/925596574 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cec004300 con 0x7f5cf0106e90 2024-06-06T03:27:54.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.497+0000 7f5ce7fff700 1 -- 172.21.15.40:0/925596574 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cec011b20 con 0x7f5cf0106e90 2024-06-06T03:27:54.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.497+0000 7f5cf72f7700 1 -- 172.21.15.40:0/925596574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cf0106e90 msgr2=0x7f5cf01072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:54.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.497+0000 7f5cf72f7700 1 --2- 172.21.15.40:0/925596574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cf0106e90 0x7f5cf01072b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0098a0 tx=0x7f5cec009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:54.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.497+0000 7f5cf72f7700 1 -- 172.21.15.40:0/925596574 shutdown_connections 2024-06-06T03:27:54.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.497+0000 7f5cf72f7700 1 --2- 172.21.15.40:0/925596574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cf0106e90 0x7f5cf01072b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:54.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.497+0000 7f5cf72f7700 1 -- 172.21.15.40:0/925596574 >> 172.21.15.40:0/925596574 conn(0x7f5cf0102430 msgr2=0x7f5cf0104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:54.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.497+0000 7f5cf72f7700 1 -- 172.21.15.40:0/925596574 shutdown_connections 2024-06-06T03:27:54.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.497+0000 7f5cf72f7700 1 -- 172.21.15.40:0/925596574 wait complete. 2024-06-06T03:27:54.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.498+0000 7f5cf72f7700 1 Processor -- start 2024-06-06T03:27:54.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.498+0000 7f5cf72f7700 1 -- start start 2024-06-06T03:27:54.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.498+0000 7f5cf72f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cf0106e90 0x7f5cf019be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:54.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.498+0000 7f5cf72f7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf010b3e0 con 0x7f5cf0106e90 2024-06-06T03:27:54.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.498+0000 7f5cf5093700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cf0106e90 0x7f5cf019be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:54.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.498+0000 7f5cf5093700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cf0106e90 0x7f5cf019be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55366/0 (socket says 172.21.15.40:55366) 2024-06-06T03:27:54.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.498+0000 7f5cf5093700 1 -- 172.21.15.40:0/2166120093 learned_addr learned my addr 172.21.15.40:0/2166120093 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:54.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.499+0000 7f5cf5093700 1 -- 172.21.15.40:0/2166120093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf019c360 con 0x7f5cf0106e90 2024-06-06T03:27:54.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.499+0000 7f5cf5093700 1 --2- 172.21.15.40:0/2166120093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cf0106e90 0x7f5cf019be20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0116a0 tx=0x7f5cec011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:54.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.499+0000 7f5ce67fc700 1 -- 172.21.15.40:0/2166120093 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cec003a40 con 0x7f5cf0106e90 2024-06-06T03:27:54.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.499+0000 7f5ce67fc700 1 -- 172.21.15.40:0/2166120093 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cec003be0 con 0x7f5cf0106e90 2024-06-06T03:27:54.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.499+0000 7f5ce67fc700 1 -- 172.21.15.40:0/2166120093 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cec01a750 con 0x7f5cf0106e90 2024-06-06T03:27:54.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.499+0000 7f5cf72f7700 1 -- 172.21.15.40:0/2166120093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf019c5f0 con 0x7f5cf0106e90 2024-06-06T03:27:54.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.499+0000 7f5cf72f7700 1 -- 172.21.15.40:0/2166120093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf019ca50 con 0x7f5cf0106e90 2024-06-06T03:27:54.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.500+0000 7f5ce67fc700 1 -- 172.21.15.40:0/2166120093 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f5cec024aa0 con 0x7f5cf0106e90 2024-06-06T03:27:54.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.500+0000 7f5ce67fc700 1 --2- 172.21.15.40:0/2166120093 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5cdc0390b0 0x7f5cdc03b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:54.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.500+0000 7f5cf72f7700 1 -- 172.21.15.40:0/2166120093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf0063af0 con 0x7f5cf0106e90 2024-06-06T03:27:54.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.500+0000 7f5ce67fc700 1 -- 172.21.15.40:0/2166120093 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5cec04d770 con 0x7f5cf0106e90 2024-06-06T03:27:54.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.501+0000 7f5cf4892700 1 --2- 172.21.15.40:0/2166120093 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5cdc0390b0 0x7f5cdc03b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:54.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.501+0000 7f5cf4892700 1 --2- 172.21.15.40:0/2166120093 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5cdc0390b0 0x7f5cdc03b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0006fd0 tx=0x7f5ce0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:54.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.506+0000 7f5ce67fc700 1 -- 172.21.15.40:0/2166120093 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5cec011e20 con 0x7f5cf0106e90 2024-06-06T03:27:54.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.717+0000 7f5cf72f7700 1 -- 172.21.15.40:0/2166120093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f5cf019f420 con 0x7f5cf0106e90 2024-06-06T03:27:54.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.718+0000 7f5ce67fc700 1 -- 172.21.15.40:0/2166120093 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f5cec0172b0 con 0x7f5cf0106e90 2024-06-06T03:27:54.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.722+0000 7f5cf72f7700 1 -- 172.21.15.40:0/2166120093 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5cdc0390b0 msgr2=0x7f5cdc03b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:54.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.722+0000 7f5cf72f7700 1 --2- 172.21.15.40:0/2166120093 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5cdc0390b0 0x7f5cdc03b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0006fd0 tx=0x7f5ce0006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:54.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.722+0000 7f5cf72f7700 1 -- 172.21.15.40:0/2166120093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cf0106e90 msgr2=0x7f5cf019be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:54.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.722+0000 7f5cf72f7700 1 --2- 172.21.15.40:0/2166120093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cf0106e90 0x7f5cf019be20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0116a0 tx=0x7f5cec011ba0 comp rx=0 tx=0).stop 2024-06-06T03:27:54.038 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.722+0000 7f5cf72f7700 1 -- 172.21.15.40:0/2166120093 shutdown_connections 2024-06-06T03:27:54.038 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.722+0000 7f5cf72f7700 1 --2- 172.21.15.40:0/2166120093 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5cdc0390b0 0x7f5cdc03b570 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:54.038 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.722+0000 7f5cf72f7700 1 --2- 172.21.15.40:0/2166120093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cf0106e90 0x7f5cf019be20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:54.038 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.722+0000 7f5cf72f7700 1 -- 172.21.15.40:0/2166120093 >> 172.21.15.40:0/2166120093 conn(0x7f5cf0102430 msgr2=0x7f5cf0102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:54.038 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.722+0000 7f5cf72f7700 1 -- 172.21.15.40:0/2166120093 shutdown_connections 2024-06-06T03:27:54.038 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:52.722+0000 7f5cf72f7700 1 -- 172.21.15.40:0/2166120093 wait complete. 2024-06-06T03:27:54.044 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T03:27:54.044 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout disabled 2024-06-06T03:27:54.053 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T03:27:54.053 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2024-06-06T03:27:54.053 INFO:teuthology.orchestra.run.smithi040.stdout:firewalld.service is not enabled 2024-06-06T03:27:54.053 INFO:teuthology.orchestra.run.smithi040.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-06T03:27:54.056 INFO:teuthology.orchestra.run.smithi040.stdout:Ceph Dashboard is now available at: 2024-06-06T03:27:54.056 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:27:54.056 INFO:teuthology.orchestra.run.smithi040.stdout: URL: https://smithi040.front.sepia.ceph.com:8443/ 2024-06-06T03:27:54.056 INFO:teuthology.orchestra.run.smithi040.stdout: User: admin 2024-06-06T03:27:54.056 INFO:teuthology.orchestra.run.smithi040.stdout: Password: gif8hnqdd4 2024-06-06T03:27:54.056 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:27:54.056 INFO:teuthology.orchestra.run.smithi040.stdout:Saving cluster configuration to /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config directory 2024-06-06T03:27:54.057 INFO:teuthology.orchestra.run.smithi040.stdout:Enabling autotune for osd_memory_target 2024-06-06T03:27:55.049 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:54 smithi040 ceph-mon[32220]: mgrmap e13: x(active, since 6s) 2024-06-06T03:27:55.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.493+0000 7fe79ccf1700 1 Processor -- start 2024-06-06T03:27:55.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.493+0000 7fe79ccf1700 1 -- start start 2024-06-06T03:27:55.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.493+0000 7fe79ccf1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798107f80 0x7fe7981083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:55.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.493+0000 7fe79ccf1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe798108970 con 0x7fe798107f80 2024-06-06T03:27:55.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.493+0000 7fe79659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798107f80 0x7fe7981083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:55.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.493+0000 7fe79659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798107f80 0x7fe7981083a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55378/0 (socket says 172.21.15.40:55378) 2024-06-06T03:27:55.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.494+0000 7fe79659c700 1 -- 172.21.15.40:0/4055147037 learned_addr learned my addr 172.21.15.40:0/4055147037 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:55.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.494+0000 7fe79659c700 1 -- 172.21.15.40:0/4055147037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe798109180 con 0x7fe798107f80 2024-06-06T03:27:55.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.494+0000 7fe79659c700 1 --2- 172.21.15.40:0/4055147037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798107f80 0x7fe7981083a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe7880098a0 tx=0x7fe788009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91df2432da08f0ea server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:55.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.494+0000 7fe79559a700 1 -- 172.21.15.40:0/4055147037 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe788004160 con 0x7fe798107f80 2024-06-06T03:27:55.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.494+0000 7fe79559a700 1 -- 172.21.15.40:0/4055147037 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe788004300 con 0x7fe798107f80 2024-06-06T03:27:55.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.494+0000 7fe79559a700 1 -- 172.21.15.40:0/4055147037 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe788011b20 con 0x7fe798107f80 2024-06-06T03:27:55.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.495+0000 7fe79ccf1700 1 -- 172.21.15.40:0/4055147037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798107f80 msgr2=0x7fe7981083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:55.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.495+0000 7fe79ccf1700 1 --2- 172.21.15.40:0/4055147037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798107f80 0x7fe7981083a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe7880098a0 tx=0x7fe788009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:55.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.495+0000 7fe79ccf1700 1 -- 172.21.15.40:0/4055147037 shutdown_connections 2024-06-06T03:27:55.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.495+0000 7fe79ccf1700 1 --2- 172.21.15.40:0/4055147037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798107f80 0x7fe7981083a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:55.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.495+0000 7fe79ccf1700 1 -- 172.21.15.40:0/4055147037 >> 172.21.15.40:0/4055147037 conn(0x7fe798103580 msgr2=0x7fe7981059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:55.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.495+0000 7fe79ccf1700 1 -- 172.21.15.40:0/4055147037 shutdown_connections 2024-06-06T03:27:55.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.495+0000 7fe79ccf1700 1 -- 172.21.15.40:0/4055147037 wait complete. 2024-06-06T03:27:55.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.496+0000 7fe79ccf1700 1 Processor -- start 2024-06-06T03:27:55.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.496+0000 7fe79ccf1700 1 -- start start 2024-06-06T03:27:55.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.496+0000 7fe79ccf1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798107f80 0x7fe79807e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:55.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.496+0000 7fe79ccf1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe79810c530 con 0x7fe798107f80 2024-06-06T03:27:55.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.496+0000 7fe79659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798107f80 0x7fe79807e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:55.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.496+0000 7fe79659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798107f80 0x7fe79807e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55390/0 (socket says 172.21.15.40:55390) 2024-06-06T03:27:55.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.496+0000 7fe79659c700 1 -- 172.21.15.40:0/1245805323 learned_addr learned my addr 172.21.15.40:0/1245805323 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:55.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.496+0000 7fe79659c700 1 -- 172.21.15.40:0/1245805323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79807e910 con 0x7fe798107f80 2024-06-06T03:27:55.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.496+0000 7fe79659c700 1 --2- 172.21.15.40:0/1245805323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798107f80 0x7fe79807e3d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe788000c00 tx=0x7fe788011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:55.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.497+0000 7fe7877fe700 1 -- 172.21.15.40:0/1245805323 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe788003b60 con 0x7fe798107f80 2024-06-06T03:27:55.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.497+0000 7fe7877fe700 1 -- 172.21.15.40:0/1245805323 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe78801a910 con 0x7fe798107f80 2024-06-06T03:27:55.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.497+0000 7fe79ccf1700 1 -- 172.21.15.40:0/1245805323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79807cb00 con 0x7fe798107f80 2024-06-06T03:27:55.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.497+0000 7fe7877fe700 1 -- 172.21.15.40:0/1245805323 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe788024930 con 0x7fe798107f80 2024-06-06T03:27:55.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.497+0000 7fe79ccf1700 1 -- 172.21.15.40:0/1245805323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79807cfc0 con 0x7fe798107f80 2024-06-06T03:27:55.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.498+0000 7fe7877fe700 1 -- 172.21.15.40:0/1245805323 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe788022020 con 0x7fe798107f80 2024-06-06T03:27:55.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.498+0000 7fe79ccf1700 1 -- 172.21.15.40:0/1245805323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe798063af0 con 0x7fe798107f80 2024-06-06T03:27:55.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.498+0000 7fe7877fe700 1 --2- 172.21.15.40:0/1245805323 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe7800391a0 0x7fe78003b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:55.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.498+0000 7fe7877fe700 1 -- 172.21.15.40:0/1245805323 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe78804d370 con 0x7fe798107f80 2024-06-06T03:27:55.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.498+0000 7fe795d9b700 1 --2- 172.21.15.40:0/1245805323 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe7800391a0 0x7fe78003b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:55.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.498+0000 7fe795d9b700 1 --2- 172.21.15.40:0/1245805323 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe7800391a0 0x7fe78003b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe78c006fd0 tx=0x7fe78c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:55.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.503+0000 7fe7877fe700 1 -- 172.21.15.40:0/1245805323 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe78802c350 con 0x7fe798107f80 2024-06-06T03:27:55.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.711+0000 7fe79ccf1700 1 -- 172.21.15.40:0/1245805323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fe79807d240 con 0x7fe798107f80 2024-06-06T03:27:55.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.711+0000 7fe7877fe700 1 -- 172.21.15.40:0/1245805323 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fe78802ad60 con 0x7fe798107f80 2024-06-06T03:27:55.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.713+0000 7fe79ccf1700 1 -- 172.21.15.40:0/1245805323 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe7800391a0 msgr2=0x7fe78003b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:55.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.713+0000 7fe79ccf1700 1 --2- 172.21.15.40:0/1245805323 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe7800391a0 0x7fe78003b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe78c006fd0 tx=0x7fe78c006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:55.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.713+0000 7fe79ccf1700 1 -- 172.21.15.40:0/1245805323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798107f80 msgr2=0x7fe79807e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:55.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.713+0000 7fe79ccf1700 1 --2- 172.21.15.40:0/1245805323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798107f80 0x7fe79807e3d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe788000c00 tx=0x7fe788011c10 comp rx=0 tx=0).stop 2024-06-06T03:27:55.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.714+0000 7fe79ccf1700 1 -- 172.21.15.40:0/1245805323 shutdown_connections 2024-06-06T03:27:55.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.714+0000 7fe79ccf1700 1 --2- 172.21.15.40:0/1245805323 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe7800391a0 0x7fe78003b660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:55.327 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.714+0000 7fe79ccf1700 1 --2- 172.21.15.40:0/1245805323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798107f80 0x7fe79807e3d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:55.327 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.714+0000 7fe79ccf1700 1 -- 172.21.15.40:0/1245805323 >> 172.21.15.40:0/1245805323 conn(0x7fe798103580 msgr2=0x7fe798104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:55.327 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.714+0000 7fe79ccf1700 1 -- 172.21.15.40:0/1245805323 shutdown_connections 2024-06-06T03:27:55.327 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:54.714+0000 7fe79ccf1700 1 -- 172.21.15.40:0/1245805323 wait complete. 2024-06-06T03:27:56.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.743+0000 7f039a76c700 1 Processor -- start 2024-06-06T03:27:56.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.743+0000 7f039a76c700 1 -- start start 2024-06-06T03:27:56.638 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.744+0000 7f039a76c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0394106840 0x7f0394106c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:56.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.744+0000 7f039a76c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f039407c640 con 0x7f0394106840 2024-06-06T03:27:56.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.744+0000 7f0393fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0394106840 0x7f0394106c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:56.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.744+0000 7f0393fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0394106840 0x7f0394106c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55396/0 (socket says 172.21.15.40:55396) 2024-06-06T03:27:56.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.744+0000 7f0393fff700 1 -- 172.21.15.40:0/2479896697 learned_addr learned my addr 172.21.15.40:0/2479896697 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:56.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.744+0000 7f0393fff700 1 -- 172.21.15.40:0/2479896697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03941071a0 con 0x7f0394106840 2024-06-06T03:27:56.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.744+0000 7f0393fff700 1 --2- 172.21.15.40:0/2479896697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0394106840 0x7f0394106c60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f03840098a0 tx=0x7f0384009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c713c259282d156b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:56.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.745+0000 7f0392ffd700 1 -- 172.21.15.40:0/2479896697 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0384004160 con 0x7f0394106840 2024-06-06T03:27:56.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.745+0000 7f0392ffd700 1 -- 172.21.15.40:0/2479896697 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0384004300 con 0x7f0394106840 2024-06-06T03:27:56.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.745+0000 7f0392ffd700 1 -- 172.21.15.40:0/2479896697 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0384011b20 con 0x7f0394106840 2024-06-06T03:27:56.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.745+0000 7f039a76c700 1 -- 172.21.15.40:0/2479896697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0394106840 msgr2=0x7f0394106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:56.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.745+0000 7f039a76c700 1 --2- 172.21.15.40:0/2479896697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0394106840 0x7f0394106c60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f03840098a0 tx=0x7f0384009bb0 comp rx=0 tx=0).stop 2024-06-06T03:27:56.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.745+0000 7f039a76c700 1 -- 172.21.15.40:0/2479896697 shutdown_connections 2024-06-06T03:27:56.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.745+0000 7f039a76c700 1 --2- 172.21.15.40:0/2479896697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0394106840 0x7f0394106c60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:56.639 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.745+0000 7f039a76c700 1 -- 172.21.15.40:0/2479896697 >> 172.21.15.40:0/2479896697 conn(0x7f0394102400 msgr2=0x7f0394104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.745+0000 7f039a76c700 1 -- 172.21.15.40:0/2479896697 shutdown_connections 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.745+0000 7f039a76c700 1 -- 172.21.15.40:0/2479896697 wait complete. 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.746+0000 7f039a76c700 1 Processor -- start 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.746+0000 7f039a76c700 1 -- start start 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.746+0000 7f039a76c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0394106840 0x7f039419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.746+0000 7f039a76c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f03941080f0 con 0x7f0394106840 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.747+0000 7f0393fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0394106840 0x7f039419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.747+0000 7f0393fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0394106840 0x7f039419dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55410/0 (socket says 172.21.15.40:55410) 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.747+0000 7f0393fff700 1 -- 172.21.15.40:0/3082552034 learned_addr learned my addr 172.21.15.40:0/3082552034 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.747+0000 7f0393fff700 1 -- 172.21.15.40:0/3082552034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039419e530 con 0x7f0394106840 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.747+0000 7f0393fff700 1 --2- 172.21.15.40:0/3082552034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0394106840 0x7f039419dff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0384011b60 tx=0x7f0384011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.747+0000 7f03917fa700 1 -- 172.21.15.40:0/3082552034 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0384003b60 con 0x7f0394106840 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.747+0000 7f03917fa700 1 -- 172.21.15.40:0/3082552034 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f038401a910 con 0x7f0394106840 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.747+0000 7f039a76c700 1 -- 172.21.15.40:0/3082552034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039419e7c0 con 0x7f0394106840 2024-06-06T03:27:56.640 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.747+0000 7f03917fa700 1 -- 172.21.15.40:0/3082552034 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0384024930 con 0x7f0394106840 2024-06-06T03:27:56.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.747+0000 7f039a76c700 1 -- 172.21.15.40:0/3082552034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039419ec20 con 0x7f0394106840 2024-06-06T03:27:56.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.748+0000 7f03917fa700 1 -- 172.21.15.40:0/3082552034 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0384022020 con 0x7f0394106840 2024-06-06T03:27:56.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.748+0000 7f039a76c700 1 -- 172.21.15.40:0/3082552034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0394063af0 con 0x7f0394106840 2024-06-06T03:27:56.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.748+0000 7f03917fa700 1 --2- 172.21.15.40:0/3082552034 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f037c0391a0 0x7f037c03b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:56.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.749+0000 7f03917fa700 1 -- 172.21.15.40:0/3082552034 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f038404d4b0 con 0x7f0394106840 2024-06-06T03:27:56.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.749+0000 7f03937fe700 1 --2- 172.21.15.40:0/3082552034 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f037c0391a0 0x7f037c03b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:56.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.749+0000 7f03937fe700 1 --2- 172.21.15.40:0/3082552034 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f037c0391a0 0x7f037c03b660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0388006fd0 tx=0x7f0388006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:56.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:55.754+0000 7f03917fa700 1 -- 172.21.15.40:0/3082552034 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f038401a4e0 con 0x7f0394106840 2024-06-06T03:27:56.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:56.073+0000 7f039a76c700 1 -- 172.21.15.40:0/3082552034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f03940630a0 con 0x7f0394106840 2024-06-06T03:27:56.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:56.076+0000 7f03917fa700 1 -- 172.21.15.40:0/3082552034 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f038402d3f0 con 0x7f0394106840 2024-06-06T03:27:56.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:56.079+0000 7f039a76c700 1 -- 172.21.15.40:0/3082552034 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f037c0391a0 msgr2=0x7f037c03b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:56.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:56.079+0000 7f039a76c700 1 --2- 172.21.15.40:0/3082552034 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f037c0391a0 0x7f037c03b660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0388006fd0 tx=0x7f0388006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:56.641 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:56.079+0000 7f039a76c700 1 -- 172.21.15.40:0/3082552034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0394106840 msgr2=0x7f039419dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:56.642 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:56.079+0000 7f039a76c700 1 --2- 172.21.15.40:0/3082552034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0394106840 0x7f039419dff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0384011b60 tx=0x7f0384011c40 comp rx=0 tx=0).stop 2024-06-06T03:27:56.642 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:56.079+0000 7f039a76c700 1 -- 172.21.15.40:0/3082552034 shutdown_connections 2024-06-06T03:27:56.642 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:56.079+0000 7f039a76c700 1 --2- 172.21.15.40:0/3082552034 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f037c0391a0 0x7f037c03b660 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:56.642 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:56.079+0000 7f039a76c700 1 --2- 172.21.15.40:0/3082552034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0394106840 0x7f039419dff0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:56.642 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:56.079+0000 7f039a76c700 1 -- 172.21.15.40:0/3082552034 >> 172.21.15.40:0/3082552034 conn(0x7f0394102400 msgr2=0x7f0394194c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:56.642 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:56.079+0000 7f039a76c700 1 -- 172.21.15.40:0/3082552034 shutdown_connections 2024-06-06T03:27:56.642 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-06-06T03:27:56.079+0000 7f039a76c700 1 -- 172.21.15.40:0/3082552034 wait complete. 2024-06-06T03:27:56.642 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-06T03:27:56.642 INFO:teuthology.orchestra.run.smithi040.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-06T03:27:56.642 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:27:56.642 INFO:teuthology.orchestra.run.smithi040.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-06T03:27:56.642 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:27:56.642 INFO:teuthology.orchestra.run.smithi040.stdout:Or, if you are only running a single cluster on this host: 2024-06-06T03:27:56.642 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:27:56.643 INFO:teuthology.orchestra.run.smithi040.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-06T03:27:56.643 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:27:56.643 INFO:teuthology.orchestra.run.smithi040.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-06T03:27:56.643 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:27:56.643 INFO:teuthology.orchestra.run.smithi040.stdout: ceph telemetry on 2024-06-06T03:27:56.643 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:27:56.643 INFO:teuthology.orchestra.run.smithi040.stdout:For more information see: 2024-06-06T03:27:56.643 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:27:56.643 INFO:teuthology.orchestra.run.smithi040.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-06T03:27:56.643 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:27:56.643 INFO:teuthology.orchestra.run.smithi040.stdout:Bootstrap complete. 2024-06-06T03:27:56.680 INFO:tasks.cephadm:Fetching config... 2024-06-06T03:27:56.680 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:27:56.680 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-06T03:27:56.697 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-06T03:27:56.697 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:27:56.698 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-06T03:27:56.753 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-06T03:27:56.754 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:27:56.754 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-06T03:27:56.820 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-06T03:27:56.820 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:27:56.820 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-06T03:27:56.876 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-06T03:27:56.876 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjInNI2ZvFZ4i16r6bn9u8f5UhAAdNkZROqZ0tGeUzNC4WqwzX3iMvIuRl9FVQSns66EM6XIxe3wCXWsuANkz4h89qYcX+xq/SNCUZaAe4F2T3Dfqff3fuereto7pHD6mO7+WduqXIGzyHoPXsTa/NL0oTodsnWdHa/pWATJu8WRI9CRyiT2q+Bmv30EMLxjqZP7HHHvj8/RwOg6cTB7lZq4JZZho6QHZUEHvT9I0EkmRRBw7Co2VTSqpJ1Zi9IWHrLnlnJvpat8NBbDS3qa1twZTfPxU1eBlkwHehkaoEFs79aH5nGewx7swSRbATFAQqoEI45UJf679D8KLv4vKBHdKQWLlI1DfW1Darv3wMAQLV2qXddtNqxvq6K4Z1S0vAinlFREhI3sAQTJUp+1N8VMt0vlsPIKJnPFIfrK1MDtuo9ZO4CnwrIIT9BZhFeTaUgBN6NL8TPECAAdVP4+vjPkqF2e4ZZo/2HCn9bJ2FycqE+kXNpKbN6NYp7ykUcTM= ceph-77c25d68-23b4-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T03:27:56.951 INFO:teuthology.orchestra.run.smithi040.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjInNI2ZvFZ4i16r6bn9u8f5UhAAdNkZROqZ0tGeUzNC4WqwzX3iMvIuRl9FVQSns66EM6XIxe3wCXWsuANkz4h89qYcX+xq/SNCUZaAe4F2T3Dfqff3fuereto7pHD6mO7+WduqXIGzyHoPXsTa/NL0oTodsnWdHa/pWATJu8WRI9CRyiT2q+Bmv30EMLxjqZP7HHHvj8/RwOg6cTB7lZq4JZZho6QHZUEHvT9I0EkmRRBw7Co2VTSqpJ1Zi9IWHrLnlnJvpat8NBbDS3qa1twZTfPxU1eBlkwHehkaoEFs79aH5nGewx7swSRbATFAQqoEI45UJf679D8KLv4vKBHdKQWLlI1DfW1Darv3wMAQLV2qXddtNqxvq6K4Z1S0vAinlFREhI3sAQTJUp+1N8VMt0vlsPIKJnPFIfrK1MDtuo9ZO4CnwrIIT9BZhFeTaUgBN6NL8TPECAAdVP4+vjPkqF2e4ZZo/2HCn9bJ2FycqE+kXNpKbN6NYp7ykUcTM= ceph-77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:27:56.963 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjInNI2ZvFZ4i16r6bn9u8f5UhAAdNkZROqZ0tGeUzNC4WqwzX3iMvIuRl9FVQSns66EM6XIxe3wCXWsuANkz4h89qYcX+xq/SNCUZaAe4F2T3Dfqff3fuereto7pHD6mO7+WduqXIGzyHoPXsTa/NL0oTodsnWdHa/pWATJu8WRI9CRyiT2q+Bmv30EMLxjqZP7HHHvj8/RwOg6cTB7lZq4JZZho6QHZUEHvT9I0EkmRRBw7Co2VTSqpJ1Zi9IWHrLnlnJvpat8NBbDS3qa1twZTfPxU1eBlkwHehkaoEFs79aH5nGewx7swSRbATFAQqoEI45UJf679D8KLv4vKBHdKQWLlI1DfW1Darv3wMAQLV2qXddtNqxvq6K4Z1S0vAinlFREhI3sAQTJUp+1N8VMt0vlsPIKJnPFIfrK1MDtuo9ZO4CnwrIIT9BZhFeTaUgBN6NL8TPECAAdVP4+vjPkqF2e4ZZo/2HCn9bJ2FycqE+kXNpKbN6NYp7ykUcTM= ceph-77c25d68-23b4-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T03:27:57.002 INFO:teuthology.orchestra.run.smithi104.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjInNI2ZvFZ4i16r6bn9u8f5UhAAdNkZROqZ0tGeUzNC4WqwzX3iMvIuRl9FVQSns66EM6XIxe3wCXWsuANkz4h89qYcX+xq/SNCUZaAe4F2T3Dfqff3fuereto7pHD6mO7+WduqXIGzyHoPXsTa/NL0oTodsnWdHa/pWATJu8WRI9CRyiT2q+Bmv30EMLxjqZP7HHHvj8/RwOg6cTB7lZq4JZZho6QHZUEHvT9I0EkmRRBw7Co2VTSqpJ1Zi9IWHrLnlnJvpat8NBbDS3qa1twZTfPxU1eBlkwHehkaoEFs79aH5nGewx7swSRbATFAQqoEI45UJf679D8KLv4vKBHdKQWLlI1DfW1Darv3wMAQLV2qXddtNqxvq6K4Z1S0vAinlFREhI3sAQTJUp+1N8VMt0vlsPIKJnPFIfrK1MDtuo9ZO4CnwrIIT9BZhFeTaUgBN6NL8TPECAAdVP4+vjPkqF2e4ZZo/2HCn9bJ2FycqE+kXNpKbN6NYp7ykUcTM= ceph-77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:27:57.014 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjInNI2ZvFZ4i16r6bn9u8f5UhAAdNkZROqZ0tGeUzNC4WqwzX3iMvIuRl9FVQSns66EM6XIxe3wCXWsuANkz4h89qYcX+xq/SNCUZaAe4F2T3Dfqff3fuereto7pHD6mO7+WduqXIGzyHoPXsTa/NL0oTodsnWdHa/pWATJu8WRI9CRyiT2q+Bmv30EMLxjqZP7HHHvj8/RwOg6cTB7lZq4JZZho6QHZUEHvT9I0EkmRRBw7Co2VTSqpJ1Zi9IWHrLnlnJvpat8NBbDS3qa1twZTfPxU1eBlkwHehkaoEFs79aH5nGewx7swSRbATFAQqoEI45UJf679D8KLv4vKBHdKQWLlI1DfW1Darv3wMAQLV2qXddtNqxvq6K4Z1S0vAinlFREhI3sAQTJUp+1N8VMt0vlsPIKJnPFIfrK1MDtuo9ZO4CnwrIIT9BZhFeTaUgBN6NL8TPECAAdVP4+vjPkqF2e4ZZo/2HCn9bJ2FycqE+kXNpKbN6NYp7ykUcTM= ceph-77c25d68-23b4-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T03:27:57.055 INFO:teuthology.orchestra.run.smithi131.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjInNI2ZvFZ4i16r6bn9u8f5UhAAdNkZROqZ0tGeUzNC4WqwzX3iMvIuRl9FVQSns66EM6XIxe3wCXWsuANkz4h89qYcX+xq/SNCUZaAe4F2T3Dfqff3fuereto7pHD6mO7+WduqXIGzyHoPXsTa/NL0oTodsnWdHa/pWATJu8WRI9CRyiT2q+Bmv30EMLxjqZP7HHHvj8/RwOg6cTB7lZq4JZZho6QHZUEHvT9I0EkmRRBw7Co2VTSqpJ1Zi9IWHrLnlnJvpat8NBbDS3qa1twZTfPxU1eBlkwHehkaoEFs79aH5nGewx7swSRbATFAQqoEI45UJf679D8KLv4vKBHdKQWLlI1DfW1Darv3wMAQLV2qXddtNqxvq6K4Z1S0vAinlFREhI3sAQTJUp+1N8VMt0vlsPIKJnPFIfrK1MDtuo9ZO4CnwrIIT9BZhFeTaUgBN6NL8TPECAAdVP4+vjPkqF2e4ZZo/2HCn9bJ2FycqE+kXNpKbN6NYp7ykUcTM= ceph-77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:27:57.065 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-06T03:27:57.192 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:27:57 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3082552034' entity='client.admin' 2024-06-06T03:27:57.317 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:27:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.656+0000 7f2fe11d4700 1 -- 172.21.15.40:0/4047203478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fdc103560 msgr2=0x7f2fdc103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.656+0000 7f2fe11d4700 1 --2- 172.21.15.40:0/4047203478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fdc103560 0x7f2fdc103980 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009910 tx=0x7f2fcc009c20 comp rx=0 tx=0).stop 2024-06-06T03:27:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.656+0000 7f2fe11d4700 1 -- 172.21.15.40:0/4047203478 shutdown_connections 2024-06-06T03:27:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.656+0000 7f2fe11d4700 1 --2- 172.21.15.40:0/4047203478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fdc103560 0x7f2fdc103980 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.656+0000 7f2fe11d4700 1 -- 172.21.15.40:0/4047203478 >> 172.21.15.40:0/4047203478 conn(0x7f2fdc0fd4e0 msgr2=0x7f2fdc0ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.656+0000 7f2fe11d4700 1 -- 172.21.15.40:0/4047203478 shutdown_connections 2024-06-06T03:27:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.656+0000 7f2fe11d4700 1 -- 172.21.15.40:0/4047203478 wait complete. 2024-06-06T03:27:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.657+0000 7f2fe11d4700 1 Processor -- start 2024-06-06T03:27:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.657+0000 7f2fe11d4700 1 -- start start 2024-06-06T03:27:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.657+0000 7f2fe11d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fdc103560 0x7f2fdc197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.657+0000 7f2fe11d4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc108030 con 0x7f2fdc103560 2024-06-06T03:27:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.657+0000 7f2fdad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fdc103560 0x7f2fdc197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.657+0000 7f2fdad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fdc103560 0x7f2fdc197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36296/0 (socket says 172.21.15.40:36296) 2024-06-06T03:27:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.657+0000 7f2fdad9d700 1 -- 172.21.15.40:0/1489949532 learned_addr learned my addr 172.21.15.40:0/1489949532 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:27:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.658+0000 7f2fdad9d700 1 -- 172.21.15.40:0/1489949532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc197c50 con 0x7f2fdc103560 2024-06-06T03:27:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.658+0000 7f2fdad9d700 1 --2- 172.21.15.40:0/1489949532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fdc103560 0x7f2fdc197710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc004fa0 tx=0x7f2fcc005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.658+0000 7f2fc3fff700 1 -- 172.21.15.40:0/1489949532 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fcc027070 con 0x7f2fdc103560 2024-06-06T03:27:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.658+0000 7f2fe11d4700 1 -- 172.21.15.40:0/1489949532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fdc197ee0 con 0x7f2fdc103560 2024-06-06T03:27:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.658+0000 7f2fc3fff700 1 -- 172.21.15.40:0/1489949532 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fcc00be40 con 0x7f2fdc103560 2024-06-06T03:27:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.658+0000 7f2fc3fff700 1 -- 172.21.15.40:0/1489949532 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fcc00f460 con 0x7f2fdc103560 2024-06-06T03:27:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.658+0000 7f2fe11d4700 1 -- 172.21.15.40:0/1489949532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fdc19abd0 con 0x7f2fdc103560 2024-06-06T03:27:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.659+0000 7f2fc3fff700 1 -- 172.21.15.40:0/1489949532 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2fcc02a070 con 0x7f2fdc103560 2024-06-06T03:27:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.659+0000 7f2fe11d4700 1 -- 172.21.15.40:0/1489949532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fdc1087b0 con 0x7f2fdc103560 2024-06-06T03:27:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.659+0000 7f2fc3fff700 1 --2- 172.21.15.40:0/1489949532 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f2fc40390f0 0x7f2fc403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:27:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.659+0000 7f2fc3fff700 1 -- 172.21.15.40:0/1489949532 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2fcc01e030 con 0x7f2fdc103560 2024-06-06T03:27:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.659+0000 7f2fda59c700 1 --2- 172.21.15.40:0/1489949532 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f2fc40390f0 0x7f2fc403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:27:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.660+0000 7f2fda59c700 1 --2- 172.21.15.40:0/1489949532 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f2fc40390f0 0x7f2fc403b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0006fd0 tx=0x7f2fd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:27:59.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.664+0000 7f2fc3fff700 1 -- 172.21.15.40:0/1489949532 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2fcc013070 con 0x7f2fdc103560 2024-06-06T03:27:59.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.869+0000 7f2fe11d4700 1 -- 172.21.15.40:0/1489949532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f2fdc02dca0 con 0x7f2fdc103560 2024-06-06T03:27:59.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.871+0000 7f2fc3fff700 1 -- 172.21.15.40:0/1489949532 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f2fcc0292c0 con 0x7f2fdc103560 2024-06-06T03:27:59.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.873+0000 7f2fe11d4700 1 -- 172.21.15.40:0/1489949532 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f2fc40390f0 msgr2=0x7f2fc403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:59.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.873+0000 7f2fe11d4700 1 --2- 172.21.15.40:0/1489949532 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f2fc40390f0 0x7f2fc403b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0006fd0 tx=0x7f2fd0006e40 comp rx=0 tx=0).stop 2024-06-06T03:27:59.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.873+0000 7f2fe11d4700 1 -- 172.21.15.40:0/1489949532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fdc103560 msgr2=0x7f2fdc197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:27:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.873+0000 7f2fe11d4700 1 --2- 172.21.15.40:0/1489949532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fdc103560 0x7f2fdc197710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc004fa0 tx=0x7f2fcc005080 comp rx=0 tx=0).stop 2024-06-06T03:27:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.874+0000 7f2fe11d4700 1 -- 172.21.15.40:0/1489949532 shutdown_connections 2024-06-06T03:27:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.874+0000 7f2fe11d4700 1 --2- 172.21.15.40:0/1489949532 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f2fc40390f0 0x7f2fc403b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.874+0000 7f2fe11d4700 1 --2- 172.21.15.40:0/1489949532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fdc103560 0x7f2fdc197710 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:27:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.874+0000 7f2fe11d4700 1 -- 172.21.15.40:0/1489949532 >> 172.21.15.40:0/1489949532 conn(0x7f2fdc0fd4e0 msgr2=0x7f2fdc0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:27:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.874+0000 7f2fe11d4700 1 -- 172.21.15.40:0/1489949532 shutdown_connections 2024-06-06T03:27:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:27:59.874+0000 7f2fe11d4700 1 -- 172.21.15.40:0/1489949532 wait complete. 2024-06-06T03:28:00.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:00.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:00.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi040", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:28:00.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:00.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:28:00.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:00.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:00 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1489949532' entity='client.admin' 2024-06-06T03:28:00.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:28:00.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:00.706 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-06T03:28:00.707 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-06T03:28:00.959 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:28:02.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.931+0000 7f9cb281a700 1 -- 172.21.15.40:0/1178958553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac100df0 msgr2=0x7f9cac101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:02.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.931+0000 7f9cb281a700 1 --2- 172.21.15.40:0/1178958553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac100df0 0x7f9cac101210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009910 tx=0x7f9c9c009c20 comp rx=0 tx=0).stop 2024-06-06T03:28:02.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.931+0000 7f9cb281a700 1 -- 172.21.15.40:0/1178958553 shutdown_connections 2024-06-06T03:28:02.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.931+0000 7f9cb281a700 1 --2- 172.21.15.40:0/1178958553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac100df0 0x7f9cac101210 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:02.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.931+0000 7f9cb281a700 1 -- 172.21.15.40:0/1178958553 >> 172.21.15.40:0/1178958553 conn(0x7f9cac0fc390 msgr2=0x7f9cac0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:02.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.931+0000 7f9cb281a700 1 -- 172.21.15.40:0/1178958553 shutdown_connections 2024-06-06T03:28:02.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.931+0000 7f9cb281a700 1 -- 172.21.15.40:0/1178958553 wait complete. 2024-06-06T03:28:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.932+0000 7f9cb281a700 1 Processor -- start 2024-06-06T03:28:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.932+0000 7f9cb281a700 1 -- start start 2024-06-06T03:28:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.933+0000 7f9cb281a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac100df0 0x7f9cac06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.933+0000 7f9cb281a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac106e80 con 0x7f9cac100df0 2024-06-06T03:28:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.933+0000 7f9cabfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac100df0 0x7f9cac06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.933+0000 7f9cabfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac100df0 0x7f9cac06abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36312/0 (socket says 172.21.15.40:36312) 2024-06-06T03:28:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.933+0000 7f9cabfff700 1 -- 172.21.15.40:0/2061746074 learned_addr learned my addr 172.21.15.40:0/2061746074 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:28:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.933+0000 7f9cabfff700 1 -- 172.21.15.40:0/2061746074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac19a330 con 0x7f9cac100df0 2024-06-06T03:28:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.933+0000 7f9cabfff700 1 --2- 172.21.15.40:0/2061746074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac100df0 0x7f9cac06abe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c006010 tx=0x7f9c9c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.933+0000 7f9ca97fa700 1 -- 172.21.15.40:0/2061746074 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c9c027070 con 0x7f9cac100df0 2024-06-06T03:28:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.933+0000 7f9ca97fa700 1 -- 172.21.15.40:0/2061746074 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c9c00be40 con 0x7f9cac100df0 2024-06-06T03:28:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.933+0000 7f9cb281a700 1 -- 172.21.15.40:0/2061746074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cac19a500 con 0x7f9cac100df0 2024-06-06T03:28:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.933+0000 7f9ca97fa700 1 -- 172.21.15.40:0/2061746074 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c9c00f460 con 0x7f9cac100df0 2024-06-06T03:28:02.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.933+0000 7f9cb281a700 1 -- 172.21.15.40:0/2061746074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cac19a920 con 0x7f9cac100df0 2024-06-06T03:28:02.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.934+0000 7f9ca97fa700 1 -- 172.21.15.40:0/2061746074 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9c9c02a070 con 0x7f9cac100df0 2024-06-06T03:28:02.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.934+0000 7f9cb281a700 1 -- 172.21.15.40:0/2061746074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cac107600 con 0x7f9cac100df0 2024-06-06T03:28:02.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.935+0000 7f9ca97fa700 1 --2- 172.21.15.40:0/2061746074 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9c94039150 0x7f9c9403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.935+0000 7f9ca97fa700 1 -- 172.21.15.40:0/2061746074 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9c9c01e030 con 0x7f9cac100df0 2024-06-06T03:28:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.935+0000 7f9cab7fe700 1 --2- 172.21.15.40:0/2061746074 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9c94039150 0x7f9c9403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.935+0000 7f9cab7fe700 1 --2- 172.21.15.40:0/2061746074 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9c94039150 0x7f9c9403b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0006fd0 tx=0x7f9ca0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:02.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:02.940+0000 7f9ca97fa700 1 -- 172.21.15.40:0/2061746074 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9c9c013070 con 0x7f9cac100df0 2024-06-06T03:28:03.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:03.152+0000 7f9cb281a700 1 -- 172.21.15.40:0/2061746074 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f9cac02df60 con 0x7f9c94039150 2024-06-06T03:28:03.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:03.155+0000 7f9ca97fa700 1 -- 172.21.15.40:0/2061746074 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9cac02df60 con 0x7f9c94039150 2024-06-06T03:28:03.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:03.157+0000 7f9cb281a700 1 -- 172.21.15.40:0/2061746074 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9c94039150 msgr2=0x7f9c9403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:03.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:03.157+0000 7f9cb281a700 1 --2- 172.21.15.40:0/2061746074 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9c94039150 0x7f9c9403b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0006fd0 tx=0x7f9ca0006e40 comp rx=0 tx=0).stop 2024-06-06T03:28:03.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:03.157+0000 7f9cb281a700 1 -- 172.21.15.40:0/2061746074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac100df0 msgr2=0x7f9cac06abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:03.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:03.157+0000 7f9cb281a700 1 --2- 172.21.15.40:0/2061746074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac100df0 0x7f9cac06abe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c006010 tx=0x7f9c9c005050 comp rx=0 tx=0).stop 2024-06-06T03:28:03.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:03.157+0000 7f9cb281a700 1 -- 172.21.15.40:0/2061746074 shutdown_connections 2024-06-06T03:28:03.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:03.157+0000 7f9cb281a700 1 --2- 172.21.15.40:0/2061746074 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9c94039150 0x7f9c9403b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:03.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:03.157+0000 7f9cb281a700 1 --2- 172.21.15.40:0/2061746074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac100df0 0x7f9cac06abe0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:03.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:03.157+0000 7f9cb281a700 1 -- 172.21.15.40:0/2061746074 >> 172.21.15.40:0/2061746074 conn(0x7f9cac0fc390 msgr2=0x7f9cac0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:03.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:03.157+0000 7f9cb281a700 1 -- 172.21.15.40:0/2061746074 shutdown_connections 2024-06-06T03:28:03.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:03.158+0000 7f9cb281a700 1 -- 172.21.15.40:0/2061746074 wait complete. 2024-06-06T03:28:03.996 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi104 2024-06-06T03:28:03.996 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-06-06T03:28:03.996 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.conf 2024-06-06T03:28:04.015 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-06-06T03:28:04.015 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:28:04.072 INFO:tasks.cephadm:Adding host smithi104 to orchestrator... 2024-06-06T03:28:04.072 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch host add smithi104 2024-06-06T03:28:04.316 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:28:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:04 smithi040 ceph-mon[32220]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:28:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:04 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:04 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:28:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:04 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:04 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:28:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:04 smithi040 ceph-mon[32220]: Updating smithi040:/etc/ceph/ceph.conf 2024-06-06T03:28:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:04 smithi040 ceph-mon[32220]: Updating smithi040:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:05.524 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:05 smithi040 ceph-mon[32220]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:28:05.524 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:05 smithi040 ceph-mon[32220]: Updating smithi040:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T03:28:05.524 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:05.524 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:05.524 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:05.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.740+0000 7fadc6ec1700 1 -- 172.21.15.40:0/4238847049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc0101040 msgr2=0x7fadc0101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:05.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.740+0000 7fadc6ec1700 1 --2- 172.21.15.40:0/4238847049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc0101040 0x7fadc0101420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009910 tx=0x7fadb0009c20 comp rx=0 tx=0).stop 2024-06-06T03:28:05.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.740+0000 7fadc6ec1700 1 -- 172.21.15.40:0/4238847049 shutdown_connections 2024-06-06T03:28:05.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.740+0000 7fadc6ec1700 1 --2- 172.21.15.40:0/4238847049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc0101040 0x7fadc0101420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:05.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.740+0000 7fadc6ec1700 1 -- 172.21.15.40:0/4238847049 >> 172.21.15.40:0/4238847049 conn(0x7fadc00768f0 msgr2=0x7fadc0076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:05.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.740+0000 7fadc6ec1700 1 -- 172.21.15.40:0/4238847049 shutdown_connections 2024-06-06T03:28:05.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.740+0000 7fadc6ec1700 1 -- 172.21.15.40:0/4238847049 wait complete. 2024-06-06T03:28:05.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.741+0000 7fadc6ec1700 1 Processor -- start 2024-06-06T03:28:05.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.741+0000 7fadc6ec1700 1 -- start start 2024-06-06T03:28:05.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.741+0000 7fadc6ec1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc0101040 0x7fadc019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:05.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.741+0000 7fadc6ec1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fadc010b3c0 con 0x7fadc0101040 2024-06-06T03:28:05.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.741+0000 7fadc4c5d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc0101040 0x7fadc019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:05.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.741+0000 7fadc4c5d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc0101040 0x7fadc019d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36332/0 (socket says 172.21.15.40:36332) 2024-06-06T03:28:05.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.741+0000 7fadc4c5d700 1 -- 172.21.15.40:0/1095547781 learned_addr learned my addr 172.21.15.40:0/1095547781 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:28:05.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.741+0000 7fadc4c5d700 1 -- 172.21.15.40:0/1095547781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc019db90 con 0x7fadc0101040 2024-06-06T03:28:05.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.742+0000 7fadc4c5d700 1 --2- 172.21.15.40:0/1095547781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc0101040 0x7fadc019d650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fadb0005850 tx=0x7fadb0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:05.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.742+0000 7fadbdffb700 1 -- 172.21.15.40:0/1095547781 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadb0026070 con 0x7fadc0101040 2024-06-06T03:28:05.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.742+0000 7fadbdffb700 1 -- 172.21.15.40:0/1095547781 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb000bbd0 con 0x7fadc0101040 2024-06-06T03:28:05.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.742+0000 7fadc6ec1700 1 -- 172.21.15.40:0/1095547781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc019de80 con 0x7fadc0101040 2024-06-06T03:28:05.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.742+0000 7fadbdffb700 1 -- 172.21.15.40:0/1095547781 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadb0003970 con 0x7fadc0101040 2024-06-06T03:28:05.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.742+0000 7fadc6ec1700 1 -- 172.21.15.40:0/1095547781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc01977d0 con 0x7fadc0101040 2024-06-06T03:28:05.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.743+0000 7fadbdffb700 1 -- 172.21.15.40:0/1095547781 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fadb0029070 con 0x7fadc0101040 2024-06-06T03:28:05.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.743+0000 7fadc6ec1700 1 -- 172.21.15.40:0/1095547781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc01024d0 con 0x7fadc0101040 2024-06-06T03:28:05.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.743+0000 7fadbdffb700 1 --2- 172.21.15.40:0/1095547781 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fadac0390f0 0x7fadac03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:05.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.743+0000 7fadbdffb700 1 -- 172.21.15.40:0/1095547781 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fadb001e070 con 0x7fadc0101040 2024-06-06T03:28:05.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.743+0000 7fadbffff700 1 --2- 172.21.15.40:0/1095547781 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fadac0390f0 0x7fadac03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:05.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.744+0000 7fadbffff700 1 --2- 172.21.15.40:0/1095547781 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fadac0390f0 0x7fadac03b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fada8006fd0 tx=0x7fada8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:05.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.748+0000 7fadbdffb700 1 -- 172.21.15.40:0/1095547781 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fadb0013070 con 0x7fadc0101040 2024-06-06T03:28:05.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:05.957+0000 7fadc6ec1700 1 -- 172.21.15.40:0/1095547781 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}) -- 0x7fadc0198d00 con 0x7fadac0390f0 2024-06-06T03:28:06.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:06 smithi040 ceph-mon[32220]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:28:07.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:07 smithi040 ceph-mon[32220]: Deploying cephadm binary to smithi104 2024-06-06T03:28:08.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:08 smithi040 ceph-mon[32220]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:10.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:10.314+0000 7fadbdffb700 1 -- 172.21.15.40:0/1095547781 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fadc0198d00 con 0x7fadac0390f0 2024-06-06T03:28:10.315 INFO:teuthology.orchestra.run.smithi040.stdout:Added host 'smithi104' with addr '172.21.15.104' 2024-06-06T03:28:10.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:10.317+0000 7fadc6ec1700 1 -- 172.21.15.40:0/1095547781 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fadac0390f0 msgr2=0x7fadac03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:10.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:10.318+0000 7fadc6ec1700 1 --2- 172.21.15.40:0/1095547781 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fadac0390f0 0x7fadac03b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fada8006fd0 tx=0x7fada8006e40 comp rx=0 tx=0).stop 2024-06-06T03:28:10.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:10.318+0000 7fadc6ec1700 1 -- 172.21.15.40:0/1095547781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc0101040 msgr2=0x7fadc019d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:10.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:10.318+0000 7fadc6ec1700 1 --2- 172.21.15.40:0/1095547781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc0101040 0x7fadc019d650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fadb0005850 tx=0x7fadb0005030 comp rx=0 tx=0).stop 2024-06-06T03:28:10.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:10.318+0000 7fadc6ec1700 1 -- 172.21.15.40:0/1095547781 shutdown_connections 2024-06-06T03:28:10.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:10.318+0000 7fadc6ec1700 1 --2- 172.21.15.40:0/1095547781 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fadac0390f0 0x7fadac03b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:10.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:10.318+0000 7fadc6ec1700 1 --2- 172.21.15.40:0/1095547781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc0101040 0x7fadc019d650 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:10.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:10.318+0000 7fadc6ec1700 1 -- 172.21.15.40:0/1095547781 >> 172.21.15.40:0/1095547781 conn(0x7fadc00768f0 msgr2=0x7fadc00698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:10.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:10.318+0000 7fadc6ec1700 1 -- 172.21.15.40:0/1095547781 shutdown_connections 2024-06-06T03:28:10.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:10.318+0000 7fadc6ec1700 1 -- 172.21.15.40:0/1095547781 wait complete. 2024-06-06T03:28:10.931 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-06T03:28:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:28:11.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:11 smithi040 ceph-mon[32220]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:11.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:11 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:11.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:11 smithi040 ceph-mon[32220]: Added host smithi104 2024-06-06T03:28:11.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:11 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:28:11.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:11 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:12.628 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:12 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.651+0000 7fab940d0700 1 -- 172.21.15.40:0/2156870142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c103110 msgr2=0x7fab8c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.651+0000 7fab940d0700 1 --2- 172.21.15.40:0/2156870142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c103110 0x7fab8c1034f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fab88009910 tx=0x7fab88009c20 comp rx=0 tx=0).stop 2024-06-06T03:28:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.652+0000 7fab940d0700 1 -- 172.21.15.40:0/2156870142 shutdown_connections 2024-06-06T03:28:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.652+0000 7fab940d0700 1 --2- 172.21.15.40:0/2156870142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c103110 0x7fab8c1034f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.652+0000 7fab940d0700 1 -- 172.21.15.40:0/2156870142 >> 172.21.15.40:0/2156870142 conn(0x7fab8c0fc920 msgr2=0x7fab8c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.652+0000 7fab940d0700 1 -- 172.21.15.40:0/2156870142 shutdown_connections 2024-06-06T03:28:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.652+0000 7fab940d0700 1 -- 172.21.15.40:0/2156870142 wait complete. 2024-06-06T03:28:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.652+0000 7fab940d0700 1 Processor -- start 2024-06-06T03:28:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.652+0000 7fab940d0700 1 -- start start 2024-06-06T03:28:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.652+0000 7fab940d0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c103110 0x7fab8c19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.652+0000 7fab940d0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c10c570 con 0x7fab8c103110 2024-06-06T03:28:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.653+0000 7fab91e6c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c103110 0x7fab8c19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:12.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.653+0000 7fab91e6c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c103110 0x7fab8c19d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55552/0 (socket says 172.21.15.40:55552) 2024-06-06T03:28:12.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.653+0000 7fab91e6c700 1 -- 172.21.15.40:0/602932839 learned_addr learned my addr 172.21.15.40:0/602932839 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:28:12.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.653+0000 7fab91e6c700 1 -- 172.21.15.40:0/602932839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c19dbf0 con 0x7fab8c103110 2024-06-06T03:28:12.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.653+0000 7fab91e6c700 1 --2- 172.21.15.40:0/602932839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c103110 0x7fab8c19d6b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fab88006010 tx=0x7fab88005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:12.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.653+0000 7fab82ffd700 1 -- 172.21.15.40:0/602932839 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab88026070 con 0x7fab8c103110 2024-06-06T03:28:12.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.653+0000 7fab82ffd700 1 -- 172.21.15.40:0/602932839 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8800bc30 con 0x7fab8c103110 2024-06-06T03:28:12.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.653+0000 7fab940d0700 1 -- 172.21.15.40:0/602932839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c19de80 con 0x7fab8c103110 2024-06-06T03:28:12.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.653+0000 7fab82ffd700 1 -- 172.21.15.40:0/602932839 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab88003ab0 con 0x7fab8c103110 2024-06-06T03:28:12.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.653+0000 7fab940d0700 1 -- 172.21.15.40:0/602932839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c197870 con 0x7fab8c103110 2024-06-06T03:28:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.654+0000 7fab82ffd700 1 -- 172.21.15.40:0/602932839 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fab88029070 con 0x7fab8c103110 2024-06-06T03:28:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.654+0000 7fab940d0700 1 -- 172.21.15.40:0/602932839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8c06ab80 con 0x7fab8c103110 2024-06-06T03:28:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.654+0000 7fab82ffd700 1 --2- 172.21.15.40:0/602932839 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fab780390f0 0x7fab7803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.654+0000 7fab82ffd700 1 -- 172.21.15.40:0/602932839 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fab8801e070 con 0x7fab8c103110 2024-06-06T03:28:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.654+0000 7fab9166b700 1 --2- 172.21.15.40:0/602932839 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fab780390f0 0x7fab7803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.655+0000 7fab9166b700 1 --2- 172.21.15.40:0/602932839 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fab780390f0 0x7fab7803b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fab7c006fd0 tx=0x7fab7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.659+0000 7fab82ffd700 1 -- 172.21.15.40:0/602932839 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fab88013070 con 0x7fab8c103110 2024-06-06T03:28:12.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.872+0000 7fab940d0700 1 -- 172.21.15.40:0/602932839 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab8c198da0 con 0x7fab780390f0 2024-06-06T03:28:12.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.873+0000 7fab82ffd700 1 -- 172.21.15.40:0/602932839 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fab8c198da0 con 0x7fab780390f0 2024-06-06T03:28:12.873 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:28:12.873 INFO:teuthology.orchestra.run.smithi040.stdout:[{"addr": "172.21.15.40", "hostname": "smithi040", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}] 2024-06-06T03:28:12.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.876+0000 7fab940d0700 1 -- 172.21.15.40:0/602932839 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fab780390f0 msgr2=0x7fab7803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:12.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.876+0000 7fab940d0700 1 --2- 172.21.15.40:0/602932839 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fab780390f0 0x7fab7803b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fab7c006fd0 tx=0x7fab7c006e40 comp rx=0 tx=0).stop 2024-06-06T03:28:12.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.876+0000 7fab940d0700 1 -- 172.21.15.40:0/602932839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c103110 msgr2=0x7fab8c19d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:12.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.876+0000 7fab940d0700 1 --2- 172.21.15.40:0/602932839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c103110 0x7fab8c19d6b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fab88006010 tx=0x7fab88005050 comp rx=0 tx=0).stop 2024-06-06T03:28:12.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.876+0000 7fab940d0700 1 -- 172.21.15.40:0/602932839 shutdown_connections 2024-06-06T03:28:12.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.876+0000 7fab940d0700 1 --2- 172.21.15.40:0/602932839 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fab780390f0 0x7fab7803b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:12.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.876+0000 7fab940d0700 1 --2- 172.21.15.40:0/602932839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c103110 0x7fab8c19d6b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:12.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.876+0000 7fab940d0700 1 -- 172.21.15.40:0/602932839 >> 172.21.15.40:0/602932839 conn(0x7fab8c0fc920 msgr2=0x7fab8c103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:12.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.876+0000 7fab940d0700 1 -- 172.21.15.40:0/602932839 shutdown_connections 2024-06-06T03:28:12.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:12.876+0000 7fab940d0700 1 -- 172.21.15.40:0/602932839 wait complete. 2024-06-06T03:28:13.499 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:13 smithi040 ceph-mon[32220]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:13.499 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:13 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:13.534 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi131 2024-06-06T03:28:13.535 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-06-06T03:28:13.535 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/etc/ceph/ceph.conf 2024-06-06T03:28:13.553 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-06-06T03:28:13.553 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:28:13.611 INFO:tasks.cephadm:Adding host smithi131 to orchestrator... 2024-06-06T03:28:13.612 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch host add smithi131 2024-06-06T03:28:13.862 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:28:14.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:14 smithi040 ceph-mon[32220]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:28:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.325+0000 7f2a06721700 1 -- 172.21.15.40:0/681381873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00106ee0 msgr2=0x7f2a001072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.325+0000 7f2a06721700 1 --2- 172.21.15.40:0/681381873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00106ee0 0x7f2a001072c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f29f0009910 tx=0x7f29f0009c20 comp rx=0 tx=0).stop 2024-06-06T03:28:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.325+0000 7f2a06721700 1 -- 172.21.15.40:0/681381873 shutdown_connections 2024-06-06T03:28:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.325+0000 7f2a06721700 1 --2- 172.21.15.40:0/681381873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00106ee0 0x7f2a001072c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.325+0000 7f2a06721700 1 -- 172.21.15.40:0/681381873 >> 172.21.15.40:0/681381873 conn(0x7f2a000fc040 msgr2=0x7f2a000fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.325+0000 7f2a06721700 1 -- 172.21.15.40:0/681381873 shutdown_connections 2024-06-06T03:28:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.326+0000 7f2a06721700 1 -- 172.21.15.40:0/681381873 wait complete. 2024-06-06T03:28:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.326+0000 7f2a06721700 1 Processor -- start 2024-06-06T03:28:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.326+0000 7f2a06721700 1 -- start start 2024-06-06T03:28:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.326+0000 7f2a06721700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00106ee0 0x7f2a0019d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.326+0000 7f2a06721700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0010c640 con 0x7f2a00106ee0 2024-06-06T03:28:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.327+0000 7f29fffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00106ee0 0x7f2a0019d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:15.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.327+0000 7f29fffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00106ee0 0x7f2a0019d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55564/0 (socket says 172.21.15.40:55564) 2024-06-06T03:28:15.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.327+0000 7f29fffff700 1 -- 172.21.15.40:0/2474734997 learned_addr learned my addr 172.21.15.40:0/2474734997 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:28:15.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.327+0000 7f29fffff700 1 -- 172.21.15.40:0/2474734997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0019dc10 con 0x7f2a00106ee0 2024-06-06T03:28:15.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.327+0000 7f29fffff700 1 --2- 172.21.15.40:0/2474734997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00106ee0 0x7f2a0019d6d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f29f0005850 tx=0x7f29f0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:15.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.327+0000 7f29fd7fa700 1 -- 172.21.15.40:0/2474734997 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f0026070 con 0x7f2a00106ee0 2024-06-06T03:28:15.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.327+0000 7f2a06721700 1 -- 172.21.15.40:0/2474734997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0019dea0 con 0x7f2a00106ee0 2024-06-06T03:28:15.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.327+0000 7f29fd7fa700 1 -- 172.21.15.40:0/2474734997 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29f000bc30 con 0x7f2a00106ee0 2024-06-06T03:28:15.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.327+0000 7f29fd7fa700 1 -- 172.21.15.40:0/2474734997 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f0003a20 con 0x7f2a00106ee0 2024-06-06T03:28:15.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.327+0000 7f2a06721700 1 -- 172.21.15.40:0/2474734997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a00197850 con 0x7f2a00106ee0 2024-06-06T03:28:15.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.328+0000 7f29fd7fa700 1 -- 172.21.15.40:0/2474734997 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f29f0029070 con 0x7f2a00106ee0 2024-06-06T03:28:15.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.328+0000 7f2a06721700 1 -- 172.21.15.40:0/2474734997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a00103620 con 0x7f2a00106ee0 2024-06-06T03:28:15.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.329+0000 7f29fd7fa700 1 --2- 172.21.15.40:0/2474734997 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f29e80390f0 0x7f29e803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:15.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.329+0000 7f29fd7fa700 1 -- 172.21.15.40:0/2474734997 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f29f001e070 con 0x7f2a00106ee0 2024-06-06T03:28:15.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.329+0000 7f29ff7fe700 1 --2- 172.21.15.40:0/2474734997 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f29e80390f0 0x7f29e803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:15.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.329+0000 7f29ff7fe700 1 --2- 172.21.15.40:0/2474734997 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f29e80390f0 0x7f29e803b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f29f4006fd0 tx=0x7f29f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:15.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.334+0000 7f29fd7fa700 1 -- 172.21.15.40:0/2474734997 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f29f0013070 con 0x7f2a00106ee0 2024-06-06T03:28:15.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:15.546+0000 7f2a06721700 1 -- 172.21.15.40:0/2474734997 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi131", "target": ["mon-mgr", ""]}) -- 0x7f2a00063af0 con 0x7f29e80390f0 2024-06-06T03:28:15.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:15 smithi040 ceph-mon[32220]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:17 smithi040 ceph-mon[32220]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi131", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:28:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:17 smithi040 ceph-mon[32220]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:17 smithi040 ceph-mon[32220]: Deploying cephadm binary to smithi131 2024-06-06T03:28:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:28:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:28:17.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:17 smithi040 ceph-mon[32220]: Updating smithi104:/etc/ceph/ceph.conf 2024-06-06T03:28:18.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:18 smithi040 ceph-mon[32220]: Updating smithi104:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:19 smithi040 ceph-mon[32220]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:19.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:19.895+0000 7f29fd7fa700 1 -- 172.21.15.40:0/2474734997 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2a00063af0 con 0x7f29e80390f0 2024-06-06T03:28:19.895 INFO:teuthology.orchestra.run.smithi040.stdout:Added host 'smithi131' with addr '172.21.15.131' 2024-06-06T03:28:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:19.897+0000 7f2a06721700 1 -- 172.21.15.40:0/2474734997 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f29e80390f0 msgr2=0x7f29e803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:19.897+0000 7f2a06721700 1 --2- 172.21.15.40:0/2474734997 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f29e80390f0 0x7f29e803b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f29f4006fd0 tx=0x7f29f4006e40 comp rx=0 tx=0).stop 2024-06-06T03:28:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:19.897+0000 7f2a06721700 1 -- 172.21.15.40:0/2474734997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00106ee0 msgr2=0x7f2a0019d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:19.897+0000 7f2a06721700 1 --2- 172.21.15.40:0/2474734997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00106ee0 0x7f2a0019d6d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f29f0005850 tx=0x7f29f0005030 comp rx=0 tx=0).stop 2024-06-06T03:28:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:19.898+0000 7f2a06721700 1 -- 172.21.15.40:0/2474734997 shutdown_connections 2024-06-06T03:28:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:19.898+0000 7f2a06721700 1 --2- 172.21.15.40:0/2474734997 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f29e80390f0 0x7f29e803b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:19.898+0000 7f2a06721700 1 --2- 172.21.15.40:0/2474734997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00106ee0 0x7f2a0019d6d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:19.898+0000 7f2a06721700 1 -- 172.21.15.40:0/2474734997 >> 172.21.15.40:0/2474734997 conn(0x7f2a000fc040 msgr2=0x7f2a00109a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:19.898+0000 7f2a06721700 1 -- 172.21.15.40:0/2474734997 shutdown_connections 2024-06-06T03:28:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:19.898+0000 7f2a06721700 1 -- 172.21.15.40:0/2474734997 wait complete. 2024-06-06T03:28:20.427 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:20 smithi040 ceph-mon[32220]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:28:20.427 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:20.427 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:20.427 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:20.428 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:20.428 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:28:20.460 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-06T03:28:20.701 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:28:21.708 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:21 smithi040 ceph-mon[32220]: Updating smithi104:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T03:28:21.708 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:21 smithi040 ceph-mon[32220]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:21.708 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:21 smithi040 ceph-mon[32220]: Added host smithi131 2024-06-06T03:28:21.709 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:21 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:21.709 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:21 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:22.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.228+0000 7f927220a700 1 -- 172.21.15.40:0/4130341955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f926c101000 msgr2=0x7f926c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.228+0000 7f927220a700 1 --2- 172.21.15.40:0/4130341955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f926c101000 0x7f926c1013e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f925c009910 tx=0x7f925c009c20 comp rx=0 tx=0).stop 2024-06-06T03:28:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.228+0000 7f927220a700 1 -- 172.21.15.40:0/4130341955 shutdown_connections 2024-06-06T03:28:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.228+0000 7f927220a700 1 --2- 172.21.15.40:0/4130341955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f926c101000 0x7f926c1013e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.228+0000 7f927220a700 1 -- 172.21.15.40:0/4130341955 >> 172.21.15.40:0/4130341955 conn(0x7f926c0fc890 msgr2=0x7f926c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.228+0000 7f927220a700 1 -- 172.21.15.40:0/4130341955 shutdown_connections 2024-06-06T03:28:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.228+0000 7f927220a700 1 -- 172.21.15.40:0/4130341955 wait complete. 2024-06-06T03:28:22.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.229+0000 7f927220a700 1 Processor -- start 2024-06-06T03:28:22.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.229+0000 7f927220a700 1 -- start start 2024-06-06T03:28:22.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.229+0000 7f927220a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f926c101000 0x7f926c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:22.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.229+0000 7f927220a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f926c10b3d0 con 0x7f926c101000 2024-06-06T03:28:22.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.229+0000 7f926b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f926c101000 0x7f926c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:22.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.229+0000 7f926b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f926c101000 0x7f926c19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52578/0 (socket says 172.21.15.40:52578) 2024-06-06T03:28:22.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.229+0000 7f926b7fe700 1 -- 172.21.15.40:0/1323224816 learned_addr learned my addr 172.21.15.40:0/1323224816 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:28:22.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.230+0000 7f926b7fe700 1 -- 172.21.15.40:0/1323224816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926c19dbb0 con 0x7f926c101000 2024-06-06T03:28:22.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.230+0000 7f926b7fe700 1 --2- 172.21.15.40:0/1323224816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f926c101000 0x7f926c19d670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f925c005850 tx=0x7f925c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:22.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.230+0000 7f92697fa700 1 -- 172.21.15.40:0/1323224816 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f925c026070 con 0x7f926c101000 2024-06-06T03:28:22.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.230+0000 7f927220a700 1 -- 172.21.15.40:0/1323224816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926c19dea0 con 0x7f926c101000 2024-06-06T03:28:22.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.230+0000 7f92697fa700 1 -- 172.21.15.40:0/1323224816 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925c00bbd0 con 0x7f926c101000 2024-06-06T03:28:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.230+0000 7f92697fa700 1 -- 172.21.15.40:0/1323224816 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f925c003970 con 0x7f926c101000 2024-06-06T03:28:22.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.230+0000 7f927220a700 1 -- 172.21.15.40:0/1323224816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f926c1977f0 con 0x7f926c101000 2024-06-06T03:28:22.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.231+0000 7f927220a700 1 -- 172.21.15.40:0/1323224816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f926c1024e0 con 0x7f926c101000 2024-06-06T03:28:22.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.231+0000 7f92697fa700 1 -- 172.21.15.40:0/1323224816 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f925c029070 con 0x7f926c101000 2024-06-06T03:28:22.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.232+0000 7f92697fa700 1 --2- 172.21.15.40:0/1323224816 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f92580390f0 0x7f925803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:22.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.232+0000 7f92697fa700 1 -- 172.21.15.40:0/1323224816 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f925c01e070 con 0x7f926c101000 2024-06-06T03:28:22.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.232+0000 7f9262ffd700 1 --2- 172.21.15.40:0/1323224816 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f92580390f0 0x7f925803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:22.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.232+0000 7f9262ffd700 1 --2- 172.21.15.40:0/1323224816 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f92580390f0 0x7f925803b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9254006fd0 tx=0x7f9254006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:22.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.237+0000 7f92697fa700 1 -- 172.21.15.40:0/1323224816 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f925c013070 con 0x7f926c101000 2024-06-06T03:28:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.451+0000 7f927220a700 1 -- 172.21.15.40:0/1323224816 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f926c198d20 con 0x7f92580390f0 2024-06-06T03:28:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.452+0000 7f92697fa700 1 -- 172.21.15.40:0/1323224816 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f926c198d20 con 0x7f92580390f0 2024-06-06T03:28:22.452 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:28:22.452 INFO:teuthology.orchestra.run.smithi040.stdout:[{"addr": "172.21.15.40", "hostname": "smithi040", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}, {"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}] 2024-06-06T03:28:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.454+0000 7f927220a700 1 -- 172.21.15.40:0/1323224816 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f92580390f0 msgr2=0x7f925803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.454+0000 7f927220a700 1 --2- 172.21.15.40:0/1323224816 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f92580390f0 0x7f925803b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9254006fd0 tx=0x7f9254006e40 comp rx=0 tx=0).stop 2024-06-06T03:28:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.454+0000 7f927220a700 1 -- 172.21.15.40:0/1323224816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f926c101000 msgr2=0x7f926c19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.454+0000 7f927220a700 1 --2- 172.21.15.40:0/1323224816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f926c101000 0x7f926c19d670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f925c005850 tx=0x7f925c005030 comp rx=0 tx=0).stop 2024-06-06T03:28:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.454+0000 7f927220a700 1 -- 172.21.15.40:0/1323224816 shutdown_connections 2024-06-06T03:28:22.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.454+0000 7f927220a700 1 --2- 172.21.15.40:0/1323224816 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f92580390f0 0x7f925803b5b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:22.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.454+0000 7f927220a700 1 --2- 172.21.15.40:0/1323224816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f926c101000 0x7f926c19d670 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:22.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.454+0000 7f927220a700 1 -- 172.21.15.40:0/1323224816 >> 172.21.15.40:0/1323224816 conn(0x7f926c0fc890 msgr2=0x7f926c06a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:22.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.455+0000 7f927220a700 1 -- 172.21.15.40:0/1323224816 shutdown_connections 2024-06-06T03:28:22.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:22.455+0000 7f927220a700 1 -- 172.21.15.40:0/1323224816 wait complete. 2024-06-06T03:28:23.103 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-06T03:28:23.103 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-06T03:28:23.349 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:28:23.726 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:23 smithi040 ceph-mon[32220]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:23.726 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:24.698 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:24 smithi040 ceph-mon[32220]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:28:24.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.810+0000 7f1311ff8700 1 -- 172.21.15.40:0/3930945900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c101040 msgr2=0x7f130c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:24.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.810+0000 7f1311ff8700 1 --2- 172.21.15.40:0/3930945900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c101040 0x7f130c101420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009910 tx=0x7f12fc009c20 comp rx=0 tx=0).stop 2024-06-06T03:28:24.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.810+0000 7f1311ff8700 1 -- 172.21.15.40:0/3930945900 shutdown_connections 2024-06-06T03:28:24.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.810+0000 7f1311ff8700 1 --2- 172.21.15.40:0/3930945900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c101040 0x7f130c101420 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:24.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.810+0000 7f1311ff8700 1 -- 172.21.15.40:0/3930945900 >> 172.21.15.40:0/3930945900 conn(0x7f130c0768f0 msgr2=0x7f130c076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:24.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.810+0000 7f1311ff8700 1 -- 172.21.15.40:0/3930945900 shutdown_connections 2024-06-06T03:28:24.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.810+0000 7f1311ff8700 1 -- 172.21.15.40:0/3930945900 wait complete. 2024-06-06T03:28:24.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.811+0000 7f1311ff8700 1 Processor -- start 2024-06-06T03:28:24.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.811+0000 7f1311ff8700 1 -- start start 2024-06-06T03:28:24.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.811+0000 7f1311ff8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c101040 0x7f130c102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:24.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.811+0000 7f1311ff8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f130c10b3c0 con 0x7f130c101040 2024-06-06T03:28:24.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.811+0000 7f130b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c101040 0x7f130c102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:24.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.811+0000 7f130b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c101040 0x7f130c102a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52594/0 (socket says 172.21.15.40:52594) 2024-06-06T03:28:24.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.811+0000 7f130b7fe700 1 -- 172.21.15.40:0/549545987 learned_addr learned my addr 172.21.15.40:0/549545987 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:28:24.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.812+0000 7f130b7fe700 1 -- 172.21.15.40:0/549545987 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130c104980 con 0x7f130c101040 2024-06-06T03:28:24.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.812+0000 7f130b7fe700 1 --2- 172.21.15.40:0/549545987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c101040 0x7f130c102a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f12fc005850 tx=0x7f12fc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:24.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.812+0000 7f1308ff9700 1 -- 172.21.15.40:0/549545987 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12fc026070 con 0x7f130c101040 2024-06-06T03:28:24.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.812+0000 7f1308ff9700 1 -- 172.21.15.40:0/549545987 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12fc00bc30 con 0x7f130c101040 2024-06-06T03:28:24.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.812+0000 7f1311ff8700 1 -- 172.21.15.40:0/549545987 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130c103030 con 0x7f130c101040 2024-06-06T03:28:24.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.812+0000 7f1308ff9700 1 -- 172.21.15.40:0/549545987 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12fc003a20 con 0x7f130c101040 2024-06-06T03:28:24.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.812+0000 7f1311ff8700 1 -- 172.21.15.40:0/549545987 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f130c1034f0 con 0x7f130c101040 2024-06-06T03:28:24.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.813+0000 7f1308ff9700 1 -- 172.21.15.40:0/549545987 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f12fc029070 con 0x7f130c101040 2024-06-06T03:28:24.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.813+0000 7f1311ff8700 1 -- 172.21.15.40:0/549545987 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f130c1024d0 con 0x7f130c101040 2024-06-06T03:28:24.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.813+0000 7f1308ff9700 1 --2- 172.21.15.40:0/549545987 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f12f4039150 0x7f12f403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:24.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.813+0000 7f1308ff9700 1 -- 172.21.15.40:0/549545987 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f12fc01e070 con 0x7f130c101040 2024-06-06T03:28:24.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.813+0000 7f130affd700 1 --2- 172.21.15.40:0/549545987 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f12f4039150 0x7f12f403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:24.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.814+0000 7f130affd700 1 --2- 172.21.15.40:0/549545987 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f12f4039150 0x7f12f403b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1300006fd0 tx=0x7f1300006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:24.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:24.818+0000 7f1308ff9700 1 -- 172.21.15.40:0/549545987 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f12fc013070 con 0x7f130c101040 2024-06-06T03:28:25.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:25.029+0000 7f1311ff8700 1 -- 172.21.15.40:0/549545987 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f130c104060 con 0x7f130c101040 2024-06-06T03:28:25.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:25.442+0000 7f1308ff9700 1 -- 172.21.15.40:0/549545987 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f12fc030020 con 0x7f130c101040 2024-06-06T03:28:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:25.445+0000 7f1311ff8700 1 -- 172.21.15.40:0/549545987 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f12f4039150 msgr2=0x7f12f403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:25.445+0000 7f1311ff8700 1 --2- 172.21.15.40:0/549545987 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f12f4039150 0x7f12f403b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1300006fd0 tx=0x7f1300006e40 comp rx=0 tx=0).stop 2024-06-06T03:28:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:25.445+0000 7f1311ff8700 1 -- 172.21.15.40:0/549545987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c101040 msgr2=0x7f130c102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:25.445+0000 7f1311ff8700 1 --2- 172.21.15.40:0/549545987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c101040 0x7f130c102a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f12fc005850 tx=0x7f12fc005030 comp rx=0 tx=0).stop 2024-06-06T03:28:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:25.445+0000 7f1311ff8700 1 -- 172.21.15.40:0/549545987 shutdown_connections 2024-06-06T03:28:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:25.445+0000 7f1311ff8700 1 --2- 172.21.15.40:0/549545987 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f12f4039150 0x7f12f403b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:25.445+0000 7f1311ff8700 1 --2- 172.21.15.40:0/549545987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c101040 0x7f130c102a90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:25.445+0000 7f1311ff8700 1 -- 172.21.15.40:0/549545987 >> 172.21.15.40:0/549545987 conn(0x7f130c0768f0 msgr2=0x7f130c0698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:25.445+0000 7f1311ff8700 1 -- 172.21.15.40:0/549545987 shutdown_connections 2024-06-06T03:28:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:25.446+0000 7f1311ff8700 1 -- 172.21.15.40:0/549545987 wait complete. 2024-06-06T03:28:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2024-06-06T03:28:25.760 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:25 smithi040 ceph-mon[32220]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:25.761 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:25 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/549545987' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-06T03:28:26.084 INFO:tasks.cephadm:Adding mon.a on smithi040 2024-06-06T03:28:26.085 INFO:tasks.cephadm:Adding mon.b on smithi104 2024-06-06T03:28:26.085 INFO:tasks.cephadm:Adding mon.c on smithi131 2024-06-06T03:28:26.085 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi040:172.21.15.40=a;smithi104:172.21.15.104=b;smithi131:172.21.15.131=c' 2024-06-06T03:28:26.316 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:26.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:26 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/549545987' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-06T03:28:26.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:26 smithi040 ceph-mon[32220]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T03:28:26.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:26.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:26.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:26.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:26.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:28:26.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:26.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:28:27.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.772+0000 7f1ee415b700 1 -- 172.21.15.131:0/2381981714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1edc103060 msgr2=0x7f1edc103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.772+0000 7f1ee415b700 1 --2- 172.21.15.131:0/2381981714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1edc103060 0x7f1edc103440 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009910 tx=0x7f1ed8009c20 comp rx=0 tx=0).stop 2024-06-06T03:28:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.772+0000 7f1ee415b700 1 -- 172.21.15.131:0/2381981714 shutdown_connections 2024-06-06T03:28:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.772+0000 7f1ee415b700 1 --2- 172.21.15.131:0/2381981714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1edc103060 0x7f1edc103440 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.772+0000 7f1ee415b700 1 -- 172.21.15.131:0/2381981714 >> 172.21.15.131:0/2381981714 conn(0x7f1edc0fc8b0 msgr2=0x7f1edc0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.772+0000 7f1ee415b700 1 -- 172.21.15.131:0/2381981714 shutdown_connections 2024-06-06T03:28:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.772+0000 7f1ee415b700 1 -- 172.21.15.131:0/2381981714 wait complete. 2024-06-06T03:28:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.773+0000 7f1ee415b700 1 Processor -- start 2024-06-06T03:28:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.773+0000 7f1ee415b700 1 -- start start 2024-06-06T03:28:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.773+0000 7f1ee415b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1edc103060 0x7f1edc19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.773+0000 7f1ee415b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc10c550 con 0x7f1edc103060 2024-06-06T03:28:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.773+0000 7f1ee1ef7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1edc103060 0x7f1edc19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.773+0000 7f1ee1ef7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1edc103060 0x7f1edc19d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.131:40336/0 (socket says 172.21.15.131:40336) 2024-06-06T03:28:27.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.773+0000 7f1ee1ef7700 1 -- 172.21.15.131:0/435577382 learned_addr learned my addr 172.21.15.131:0/435577382 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-06-06T03:28:27.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.773+0000 7f1ee1ef7700 1 -- 172.21.15.131:0/435577382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1edc19dc00 con 0x7f1edc103060 2024-06-06T03:28:27.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.774+0000 7f1ee1ef7700 1 --2- 172.21.15.131:0/435577382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1edc103060 0x7f1edc19d6c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8006010 tx=0x7f1ed8005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:27.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.774+0000 7f1ed2ffd700 1 -- 172.21.15.131:0/435577382 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ed8026070 con 0x7f1edc103060 2024-06-06T03:28:27.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.774+0000 7f1ee415b700 1 -- 172.21.15.131:0/435577382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1edc19de90 con 0x7f1edc103060 2024-06-06T03:28:27.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.774+0000 7f1ed2ffd700 1 -- 172.21.15.131:0/435577382 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed800bc30 con 0x7f1edc103060 2024-06-06T03:28:27.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.774+0000 7f1ed2ffd700 1 -- 172.21.15.131:0/435577382 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ed8003ab0 con 0x7f1edc103060 2024-06-06T03:28:27.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.774+0000 7f1ee415b700 1 -- 172.21.15.131:0/435577382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1edc197840 con 0x7f1edc103060 2024-06-06T03:28:27.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.775+0000 7f1ee415b700 1 -- 172.21.15.131:0/435577382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1edc06ac20 con 0x7f1edc103060 2024-06-06T03:28:27.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.775+0000 7f1ed2ffd700 1 -- 172.21.15.131:0/435577382 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1ed8029070 con 0x7f1edc103060 2024-06-06T03:28:27.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.775+0000 7f1ed2ffd700 1 --2- 172.21.15.131:0/435577382 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1ec80390f0 0x7f1ec803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:27.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.775+0000 7f1ed2ffd700 1 -- 172.21.15.131:0/435577382 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1ed801e070 con 0x7f1edc103060 2024-06-06T03:28:27.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.775+0000 7f1ee16f6700 1 --2- 172.21.15.131:0/435577382 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1ec80390f0 0x7f1ec803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:27.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.776+0000 7f1ee16f6700 1 --2- 172.21.15.131:0/435577382 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1ec80390f0 0x7f1ec803b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc006fd0 tx=0x7f1ecc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:27.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.781+0000 7f1ed2ffd700 1 -- 172.21.15.131:0/435577382 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1ed8013070 con 0x7f1edc103060 2024-06-06T03:28:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:27 smithi040 ceph-mon[32220]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:27 smithi040 ceph-mon[32220]: Updating smithi131:/etc/ceph/ceph.conf 2024-06-06T03:28:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:27 smithi040 ceph-mon[32220]: Updating smithi131:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:27 smithi040 ceph-mon[32220]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:28:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:27 smithi040 ceph-mon[32220]: Updating smithi131:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T03:28:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:27 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:27 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:27.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:27 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:27.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.986+0000 7f1ee415b700 1 -- 172.21.15.131:0/435577382 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi040:172.21.15.40=a;smithi104:172.21.15.104=b;smithi131:172.21.15.131=c", "target": ["mon-mgr", ""]}) -- 0x7f1edc198d70 con 0x7f1ec80390f0 2024-06-06T03:28:27.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.993+0000 7f1ed2ffd700 1 -- 172.21.15.131:0/435577382 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1edc198d70 con 0x7f1ec80390f0 2024-06-06T03:28:27.993 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled mon update... 2024-06-06T03:28:27.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.995+0000 7f1ee415b700 1 -- 172.21.15.131:0/435577382 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1ec80390f0 msgr2=0x7f1ec803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:27.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.995+0000 7f1ee415b700 1 --2- 172.21.15.131:0/435577382 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1ec80390f0 0x7f1ec803b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc006fd0 tx=0x7f1ecc006e40 comp rx=0 tx=0).stop 2024-06-06T03:28:27.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.995+0000 7f1ee415b700 1 -- 172.21.15.131:0/435577382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1edc103060 msgr2=0x7f1edc19d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:27.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.995+0000 7f1ee415b700 1 --2- 172.21.15.131:0/435577382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1edc103060 0x7f1edc19d6c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8006010 tx=0x7f1ed8005050 comp rx=0 tx=0).stop 2024-06-06T03:28:27.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.995+0000 7f1ee415b700 1 -- 172.21.15.131:0/435577382 shutdown_connections 2024-06-06T03:28:27.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.995+0000 7f1ee415b700 1 --2- 172.21.15.131:0/435577382 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1ec80390f0 0x7f1ec803b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:27.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.995+0000 7f1ee415b700 1 --2- 172.21.15.131:0/435577382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1edc103060 0x7f1edc19d6c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:27.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.995+0000 7f1ee415b700 1 -- 172.21.15.131:0/435577382 >> 172.21.15.131:0/435577382 conn(0x7f1edc0fc8b0 msgr2=0x7f1edc103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:27.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.995+0000 7f1ee415b700 1 -- 172.21.15.131:0/435577382 shutdown_connections 2024-06-06T03:28:27.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:27.995+0000 7f1ee415b700 1 -- 172.21.15.131:0/435577382 wait complete. 2024-06-06T03:28:28.581 DEBUG:teuthology.orchestra.run.smithi104:mon.b> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mon.b.service 2024-06-06T03:28:28.584 DEBUG:teuthology.orchestra.run.smithi131:mon.c> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mon.c.service 2024-06-06T03:28:28.586 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T03:28:28.586 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T03:28:29.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:28 smithi040 ceph-mon[32220]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:29.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:28 smithi040 ceph-mon[32220]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi040:172.21.15.40=a;smithi104:172.21.15.104=b;smithi131:172.21.15.131=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:28:29.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:28 smithi040 ceph-mon[32220]: Saving service mon spec with placement smithi040:172.21.15.40=a;smithi104:172.21.15.104=b;smithi131:172.21.15.131=c;count:3 2024-06-06T03:28:29.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:28 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:29.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:28 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:28:29.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:28 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:29.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:28 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:28:29.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:28 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:29.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:28 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T03:28:29.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:28 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:29.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:28 smithi040 ceph-mon[32220]: Deploying daemon mon.c on smithi131 2024-06-06T03:28:29.681 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:31.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:30 smithi040 ceph-mon[32220]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:32.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.526+0000 7f4233363700 1 -- 172.21.15.131:0/2299412010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c101000 msgr2=0x7f422c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:32.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.526+0000 7f4233363700 1 --2- 172.21.15.131:0/2299412010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c101000 0x7f422c1013e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4228009910 tx=0x7f4228009c20 comp rx=0 tx=0).stop 2024-06-06T03:28:32.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.526+0000 7f4233363700 1 -- 172.21.15.131:0/2299412010 shutdown_connections 2024-06-06T03:28:32.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.526+0000 7f4233363700 1 --2- 172.21.15.131:0/2299412010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c101000 0x7f422c1013e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:32.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.526+0000 7f4233363700 1 -- 172.21.15.131:0/2299412010 >> 172.21.15.131:0/2299412010 conn(0x7f422c0fc890 msgr2=0x7f422c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:32.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.527+0000 7f4233363700 1 -- 172.21.15.131:0/2299412010 shutdown_connections 2024-06-06T03:28:32.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.527+0000 7f4233363700 1 -- 172.21.15.131:0/2299412010 wait complete. 2024-06-06T03:28:32.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.527+0000 7f4233363700 1 Processor -- start 2024-06-06T03:28:32.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.527+0000 7f4233363700 1 -- start start 2024-06-06T03:28:32.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.527+0000 7f4233363700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c101000 0x7f422c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:32.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.527+0000 7f4233363700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f422c10b3d0 con 0x7f422c101000 2024-06-06T03:28:32.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.528+0000 7f42310ff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c101000 0x7f422c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:32.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.528+0000 7f42310ff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c101000 0x7f422c198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.131:40366/0 (socket says 172.21.15.131:40366) 2024-06-06T03:28:32.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.528+0000 7f42310ff700 1 -- 172.21.15.131:0/2879988469 learned_addr learned my addr 172.21.15.131:0/2879988469 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-06-06T03:28:32.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.528+0000 7f42310ff700 1 -- 172.21.15.131:0/2879988469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422c198da0 con 0x7f422c101000 2024-06-06T03:28:32.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.528+0000 7f42310ff700 1 --2- 172.21.15.131:0/2879988469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c101000 0x7f422c198860 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4228005850 tx=0x7f4228005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:32.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.528+0000 7f42227fc700 1 -- 172.21.15.131:0/2879988469 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4228026070 con 0x7f422c101000 2024-06-06T03:28:32.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.528+0000 7f4233363700 1 -- 172.21.15.131:0/2879988469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422c199090 con 0x7f422c101000 2024-06-06T03:28:32.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.528+0000 7f42227fc700 1 -- 172.21.15.131:0/2879988469 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f422800bbd0 con 0x7f422c101000 2024-06-06T03:28:32.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.528+0000 7f42227fc700 1 -- 172.21.15.131:0/2879988469 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4228003970 con 0x7f422c101000 2024-06-06T03:28:32.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.528+0000 7f4233363700 1 -- 172.21.15.131:0/2879988469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f422c199550 con 0x7f422c101000 2024-06-06T03:28:32.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.529+0000 7f42227fc700 1 -- 172.21.15.131:0/2879988469 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4228029070 con 0x7f422c101000 2024-06-06T03:28:32.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.529+0000 7f4233363700 1 -- 172.21.15.131:0/2879988469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f422c1024e0 con 0x7f422c101000 2024-06-06T03:28:32.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.529+0000 7f42227fc700 1 --2- 172.21.15.131:0/2879988469 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42180390f0 0x7f421803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:32.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.530+0000 7f42227fc700 1 -- 172.21.15.131:0/2879988469 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f422801e070 con 0x7f422c101000 2024-06-06T03:28:32.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.530+0000 7f42308fe700 1 --2- 172.21.15.131:0/2879988469 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42180390f0 0x7f421803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:32.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.530+0000 7f42308fe700 1 --2- 172.21.15.131:0/2879988469 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42180390f0 0x7f421803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f421c006fd0 tx=0x7f421c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:32.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.535+0000 7f42227fc700 1 -- 172.21.15.131:0/2879988469 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4228013070 con 0x7f422c101000 2024-06-06T03:28:32.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.820+0000 7f4233363700 1 -- 172.21.15.131:0/2879988469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f422c063af0 con 0x7f422c101000 2024-06-06T03:28:32.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.821+0000 7f42227fc700 1 -- 172.21.15.131:0/2879988469 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f4228030020 con 0x7f422c101000 2024-06-06T03:28:32.821 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:28:32.822 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":1,"fsid":"77c25d68-23b4-11ef-bc9b-c7b262605968","modified":"2024-06-06T03:26:49.934047Z","created":"2024-06-06T03:26:49.934047Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-06T03:28:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.823+0000 7f4233363700 1 -- 172.21.15.131:0/2879988469 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42180390f0 msgr2=0x7f421803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.823+0000 7f4233363700 1 --2- 172.21.15.131:0/2879988469 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42180390f0 0x7f421803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f421c006fd0 tx=0x7f421c006e40 comp rx=0 tx=0).stop 2024-06-06T03:28:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.823+0000 7f4233363700 1 -- 172.21.15.131:0/2879988469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c101000 msgr2=0x7f422c198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.823+0000 7f4233363700 1 --2- 172.21.15.131:0/2879988469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c101000 0x7f422c198860 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4228005850 tx=0x7f4228005030 comp rx=0 tx=0).stop 2024-06-06T03:28:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.823+0000 7f4233363700 1 -- 172.21.15.131:0/2879988469 shutdown_connections 2024-06-06T03:28:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.823+0000 7f4233363700 1 --2- 172.21.15.131:0/2879988469 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42180390f0 0x7f421803b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.823+0000 7f4233363700 1 --2- 172.21.15.131:0/2879988469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c101000 0x7f422c198860 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.823+0000 7f4233363700 1 -- 172.21.15.131:0/2879988469 >> 172.21.15.131:0/2879988469 conn(0x7f422c0fc890 msgr2=0x7f422c103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.824+0000 7f4233363700 1 -- 172.21.15.131:0/2879988469 shutdown_connections 2024-06-06T03:28:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:32.824+0000 7f4233363700 1 -- 172.21.15.131:0/2879988469 wait complete. 2024-06-06T03:28:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:dumped monmap epoch 1 2024-06-06T03:28:32.880 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 podman[36633]: 2024-06-06 03:28:32.857824237 +0000 UTC m=+0.511094963 container start 39bf71dc49191517ecfd7711c7ff7df3a114b8c4f99584ffbff615bbf1819023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-06-06T03:28:32.880 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: set uid:gid to 167:167 (ceph:ceph) 2024-06-06T03:28:32.880 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-06T03:28:32.881 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: pidfile_write: ignore empty --pid-file 2024-06-06T03:28:33.264 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: load: jerasure load: lrc 2024-06-06T03:28:33.369 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: RocksDB version: 7.9.2 2024-06-06T03:28:33.370 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Git sha 0 2024-06-06T03:28:33.370 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-06T03:28:33.370 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: DB SUMMARY 2024-06-06T03:28:33.370 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: DB Session ID: OJ9Y5DWWB3A6FK4D0MEX 2024-06-06T03:28:33.370 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: CURRENT file: CURRENT 2024-06-06T03:28:33.370 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: IDENTITY file: IDENTITY 2024-06-06T03:28:33.370 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-06-06T03:28:33.370 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-06-06T03:28:33.371 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-06-06T03:28:33.371 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.error_if_exists: 0 2024-06-06T03:28:33.371 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.create_if_missing: 0 2024-06-06T03:28:33.371 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.paranoid_checks: 1 2024-06-06T03:28:33.371 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-06T03:28:33.371 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-06T03:28:33.371 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-06T03:28:33.371 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.env: 0x560e2f45cc60 2024-06-06T03:28:33.371 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.fs: PosixFileSystem 2024-06-06T03:28:33.372 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.info_log: 0x560e2feedb20 2024-06-06T03:28:33.372 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_file_opening_threads: 16 2024-06-06T03:28:33.372 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.statistics: (nil) 2024-06-06T03:28:33.372 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.use_fsync: 0 2024-06-06T03:28:33.372 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_log_file_size: 0 2024-06-06T03:28:33.372 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-06T03:28:33.372 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-06T03:28:33.372 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.keep_log_file_num: 1000 2024-06-06T03:28:33.372 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.recycle_log_file_num: 0 2024-06-06T03:28:33.373 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.allow_fallocate: 1 2024-06-06T03:28:33.373 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.allow_mmap_reads: 0 2024-06-06T03:28:33.373 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.allow_mmap_writes: 0 2024-06-06T03:28:33.373 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.use_direct_reads: 0 2024-06-06T03:28:33.373 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-06T03:28:33.373 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.create_missing_column_families: 0 2024-06-06T03:28:33.373 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.db_log_dir: 2024-06-06T03:28:33.373 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.wal_dir: 2024-06-06T03:28:33.374 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-06T03:28:33.374 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-06T03:28:33.374 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-06T03:28:33.374 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-06T03:28:33.377 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-06T03:28:33.377 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-06T03:28:33.377 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.advise_random_on_open: 1 2024-06-06T03:28:33.377 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.db_write_buffer_size: 0 2024-06-06T03:28:33.377 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.write_buffer_manager: 0x560e2febe1e0 2024-06-06T03:28:33.377 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-06T03:28:33.377 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-06T03:28:33.377 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-06T03:28:33.378 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.rate_limiter: (nil) 2024-06-06T03:28:33.378 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-06T03:28:33.378 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.wal_recovery_mode: 2 2024-06-06T03:28:33.378 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.enable_thread_tracking: 0 2024-06-06T03:28:33.378 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.enable_pipelined_write: 0 2024-06-06T03:28:33.378 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.unordered_write: 0 2024-06-06T03:28:33.378 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-06T03:28:33.378 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-06T03:28:33.378 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-06T03:28:33.378 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-06T03:28:33.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.row_cache: None 2024-06-06T03:28:33.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.wal_filter: None 2024-06-06T03:28:33.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-06T03:28:33.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.allow_ingest_behind: 0 2024-06-06T03:28:33.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.two_write_queues: 0 2024-06-06T03:28:33.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.manual_wal_flush: 0 2024-06-06T03:28:33.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.wal_compression: 0 2024-06-06T03:28:33.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.atomic_flush: 0 2024-06-06T03:28:33.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-06T03:28:33.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-06T03:28:33.380 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-06T03:28:33.380 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.log_readahead_size: 0 2024-06-06T03:28:33.380 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-06T03:28:33.380 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.best_efforts_recovery: 0 2024-06-06T03:28:33.380 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-06T03:28:33.380 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-06T03:28:33.380 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.allow_data_in_errors: 0 2024-06-06T03:28:33.380 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.db_host_id: __hostname__ 2024-06-06T03:28:33.380 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-06T03:28:33.380 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_background_jobs: 2 2024-06-06T03:28:33.381 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_background_compactions: -1 2024-06-06T03:28:33.381 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_subcompactions: 1 2024-06-06T03:28:33.381 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-06T03:28:33.381 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-06T03:28:33.381 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-06T03:28:33.381 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_total_wal_size: 0 2024-06-06T03:28:33.381 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-06T03:28:33.381 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-06T03:28:33.381 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-06T03:28:33.381 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-06T03:28:33.381 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_open_files: -1 2024-06-06T03:28:33.382 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.bytes_per_sync: 0 2024-06-06T03:28:33.382 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-06T03:28:33.382 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-06T03:28:33.382 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compaction_readahead_size: 0 2024-06-06T03:28:33.382 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_background_flushes: -1 2024-06-06T03:28:33.382 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Compression algorithms supported: 2024-06-06T03:28:33.382 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-06T03:28:33.382 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: kZSTD supported: 0 2024-06-06T03:28:33.382 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: kXpressCompression supported: 0 2024-06-06T03:28:33.382 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: kLZ4HCCompression supported: 1 2024-06-06T03:28:33.383 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: kZlibCompression supported: 1 2024-06-06T03:28:33.383 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: kSnappyCompression supported: 1 2024-06-06T03:28:33.383 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: kLZ4Compression supported: 1 2024-06-06T03:28:33.383 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: kBZip2Compression supported: 0 2024-06-06T03:28:33.383 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-06T03:28:33.383 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-06T03:28:33.383 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-06-06T03:28:33.383 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-06T03:28:33.383 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-06T03:28:33.384 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.merge_operator: 2024-06-06T03:28:33.384 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compaction_filter: None 2024-06-06T03:28:33.384 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compaction_filter_factory: None 2024-06-06T03:28:33.384 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.sst_partitioner_factory: None 2024-06-06T03:28:33.384 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-06T03:28:33.384 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-06T03:28:33.384 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560e2feedc40) 2024-06-06T03:28:33.384 INFO:journalctl@ceph.mon.c.smithi131.stdout: cache_index_and_filter_blocks: 1 2024-06-06T03:28:33.384 INFO:journalctl@ceph.mon.c.smithi131.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-06T03:28:33.384 INFO:journalctl@ceph.mon.c.smithi131.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-06T03:28:33.385 INFO:journalctl@ceph.mon.c.smithi131.stdout: pin_top_level_index_and_filter: 1 2024-06-06T03:28:33.385 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_type: 0 2024-06-06T03:28:33.385 INFO:journalctl@ceph.mon.c.smithi131.stdout: data_block_index_type: 0 2024-06-06T03:28:33.385 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_shortening: 1 2024-06-06T03:28:33.385 INFO:journalctl@ceph.mon.c.smithi131.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-06T03:28:33.385 INFO:journalctl@ceph.mon.c.smithi131.stdout: checksum: 4 2024-06-06T03:28:33.385 INFO:journalctl@ceph.mon.c.smithi131.stdout: no_block_cache: 0 2024-06-06T03:28:33.385 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache: 0x560e2feef090 2024-06-06T03:28:33.385 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_name: BinnedLRUCache 2024-06-06T03:28:33.385 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_options: 2024-06-06T03:28:33.386 INFO:journalctl@ceph.mon.c.smithi131.stdout: capacity : 536870912 2024-06-06T03:28:33.386 INFO:journalctl@ceph.mon.c.smithi131.stdout: num_shard_bits : 4 2024-06-06T03:28:33.386 INFO:journalctl@ceph.mon.c.smithi131.stdout: strict_capacity_limit : 0 2024-06-06T03:28:33.386 INFO:journalctl@ceph.mon.c.smithi131.stdout: high_pri_pool_ratio: 0.000 2024-06-06T03:28:33.386 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_compressed: (nil) 2024-06-06T03:28:33.386 INFO:journalctl@ceph.mon.c.smithi131.stdout: persistent_cache: (nil) 2024-06-06T03:28:33.386 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_size: 4096 2024-06-06T03:28:33.386 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_size_deviation: 10 2024-06-06T03:28:33.387 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_restart_interval: 16 2024-06-06T03:28:33.387 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_block_restart_interval: 1 2024-06-06T03:28:33.387 INFO:journalctl@ceph.mon.c.smithi131.stdout: metadata_block_size: 4096 2024-06-06T03:28:33.387 INFO:journalctl@ceph.mon.c.smithi131.stdout: partition_filters: 0 2024-06-06T03:28:33.387 INFO:journalctl@ceph.mon.c.smithi131.stdout: use_delta_encoding: 1 2024-06-06T03:28:33.387 INFO:journalctl@ceph.mon.c.smithi131.stdout: filter_policy: bloomfilter 2024-06-06T03:28:33.387 INFO:journalctl@ceph.mon.c.smithi131.stdout: whole_key_filtering: 1 2024-06-06T03:28:33.387 INFO:journalctl@ceph.mon.c.smithi131.stdout: verify_compression: 0 2024-06-06T03:28:33.387 INFO:journalctl@ceph.mon.c.smithi131.stdout: read_amp_bytes_per_bit: 0 2024-06-06T03:28:33.388 INFO:journalctl@ceph.mon.c.smithi131.stdout: format_version: 5 2024-06-06T03:28:33.388 INFO:journalctl@ceph.mon.c.smithi131.stdout: enable_index_compression: 1 2024-06-06T03:28:33.388 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_align: 0 2024-06-06T03:28:33.388 INFO:journalctl@ceph.mon.c.smithi131.stdout: max_auto_readahead_size: 262144 2024-06-06T03:28:33.388 INFO:journalctl@ceph.mon.c.smithi131.stdout: prepopulate_block_cache: 0 2024-06-06T03:28:33.388 INFO:journalctl@ceph.mon.c.smithi131.stdout: initial_auto_readahead_size: 8192 2024-06-06T03:28:33.388 INFO:journalctl@ceph.mon.c.smithi131.stdout: num_file_reads_for_auto_readahead: 2 2024-06-06T03:28:33.388 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.write_buffer_size: 33554432 2024-06-06T03:28:33.388 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_write_buffer_number: 2 2024-06-06T03:28:33.389 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compression: NoCompression 2024-06-06T03:28:33.389 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.bottommost_compression: Disabled 2024-06-06T03:28:33.389 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.prefix_extractor: nullptr 2024-06-06T03:28:33.389 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-06T03:28:33.389 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.num_levels: 7 2024-06-06T03:28:33.389 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-06T03:28:33.389 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-06T03:28:33.389 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-06T03:28:33.389 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-06T03:28:33.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-06T03:28:33.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-06T03:28:33.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-06T03:28:33.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-06T03:28:33.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-06T03:28:33.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-06T03:28:33.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-06T03:28:33.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-06T03:28:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-06T03:28:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compression_opts.level: 32767 2024-06-06T03:28:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compression_opts.strategy: 0 2024-06-06T03:28:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-06T03:28:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-06T03:28:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-06T03:28:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-06T03:28:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compression_opts.enabled: false 2024-06-06T03:28:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-06T03:28:33.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-06T03:28:33.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-06T03:28:33.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-06T03:28:33.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.target_file_size_base: 67108864 2024-06-06T03:28:33.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-06T03:28:33.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-06T03:28:33.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-06T03:28:33.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-06T03:28:33.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-06T03:28:33.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-06T03:28:33.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-06T03:28:33.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-06T03:28:33.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-06T03:28:33.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-06T03:28:33.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-06T03:28:33.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-06T03:28:33.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-06T03:28:33.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-06T03:28:33.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.arena_block_size: 1048576 2024-06-06T03:28:33.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-06T03:28:33.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-06T03:28:33.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.disable_auto_compactions: 0 2024-06-06T03:28:33.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-06T03:28:33.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-06T03:28:33.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-06T03:28:33.395 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-06T03:28:33.395 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-06T03:28:33.395 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-06T03:28:33.395 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-06T03:28:33.395 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-06T03:28:33.395 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-06T03:28:33.395 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-06T03:28:33.395 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.table_properties_collectors: 2024-06-06T03:28:33.395 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.inplace_update_support: 0 2024-06-06T03:28:33.396 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-06T03:28:33.396 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-06T03:28:33.396 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-06T03:28:33.396 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-06T03:28:33.396 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.bloom_locality: 0 2024-06-06T03:28:33.396 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.max_successive_merges: 0 2024-06-06T03:28:33.396 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-06T03:28:33.396 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.paranoid_file_checks: 0 2024-06-06T03:28:33.396 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.force_consistency_checks: 1 2024-06-06T03:28:33.396 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.report_bg_io_stats: 0 2024-06-06T03:28:33.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.ttl: 2592000 2024-06-06T03:28:33.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-06T03:28:33.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-06T03:28:33.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-06T03:28:33.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.enable_blob_files: false 2024-06-06T03:28:33.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.min_blob_size: 0 2024-06-06T03:28:33.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.blob_file_size: 268435456 2024-06-06T03:28:33.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-06T03:28:33.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-06T03:28:33.398 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-06T03:28:33.398 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-06T03:28:33.398 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-06T03:28:33.398 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.blob_file_starting_level: 0 2024-06-06T03:28:33.398 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-06T03:28:33.398 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-06T03:28:33.398 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-06-06T03:28:33.398 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: aefbcc94-8b7a-42ed-ae64-5de6d78d7271 2024-06-06T03:28:33.398 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717644512898606, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-06T03:28:33.399 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-06-06T03:28:33.399 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717644512899037, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717644512, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "aefbcc94-8b7a-42ed-ae64-5de6d78d7271", "db_session_id": "OJ9Y5DWWB3A6FK4D0MEX", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-06-06T03:28:33.399 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717644512899132, "job": 1, "event": "recovery_finished"} 2024-06-06T03:28:33.399 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-06-06T03:28:33.399 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-06T03:28:33.399 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560e2ffde000 2024-06-06T03:28:33.399 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: DB pointer 0x560e2ffc8000 2024-06-06T03:28:33.399 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-06T03:28:33.399 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-06T03:28:33.400 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** DB Stats ** 2024-06-06T03:28:33.400 INFO:journalctl@ceph.mon.c.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-06T03:28:33.400 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-06T03:28:33.400 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-06T03:28:33.400 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-06T03:28:33.400 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-06T03:28:33.400 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-06T03:28:33.400 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-06T03:28:33.400 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-06-06T03:28:33.400 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** Compaction Stats [default] ** 2024-06-06T03:28:33.400 INFO:journalctl@ceph.mon.c.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-06T03:28:33.400 INFO:journalctl@ceph.mon.c.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-06T03:28:33.401 INFO:journalctl@ceph.mon.c.smithi131.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-06T03:28:33.401 INFO:journalctl@ceph.mon.c.smithi131.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-06T03:28:33.401 INFO:journalctl@ceph.mon.c.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-06T03:28:33.401 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-06-06T03:28:33.401 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** Compaction Stats [default] ** 2024-06-06T03:28:33.401 INFO:journalctl@ceph.mon.c.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-06T03:28:33.401 INFO:journalctl@ceph.mon.c.smithi131.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-06T03:28:33.401 INFO:journalctl@ceph.mon.c.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-06T03:28:33.401 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-06-06T03:28:33.401 INFO:journalctl@ceph.mon.c.smithi131.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-06T03:28:33.402 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-06-06T03:28:33.402 INFO:journalctl@ceph.mon.c.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-06T03:28:33.402 INFO:journalctl@ceph.mon.c.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-06T03:28:33.402 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-06T03:28:33.402 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-06T03:28:33.402 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-06T03:28:33.402 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-06T03:28:33.402 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-06T03:28:33.402 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-06T03:28:33.403 INFO:journalctl@ceph.mon.c.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-06T03:28:33.403 INFO:journalctl@ceph.mon.c.smithi131.stdout: Block cache BinnedLRUCache@0x560e2feef090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-06T03:28:33.403 INFO:journalctl@ceph.mon.c.smithi131.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-06-06T03:28:33.403 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-06-06T03:28:33.403 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-06T03:28:33.403 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-06-06T03:28:33.403 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: using public_addr v2:172.21.15.131:0/0 -> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] 2024-06-06T03:28:33.403 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: starting mon.c rank -1 at public addrs [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] at bind addrs [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:28:33.403 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c@-1(???) e0 preinit fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:28:33.403 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c@-1(synchronizing).mds e1 new map 2024-06-06T03:28:33.404 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c@-1(synchronizing).mds e1 print_map 2024-06-06T03:28:33.404 INFO:journalctl@ceph.mon.c.smithi131.stdout: e1 2024-06-06T03:28:33.404 INFO:journalctl@ceph.mon.c.smithi131.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T03:28:33.404 INFO:journalctl@ceph.mon.c.smithi131.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T03:28:33.404 INFO:journalctl@ceph.mon.c.smithi131.stdout: legacy client fscid: -1 2024-06-06T03:28:33.404 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-06-06T03:28:33.404 INFO:journalctl@ceph.mon.c.smithi131.stdout: No filesystems configured 2024-06-06T03:28:33.404 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-06T03:28:33.404 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-06T03:28:33.405 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-06T03:28:33.405 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-06T03:28:33.405 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-06T03:28:33.405 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-06T03:28:33.405 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-06T03:28:33.405 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T03:28:33.405 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T03:28:33.405 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T03:28:33.405 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:28:33.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:33.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi131", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:28:33.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:33.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: Deploying cephadm binary to smithi131 2024-06-06T03:28:33.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.407 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.407 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.407 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:28:33.407 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:33.407 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:28:33.407 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: Updating smithi104:/etc/ceph/ceph.conf 2024-06-06T03:28:33.407 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: Updating smithi104:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:33.408 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:33.408 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:28:33.408 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.408 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.408 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.408 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.408 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:28:33.408 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: Updating smithi104:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T03:28:33.409 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:33.409 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: Added host smithi131 2024-06-06T03:28:33.409 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.409 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.409 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:33.409 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.409 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:28:33.409 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:33.410 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/549545987' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-06T03:28:33.410 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/549545987' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-06T03:28:33.410 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T03:28:33.410 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.410 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.410 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.410 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.410 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:28:33.411 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:33.411 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:28:33.411 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:33.411 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: Updating smithi131:/etc/ceph/ceph.conf 2024-06-06T03:28:33.411 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: Updating smithi131:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:33.411 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:28:33.411 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: Updating smithi131:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T03:28:33.411 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.411 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.412 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.412 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:33.412 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi040:172.21.15.40=a;smithi104:172.21.15.104=b;smithi131:172.21.15.131=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:28:33.412 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: Saving service mon spec with placement smithi040:172.21.15.40=a;smithi104:172.21.15.104=b;smithi131:172.21.15.131=c;count:3 2024-06-06T03:28:33.412 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.412 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:28:33.412 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:33.412 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:28:33.413 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:33.413 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T03:28:33.413 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:33.413 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: Deploying daemon mon.c on smithi131 2024-06-06T03:28:33.413 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:33.413 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:32 smithi131 ceph-mon[36687]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-06T03:28:33.413 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:33 smithi131 bash[36633]: 39bf71dc49191517ecfd7711c7ff7df3a114b8c4f99584ffbff615bbf1819023 2024-06-06T03:28:33.413 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:33 smithi131 systemd[1]: Started Ceph mon.c for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:28:34.635 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T03:28:34.635 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T03:28:34.866 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:28:38.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:28:38.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: mon.a calling monitor election 2024-06-06T03:28:38.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:38.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:38.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:38.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:38.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: mon.c calling monitor election 2024-06-06T03:28:38.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:38.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:38.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:38.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:38.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T03:28:38.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: monmap epoch 2 2024-06-06T03:28:38.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:28:38.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: last_changed 2024-06-06T03:28:32.922233+0000 2024-06-06T03:28:38.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: created 2024-06-06T03:26:49.934047+0000 2024-06-06T03:28:38.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: min_mon_release 19 (squid) 2024-06-06T03:28:38.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: election_strategy: 1 2024-06-06T03:28:38.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-06-06T03:28:38.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-06-06T03:28:38.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: fsmap 2024-06-06T03:28:38.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T03:28:38.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: mgrmap e13: x(active, since 50s) 2024-06-06T03:28:38.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: overall HEALTH_OK 2024-06-06T03:28:38.285 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:38.285 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:38.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:28:38.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: mon.a calling monitor election 2024-06-06T03:28:38.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:38.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:38.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:38.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:38.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: mon.c calling monitor election 2024-06-06T03:28:38.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:38.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:38.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:38.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:38.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T03:28:38.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: monmap epoch 2 2024-06-06T03:28:38.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:28:38.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: last_changed 2024-06-06T03:28:32.922233+0000 2024-06-06T03:28:38.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: created 2024-06-06T03:26:49.934047+0000 2024-06-06T03:28:38.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: min_mon_release 19 (squid) 2024-06-06T03:28:38.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: election_strategy: 1 2024-06-06T03:28:38.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-06-06T03:28:38.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-06-06T03:28:38.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: fsmap 2024-06-06T03:28:38.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T03:28:38.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: mgrmap e13: x(active, since 50s) 2024-06-06T03:28:38.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: overall HEALTH_OK 2024-06-06T03:28:38.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:38.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:39.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:39 smithi131 ceph-mon[36687]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:39.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:39 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:39.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:39 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T03:28:39.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:39 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:39.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:39 smithi131 ceph-mon[36687]: Deploying daemon mon.b on smithi104 2024-06-06T03:28:39.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:39 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:39.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:39 smithi040 ceph-mon[32220]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:39.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:39 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:39.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:39 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T03:28:39.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:39 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:39.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:39 smithi040 ceph-mon[32220]: Deploying daemon mon.b on smithi104 2024-06-06T03:28:39.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:39 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:39.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.434+0000 7f4bf0446700 1 -- 172.21.15.131:0/1048551485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bd00035c0 msgr2=0x7f4bd0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:39.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.434+0000 7f4bf0446700 1 --2- 172.21.15.131:0/1048551485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bd00035c0 0x7f4bd0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4be8100dd0 tx=0x7f4be0009e80 comp rx=0 tx=0).stop 2024-06-06T03:28:39.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.434+0000 7f4bf0446700 1 -- 172.21.15.131:0/1048551485 shutdown_connections 2024-06-06T03:28:39.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.434+0000 7f4bf0446700 1 --2- 172.21.15.131:0/1048551485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bd00035c0 0x7f4bd0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:39.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.434+0000 7f4bf0446700 1 -- 172.21.15.131:0/1048551485 >> 172.21.15.131:0/1048551485 conn(0x7f4be80fc990 msgr2=0x7f4be80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:39.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.434+0000 7f4bf0446700 1 -- 172.21.15.131:0/1048551485 shutdown_connections 2024-06-06T03:28:39.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.434+0000 7f4bf0446700 1 -- 172.21.15.131:0/1048551485 wait complete. 2024-06-06T03:28:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.435+0000 7f4bf0446700 1 Processor -- start 2024-06-06T03:28:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.435+0000 7f4bf0446700 1 -- start start 2024-06-06T03:28:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.435+0000 7f4bf0446700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd00035c0 0x7f4be8199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.435+0000 7f4bf0446700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4be819a2a0 0x7f4be819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.435+0000 7f4bf0446700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4be8107f00 con 0x7f4be819a2a0 2024-06-06T03:28:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.435+0000 7f4bf0446700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4be8107d80 con 0x7f4bd00035c0 2024-06-06T03:28:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.435+0000 7f4bee1e2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd00035c0 0x7f4be8199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.436+0000 7f4bee1e2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd00035c0 0x7f4be8199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59446/0 (socket says 172.21.15.131:59446) 2024-06-06T03:28:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.436+0000 7f4bee1e2700 1 -- 172.21.15.131:0/274606515 learned_addr learned my addr 172.21.15.131:0/274606515 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-06-06T03:28:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.436+0000 7f4bee1e2700 1 -- 172.21.15.131:0/274606515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd00035c0 msgr2=0x7f4be8199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-06T03:28:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.436+0000 7f4bee1e2700 1 -- 172.21.15.131:0/274606515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd00035c0 msgr2=0x7f4be8199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-06T03:28:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.436+0000 7f4bee1e2700 1 --2- 172.21.15.131:0/274606515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd00035c0 0x7f4be8199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-06T03:28:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.436+0000 7f4bee1e2700 1 --2- 172.21.15.131:0/274606515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd00035c0 0x7f4be8199d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T03:28:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.436+0000 7f4bed9e1700 1 --2- 172.21.15.131:0/274606515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4be819a2a0 0x7f4be819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.436+0000 7f4bed9e1700 1 -- 172.21.15.131:0/274606515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd00035c0 msgr2=0x7f4be8199d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:28:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.436+0000 7f4bed9e1700 1 --2- 172.21.15.131:0/274606515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd00035c0 0x7f4be8199d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.436+0000 7f4bed9e1700 1 -- 172.21.15.131:0/274606515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be819a8e0 con 0x7f4be819a2a0 2024-06-06T03:28:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.436+0000 7f4bed9e1700 1 --2- 172.21.15.131:0/274606515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4be819a2a0 0x7f4be819e630 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00b4d0 tx=0x7f4bdc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.436+0000 7f4be77fe700 1 -- 172.21.15.131:0/274606515 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4bdc011630 con 0x7f4be819a2a0 2024-06-06T03:28:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.437+0000 7f4be77fe700 1 -- 172.21.15.131:0/274606515 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bdc011cb0 con 0x7f4be819a2a0 2024-06-06T03:28:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.437+0000 7f4bf0446700 1 -- 172.21.15.131:0/274606515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be819eb70 con 0x7f4be819a2a0 2024-06-06T03:28:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.437+0000 7f4be77fe700 1 -- 172.21.15.131:0/274606515 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4bdc010e60 con 0x7f4be819a2a0 2024-06-06T03:28:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.437+0000 7f4bf0446700 1 -- 172.21.15.131:0/274606515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be819f0d0 con 0x7f4be819a2a0 2024-06-06T03:28:39.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.438+0000 7f4be77fe700 1 -- 172.21.15.131:0/274606515 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4bdc018020 con 0x7f4be819a2a0 2024-06-06T03:28:39.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.438+0000 7f4bf0446700 1 -- 172.21.15.131:0/274606515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be81025e0 con 0x7f4be819a2a0 2024-06-06T03:28:39.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.438+0000 7f4be77fe700 1 --2- 172.21.15.131:0/274606515 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4bd4038e30 0x7f4bd403b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:39.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.438+0000 7f4be77fe700 1 -- 172.21.15.131:0/274606515 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4bdc04bbf0 con 0x7f4be819a2a0 2024-06-06T03:28:39.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.438+0000 7f4bee1e2700 1 --2- 172.21.15.131:0/274606515 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4bd4038e30 0x7f4bd403b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:39.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.439+0000 7f4bee1e2700 1 --2- 172.21.15.131:0/274606515 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4bd4038e30 0x7f4bd403b2f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009160 tx=0x7f4be00090d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:39.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.443+0000 7f4be77fe700 1 -- 172.21.15.131:0/274606515 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4bdc01f020 con 0x7f4be819a2a0 2024-06-06T03:28:39.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.719+0000 7f4bf0446700 1 -- 172.21.15.131:0/274606515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4be819f3e0 con 0x7f4be819a2a0 2024-06-06T03:28:39.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.720+0000 7f4be77fe700 1 -- 172.21.15.131:0/274606515 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f4bdc00fe40 con 0x7f4be819a2a0 2024-06-06T03:28:39.720 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:28:39.721 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":2,"fsid":"77c25d68-23b4-11ef-bc9b-c7b262605968","modified":"2024-06-06T03:28:32.922233Z","created":"2024-06-06T03:26:49.934047Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-06T03:28:39.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.723+0000 7f4bf0446700 1 -- 172.21.15.131:0/274606515 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4bd4038e30 msgr2=0x7f4bd403b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:39.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.723+0000 7f4bf0446700 1 --2- 172.21.15.131:0/274606515 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4bd4038e30 0x7f4bd403b2f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009160 tx=0x7f4be00090d0 comp rx=0 tx=0).stop 2024-06-06T03:28:39.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.723+0000 7f4bf0446700 1 -- 172.21.15.131:0/274606515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4be819a2a0 msgr2=0x7f4be819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:39.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.723+0000 7f4bf0446700 1 --2- 172.21.15.131:0/274606515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4be819a2a0 0x7f4be819e630 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00b4d0 tx=0x7f4bdc00b890 comp rx=0 tx=0).stop 2024-06-06T03:28:39.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.723+0000 7f4bf0446700 1 -- 172.21.15.131:0/274606515 shutdown_connections 2024-06-06T03:28:39.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.723+0000 7f4bf0446700 1 --2- 172.21.15.131:0/274606515 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4bd4038e30 0x7f4bd403b2f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:39.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.723+0000 7f4bf0446700 1 --2- 172.21.15.131:0/274606515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bd00035c0 0x7f4be8199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:39.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.723+0000 7f4bf0446700 1 --2- 172.21.15.131:0/274606515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4be819a2a0 0x7f4be819e630 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:39.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.723+0000 7f4bf0446700 1 -- 172.21.15.131:0/274606515 >> 172.21.15.131:0/274606515 conn(0x7f4be80fc990 msgr2=0x7f4be80ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:39.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.723+0000 7f4bf0446700 1 -- 172.21.15.131:0/274606515 shutdown_connections 2024-06-06T03:28:39.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:39.723+0000 7f4bf0446700 1 -- 172.21.15.131:0/274606515 wait complete. 2024-06-06T03:28:39.724 INFO:teuthology.orchestra.run.smithi131.stderr:dumped monmap epoch 2 2024-06-06T03:28:40.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:40 smithi131 ceph-mon[36687]: from='client.? 172.21.15.131:0/274606515' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T03:28:40.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:40 smithi040 ceph-mon[32220]: from='client.? 172.21.15.131:0/274606515' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T03:28:40.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:28:39 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:28:39.921+0000 7f49f4f6f700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-06-06T03:28:41.248 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T03:28:41.249 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T03:28:41.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:41 smithi131 ceph-mon[36687]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:41.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:41 smithi040 ceph-mon[32220]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:41.491 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:28:41.627 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:41 smithi104 ceph-mon[36569]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-06T03:28:41.628 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:41 smithi104 bash[36550]: 7589498360b12698598a574efbd470a9eb193fac0346367d30d75d51cb9b6e94 2024-06-06T03:28:42.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:41 smithi104 systemd[1]: Started Ceph mon.b for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:28:47.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:47.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:28:47.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: mon.c calling monitor election 2024-06-06T03:28:47.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: mon.a calling monitor election 2024-06-06T03:28:47.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:47.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:47.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:47.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:47.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:47.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:47.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:47.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:47.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:47.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T03:28:47.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: monmap epoch 3 2024-06-06T03:28:47.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:28:47.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: last_changed 2024-06-06T03:28:41.617667+0000 2024-06-06T03:28:47.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: created 2024-06-06T03:26:49.934047+0000 2024-06-06T03:28:47.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: min_mon_release 19 (squid) 2024-06-06T03:28:47.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: election_strategy: 1 2024-06-06T03:28:47.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-06-06T03:28:47.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-06-06T03:28:47.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-06-06T03:28:47.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: fsmap 2024-06-06T03:28:47.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T03:28:47.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: mgrmap e13: x(active, since 59s) 2024-06-06T03:28:47.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: overall HEALTH_OK 2024-06-06T03:28:47.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:47.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:47.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:47.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:46 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:47.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:47.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:28:47.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: mon.c calling monitor election 2024-06-06T03:28:47.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: mon.a calling monitor election 2024-06-06T03:28:47.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:47.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:47.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:47.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:47.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:47.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:47.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:47.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:47.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:47.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T03:28:47.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: monmap epoch 3 2024-06-06T03:28:47.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:28:47.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: last_changed 2024-06-06T03:28:41.617667+0000 2024-06-06T03:28:47.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: created 2024-06-06T03:26:49.934047+0000 2024-06-06T03:28:47.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: min_mon_release 19 (squid) 2024-06-06T03:28:47.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: election_strategy: 1 2024-06-06T03:28:47.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-06-06T03:28:47.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-06-06T03:28:47.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-06-06T03:28:47.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: fsmap 2024-06-06T03:28:47.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T03:28:47.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: mgrmap e13: x(active, since 59s) 2024-06-06T03:28:47.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: overall HEALTH_OK 2024-06-06T03:28:47.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:47.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:47.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:47.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:46 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:47.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.332+0000 7f3b90d42700 1 -- 172.21.15.131:0/980137537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b780057d0 msgr2=0x7f3b78007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:47.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.332+0000 7f3b90d42700 1 --2- 172.21.15.131:0/980137537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b780057d0 0x7f3b78007c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c100db0 tx=0x7f3b7c009e80 comp rx=0 tx=0).stop 2024-06-06T03:28:47.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.332+0000 7f3b90d42700 1 -- 172.21.15.131:0/980137537 shutdown_connections 2024-06-06T03:28:47.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.332+0000 7f3b90d42700 1 --2- 172.21.15.131:0/980137537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b780057d0 0x7f3b78007c60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:47.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.332+0000 7f3b90d42700 1 -- 172.21.15.131:0/980137537 >> 172.21.15.131:0/980137537 conn(0x7f3b8c0fc970 msgr2=0x7f3b8c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:47.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.332+0000 7f3b90d42700 1 -- 172.21.15.131:0/980137537 shutdown_connections 2024-06-06T03:28:47.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.332+0000 7f3b90d42700 1 -- 172.21.15.131:0/980137537 wait complete. 2024-06-06T03:28:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b90d42700 1 Processor -- start 2024-06-06T03:28:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b90d42700 1 -- start start 2024-06-06T03:28:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b90d42700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b780057d0 0x7f3b8c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b90d42700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b8c19a2f0 0x7f3b8c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b90d42700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c19ebc0 0x7f3b8c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b90d42700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c107ee0 con 0x7f3b8c19a2f0 2024-06-06T03:28:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b90d42700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c107d60 con 0x7f3b780057d0 2024-06-06T03:28:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b90d42700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c108060 con 0x7f3b8c19ebc0 2024-06-06T03:28:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b8ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c19ebc0 0x7f3b8c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b8ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c19ebc0 0x7f3b8c19f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59360/0 (socket says 172.21.15.131:59360) 2024-06-06T03:28:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b8ad9d700 1 -- 172.21.15.131:0/2423806303 learned_addr learned my addr 172.21.15.131:0/2423806303 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-06-06T03:28:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b8a59c700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b780057d0 0x7f3b8c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b8ad9d700 1 -- 172.21.15.131:0/2423806303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c19ebc0 msgr2=0x7f3b8c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-06-06T03:28:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b8ad9d700 1 -- 172.21.15.131:0/2423806303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c19ebc0 msgr2=0x7f3b8c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-06T03:28:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b8ad9d700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c19ebc0 0x7f3b8c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-06T03:28:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b8ad9d700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c19ebc0 0x7f3b8c19f070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T03:28:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.333+0000 7f3b81d9b700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b8c19a2f0 0x7f3b8c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b8a59c700 1 -- 172.21.15.131:0/2423806303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b780057d0 msgr2=0x7f3b8c199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-06T03:28:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b8a59c700 1 -- 172.21.15.131:0/2423806303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b780057d0 msgr2=0x7f3b8c199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-06T03:28:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b8a59c700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b780057d0 0x7f3b8c199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-06T03:28:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b8a59c700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b780057d0 0x7f3b8c199db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T03:28:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b81d9b700 1 -- 172.21.15.131:0/2423806303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c19ebc0 msgr2=0x7f3b8c19f070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:28:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b81d9b700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c19ebc0 0x7f3b8c19f070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b81d9b700 1 -- 172.21.15.131:0/2423806303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b780057d0 msgr2=0x7f3b8c199db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:28:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b81d9b700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b780057d0 0x7f3b8c199db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b81d9b700 1 -- 172.21.15.131:0/2423806303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8c1a6930 con 0x7f3b8c19a2f0 2024-06-06T03:28:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b81d9b700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b8c19a2f0 0x7f3b8c19e680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400b550 tx=0x7f3b7400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b83fff700 1 -- 172.21.15.131:0/2423806303 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74004230 con 0x7f3b8c19a2f0 2024-06-06T03:28:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b83fff700 1 -- 172.21.15.131:0/2423806303 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b740043d0 con 0x7f3b8c19a2f0 2024-06-06T03:28:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b90d42700 1 -- 172.21.15.131:0/2423806303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8c1a6b60 con 0x7f3b8c19a2f0 2024-06-06T03:28:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b83fff700 1 -- 172.21.15.131:0/2423806303 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74011c20 con 0x7f3b8c19a2f0 2024-06-06T03:28:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.334+0000 7f3b90d42700 1 -- 172.21.15.131:0/2423806303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8c1a70a0 con 0x7f3b8c19a2f0 2024-06-06T03:28:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.335+0000 7f3b83fff700 1 -- 172.21.15.131:0/2423806303 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3b7401a400 con 0x7f3b8c19a2f0 2024-06-06T03:28:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.335+0000 7f3b83fff700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f3b6c039580 0x7f3b6c03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.335+0000 7f3b90d42700 1 -- 172.21.15.131:0/2423806303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b8c063af0 con 0x7f3b8c19a2f0 2024-06-06T03:28:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.336+0000 7f3b83fff700 1 -- 172.21.15.131:0/2423806303 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3b7404bdd0 con 0x7f3b8c19a2f0 2024-06-06T03:28:47.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.336+0000 7f3b8a59c700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f3b6c039580 0x7f3b6c03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:47.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.336+0000 7f3b8a59c700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f3b6c039580 0x7f3b6c03ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009a30 tx=0x7f3b7c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:47.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.341+0000 7f3b83fff700 1 -- 172.21.15.131:0/2423806303 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3b740151c0 con 0x7f3b8c19a2f0 2024-06-06T03:28:47.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.637+0000 7f3b90d42700 1 -- 172.21.15.131:0/2423806303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3b8c02de60 con 0x7f3b8c19a2f0 2024-06-06T03:28:47.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.638+0000 7f3b83fff700 1 -- 172.21.15.131:0/2423806303 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f3b74003c80 con 0x7f3b8c19a2f0 2024-06-06T03:28:47.637 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-06-06T03:28:47.637 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":3,"fsid":"77c25d68-23b4-11ef-bc9b-c7b262605968","modified":"2024-06-06T03:28:41.617667Z","created":"2024-06-06T03:26:49.934047Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:3300","nonce":0},{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-06T03:28:47.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.640+0000 7f3b90d42700 1 -- 172.21.15.131:0/2423806303 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f3b6c039580 msgr2=0x7f3b6c03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:47.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.640+0000 7f3b90d42700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f3b6c039580 0x7f3b6c03ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009a30 tx=0x7f3b7c00b040 comp rx=0 tx=0).stop 2024-06-06T03:28:47.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.640+0000 7f3b90d42700 1 -- 172.21.15.131:0/2423806303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b8c19a2f0 msgr2=0x7f3b8c19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:47.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.640+0000 7f3b90d42700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b8c19a2f0 0x7f3b8c19e680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400b550 tx=0x7f3b7400b910 comp rx=0 tx=0).stop 2024-06-06T03:28:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.640+0000 7f3b90d42700 1 -- 172.21.15.131:0/2423806303 shutdown_connections 2024-06-06T03:28:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.640+0000 7f3b90d42700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f3b6c039580 0x7f3b6c03ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.640+0000 7f3b90d42700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c19ebc0 0x7f3b8c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.640+0000 7f3b90d42700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b780057d0 0x7f3b8c199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.640+0000 7f3b90d42700 1 --2- 172.21.15.131:0/2423806303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b8c19a2f0 0x7f3b8c19e680 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.640+0000 7f3b90d42700 1 -- 172.21.15.131:0/2423806303 >> 172.21.15.131:0/2423806303 conn(0x7f3b8c0fc970 msgr2=0x7f3b8c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.640+0000 7f3b90d42700 1 -- 172.21.15.131:0/2423806303 shutdown_connections 2024-06-06T03:28:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:47.640+0000 7f3b90d42700 1 -- 172.21.15.131:0/2423806303 wait complete. 2024-06-06T03:28:47.640 INFO:teuthology.orchestra.run.smithi131.stderr:dumped monmap epoch 3 2024-06-06T03:28:48.025 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:47 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:28:48.026 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:47 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:48.026 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:47 smithi131 ceph-mon[36687]: from='client.? 172.21.15.131:0/2423806303' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T03:28:48.026 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:47 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:48.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:28:48.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:48.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:47 smithi040 ceph-mon[32220]: from='client.? 172.21.15.131:0/2423806303' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T03:28:48.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:48.281 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-06T03:28:48.281 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-06T03:28:48.539 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:28:49.017 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:49.017 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:28:49.017 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: mon.c calling monitor election 2024-06-06T03:28:49.017 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: mon.a calling monitor election 2024-06-06T03:28:49.017 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:49.017 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:28:49.018 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:49.018 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:49.018 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:49.018 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:49.018 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:49.018 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:49.018 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:49.018 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T03:28:49.018 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: monmap epoch 3 2024-06-06T03:28:49.018 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:28:49.018 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: last_changed 2024-06-06T03:28:41.617667+0000 2024-06-06T03:28:49.018 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: created 2024-06-06T03:26:49.934047+0000 2024-06-06T03:28:49.018 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: min_mon_release 19 (squid) 2024-06-06T03:28:49.019 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: election_strategy: 1 2024-06-06T03:28:49.019 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-06-06T03:28:49.019 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-06-06T03:28:49.019 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-06-06T03:28:49.019 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: fsmap 2024-06-06T03:28:49.019 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T03:28:49.019 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: mgrmap e13: x(active, since 59s) 2024-06-06T03:28:49.019 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: overall HEALTH_OK 2024-06-06T03:28:49.019 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:49.019 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:49.019 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:49.019 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:49.019 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:28:49.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:49.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='client.? 172.21.15.131:0/2423806303' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T03:28:49.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:49.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: mon.b calling monitor election 2024-06-06T03:28:49.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: mon.b calling monitor election 2024-06-06T03:28:49.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: mon.c calling monitor election 2024-06-06T03:28:49.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: mon.a calling monitor election 2024-06-06T03:28:49.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T03:28:49.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: monmap epoch 3 2024-06-06T03:28:49.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:28:49.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: last_changed 2024-06-06T03:28:41.617667+0000 2024-06-06T03:28:49.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: created 2024-06-06T03:26:49.934047+0000 2024-06-06T03:28:49.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: min_mon_release 19 (squid) 2024-06-06T03:28:49.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: election_strategy: 1 2024-06-06T03:28:49.021 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-06-06T03:28:49.021 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-06-06T03:28:49.021 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-06-06T03:28:49.021 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: fsmap 2024-06-06T03:28:49.021 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T03:28:49.021 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: mgrmap e13: x(active, since 61s) 2024-06-06T03:28:49.021 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: overall HEALTH_OK 2024-06-06T03:28:49.021 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:49.021 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:49.021 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:49.021 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:28:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: mon.b calling monitor election 2024-06-06T03:28:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: mon.b calling monitor election 2024-06-06T03:28:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: mon.c calling monitor election 2024-06-06T03:28:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: mon.a calling monitor election 2024-06-06T03:28:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T03:28:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: monmap epoch 3 2024-06-06T03:28:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:28:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: last_changed 2024-06-06T03:28:41.617667+0000 2024-06-06T03:28:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: created 2024-06-06T03:26:49.934047+0000 2024-06-06T03:28:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: min_mon_release 19 (squid) 2024-06-06T03:28:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: election_strategy: 1 2024-06-06T03:28:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-06-06T03:28:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-06-06T03:28:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-06-06T03:28:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: fsmap 2024-06-06T03:28:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T03:28:49.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: mgrmap e13: x(active, since 61s) 2024-06-06T03:28:49.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: overall HEALTH_OK 2024-06-06T03:28:49.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:49.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:49.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:49.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:28:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: mon.b calling monitor election 2024-06-06T03:28:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: mon.b calling monitor election 2024-06-06T03:28:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: mon.c calling monitor election 2024-06-06T03:28:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: mon.a calling monitor election 2024-06-06T03:28:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T03:28:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: monmap epoch 3 2024-06-06T03:28:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:28:49.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: last_changed 2024-06-06T03:28:41.617667+0000 2024-06-06T03:28:49.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: created 2024-06-06T03:26:49.934047+0000 2024-06-06T03:28:49.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: min_mon_release 19 (squid) 2024-06-06T03:28:49.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: election_strategy: 1 2024-06-06T03:28:49.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: 0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-06-06T03:28:49.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-06-06T03:28:49.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-06-06T03:28:49.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: fsmap 2024-06-06T03:28:49.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T03:28:49.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: mgrmap e13: x(active, since 61s) 2024-06-06T03:28:49.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: overall HEALTH_OK 2024-06-06T03:28:49.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:49.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:49.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:49.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:28:50.260 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: Updating smithi040:/etc/ceph/ceph.conf 2024-06-06T03:28:50.260 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: Updating smithi104:/etc/ceph/ceph.conf 2024-06-06T03:28:50.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: Updating smithi131:/etc/ceph/ceph.conf 2024-06-06T03:28:50.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: Updating smithi104:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:50.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: Updating smithi040:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:50.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: Updating smithi131:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:50.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.262 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.262 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T03:28:50.262 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T03:28:50.262 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T03:28:50.262 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:50.262 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: Reconfiguring daemon mon.a on smithi040 2024-06-06T03:28:50.262 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:50.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: Updating smithi040:/etc/ceph/ceph.conf 2024-06-06T03:28:50.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: Updating smithi104:/etc/ceph/ceph.conf 2024-06-06T03:28:50.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: Updating smithi131:/etc/ceph/ceph.conf 2024-06-06T03:28:50.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: Updating smithi104:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:50.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: Updating smithi040:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:50.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: Updating smithi131:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:50.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T03:28:50.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T03:28:50.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T03:28:50.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:50.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: Reconfiguring daemon mon.a on smithi040 2024-06-06T03:28:50.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:50.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.488+0000 7fe8ff624700 1 -- 172.21.15.40:0/3876710512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f81065e0 msgr2=0x7fe8f81069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:50.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.488+0000 7fe8ff624700 1 --2- 172.21.15.40:0/3876710512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f81065e0 0x7fe8f81069c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009910 tx=0x7fe8f4009c20 comp rx=0 tx=0).stop 2024-06-06T03:28:50.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.488+0000 7fe8ff624700 1 -- 172.21.15.40:0/3876710512 shutdown_connections 2024-06-06T03:28:50.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.488+0000 7fe8ff624700 1 --2- 172.21.15.40:0/3876710512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f81065e0 0x7fe8f81069c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:50.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.488+0000 7fe8ff624700 1 -- 172.21.15.40:0/3876710512 >> 172.21.15.40:0/3876710512 conn(0x7fe8f8077510 msgr2=0x7fe8f8077920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:50.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.488+0000 7fe8ff624700 1 -- 172.21.15.40:0/3876710512 shutdown_connections 2024-06-06T03:28:50.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.489+0000 7fe8ff624700 1 -- 172.21.15.40:0/3876710512 wait complete. 2024-06-06T03:28:50.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.490+0000 7fe8ff624700 1 Processor -- start 2024-06-06T03:28:50.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.490+0000 7fe8ff624700 1 -- start start 2024-06-06T03:28:50.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.490+0000 7fe8ff624700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f81065e0 0x7fe8f8198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:50.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.490+0000 7fe8ff624700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f8199050 0x7fe8f81994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:50.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.490+0000 7fe8ff624700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8f81a1b80 0x7fe8f81a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:50.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.490+0000 7fe8ff624700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f810ab50 con 0x7fe8f81065e0 2024-06-06T03:28:50.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.490+0000 7fe8ff624700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f810a9d0 con 0x7fe8f81a1b80 2024-06-06T03:28:50.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.490+0000 7fe8ff624700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f810acd0 con 0x7fe8f8199050 2024-06-06T03:28:50.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.490+0000 7fe8fd3c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f81065e0 0x7fe8f8198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:50.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.490+0000 7fe8fd3c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f81065e0 0x7fe8f8198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60448/0 (socket says 172.21.15.40:60448) 2024-06-06T03:28:50.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.490+0000 7fe8fd3c0700 1 -- 172.21.15.40:0/3122894213 learned_addr learned my addr 172.21.15.40:0/3122894213 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:28:50.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.491+0000 7fe8fdbc1700 1 --2- 172.21.15.40:0/3122894213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8f81a1b80 0x7fe8f81a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:50.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.491+0000 7fe8fcbbf700 1 --2- 172.21.15.40:0/3122894213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f8199050 0x7fe8f81994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:50.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.491+0000 7fe8fd3c0700 1 -- 172.21.15.40:0/3122894213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f8199050 msgr2=0x7fe8f81994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:50.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.491+0000 7fe8fd3c0700 1 --2- 172.21.15.40:0/3122894213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f8199050 0x7fe8f81994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:50.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.491+0000 7fe8fd3c0700 1 -- 172.21.15.40:0/3122894213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8f81a1b80 msgr2=0x7fe8f81a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:50.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.491+0000 7fe8fd3c0700 1 --2- 172.21.15.40:0/3122894213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8f81a1b80 0x7fe8f81a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:50.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.491+0000 7fe8fd3c0700 1 -- 172.21.15.40:0/3122894213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f81a4730 con 0x7fe8f81065e0 2024-06-06T03:28:50.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.491+0000 7fe8fd3c0700 1 --2- 172.21.15.40:0/3122894213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f81065e0 0x7fe8f8198b10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400bdc0 tx=0x7fe8f400bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:50.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.491+0000 7fe8ee7fc700 1 -- 172.21.15.40:0/3122894213 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4026070 con 0x7fe8f81065e0 2024-06-06T03:28:50.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.491+0000 7fe8ff624700 1 -- 172.21.15.40:0/3122894213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f81a49c0 con 0x7fe8f81065e0 2024-06-06T03:28:50.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.491+0000 7fe8ee7fc700 1 -- 172.21.15.40:0/3122894213 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8f4005da0 con 0x7fe8f81065e0 2024-06-06T03:28:50.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.491+0000 7fe8ee7fc700 1 -- 172.21.15.40:0/3122894213 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40038e0 con 0x7fe8f81065e0 2024-06-06T03:28:50.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.491+0000 7fe8ff624700 1 -- 172.21.15.40:0/3122894213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f81a4ef0 con 0x7fe8f81065e0 2024-06-06T03:28:50.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.492+0000 7fe8ee7fc700 1 -- 172.21.15.40:0/3122894213 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe8f4029070 con 0x7fe8f81065e0 2024-06-06T03:28:50.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.492+0000 7fe8ff624700 1 -- 172.21.15.40:0/3122894213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f8063af0 con 0x7fe8f81065e0 2024-06-06T03:28:50.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.492+0000 7fe8ee7fc700 1 --2- 172.21.15.40:0/3122894213 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe8e4039580 0x7fe8e403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:50.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.493+0000 7fe8ee7fc700 1 -- 172.21.15.40:0/3122894213 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe8f404ccd0 con 0x7fe8f81065e0 2024-06-06T03:28:50.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.493+0000 7fe8fcbbf700 1 --2- 172.21.15.40:0/3122894213 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe8e4039580 0x7fe8e403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:50.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.493+0000 7fe8fcbbf700 1 --2- 172.21.15.40:0/3122894213 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe8e4039580 0x7fe8e403ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009ad0 tx=0x7fe8e8009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:50.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.498+0000 7fe8ee7fc700 1 -- 172.21.15.40:0/3122894213 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe8f4013070 con 0x7fe8f81065e0 2024-06-06T03:28:50.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: Updating smithi040:/etc/ceph/ceph.conf 2024-06-06T03:28:50.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: Updating smithi104:/etc/ceph/ceph.conf 2024-06-06T03:28:50.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: Updating smithi131:/etc/ceph/ceph.conf 2024-06-06T03:28:50.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: Updating smithi104:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:50.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: Updating smithi040:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:50.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: Updating smithi131:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:28:50.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T03:28:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T03:28:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T03:28:50.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:50.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: Reconfiguring daemon mon.a on smithi040 2024-06-06T03:28:50.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:28:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.710+0000 7fe8ff624700 1 -- 172.21.15.40:0/3122894213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe8f81a5120 con 0x7fe8f81065e0 2024-06-06T03:28:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.710+0000 7fe8ee7fc700 1 -- 172.21.15.40:0/3122894213 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fe8f4030020 con 0x7fe8f81065e0 2024-06-06T03:28:50.710 INFO:teuthology.orchestra.run.smithi040.stdout:# minimal ceph.conf for 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:28:50.710 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2024-06-06T03:28:50.710 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:28:50.710 INFO:teuthology.orchestra.run.smithi040.stdout: mon_host = [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] 2024-06-06T03:28:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.713+0000 7fe8ff624700 1 -- 172.21.15.40:0/3122894213 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe8e4039580 msgr2=0x7fe8e403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.713+0000 7fe8ff624700 1 --2- 172.21.15.40:0/3122894213 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe8e4039580 0x7fe8e403ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009ad0 tx=0x7fe8e8009210 comp rx=0 tx=0).stop 2024-06-06T03:28:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.713+0000 7fe8ff624700 1 -- 172.21.15.40:0/3122894213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f81065e0 msgr2=0x7fe8f8198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.713+0000 7fe8ff624700 1 --2- 172.21.15.40:0/3122894213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f81065e0 0x7fe8f8198b10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400bdc0 tx=0x7fe8f400bdf0 comp rx=0 tx=0).stop 2024-06-06T03:28:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.713+0000 7fe8ff624700 1 -- 172.21.15.40:0/3122894213 shutdown_connections 2024-06-06T03:28:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.713+0000 7fe8ff624700 1 --2- 172.21.15.40:0/3122894213 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe8e4039580 0x7fe8e403ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.713+0000 7fe8ff624700 1 --2- 172.21.15.40:0/3122894213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8f81a1b80 0x7fe8f81a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.713+0000 7fe8ff624700 1 --2- 172.21.15.40:0/3122894213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f81065e0 0x7fe8f8198b10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.713+0000 7fe8ff624700 1 --2- 172.21.15.40:0/3122894213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8f8199050 0x7fe8f81994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.713+0000 7fe8ff624700 1 -- 172.21.15.40:0/3122894213 >> 172.21.15.40:0/3122894213 conn(0x7fe8f8077510 msgr2=0x7fe8f8108bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.713+0000 7fe8ff624700 1 -- 172.21.15.40:0/3122894213 shutdown_connections 2024-06-06T03:28:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:28:50.713+0000 7fe8ff624700 1 -- 172.21.15.40:0/3122894213 wait complete. 2024-06-06T03:28:51.542 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:51 smithi040 ceph-mon[32220]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:51.542 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:51 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3122894213' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:51.542 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:51 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:51.542 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:51 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:51.542 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:51 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T03:28:51.543 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:51 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T03:28:51.543 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:51 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:51.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:51 smithi131 ceph-mon[36687]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:51.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:51 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3122894213' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:51.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:51 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:51.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:51 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:51.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:51 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T03:28:51.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:51 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T03:28:51.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:51 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:51.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:51 smithi104 ceph-mon[36569]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:51.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:51 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3122894213' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:51.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:51 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:51.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:51 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:51.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:51 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T03:28:51.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:51 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T03:28:51.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:51 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:51.923 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-06T03:28:51.924 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:28:51.924 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T03:28:51.953 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:28:51.954 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:28:52.021 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-06-06T03:28:52.021 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T03:28:52.047 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-06-06T03:28:52.047 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:28:52.110 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-06-06T03:28:52.111 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T03:28:52.141 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-06-06T03:28:52.141 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:28:52.209 INFO:tasks.cephadm:Adding mgr.x on smithi040 2024-06-06T03:28:52.209 INFO:tasks.cephadm:Adding mgr.y on smithi104 2024-06-06T03:28:52.209 INFO:tasks.cephadm:Adding mgr.z on smithi131 2024-06-06T03:28:52.209 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi040=x;smithi104=y;smithi131=z' 2024-06-06T03:28:52.482 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:28:52.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:52 smithi131 ceph-mon[36687]: Reconfiguring mon.b (monmap changed)... 2024-06-06T03:28:52.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:52 smithi131 ceph-mon[36687]: Reconfiguring daemon mon.b on smithi104 2024-06-06T03:28:52.710 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:52 smithi104 ceph-mon[36569]: Reconfiguring mon.b (monmap changed)... 2024-06-06T03:28:52.710 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:52 smithi104 ceph-mon[36569]: Reconfiguring daemon mon.b on smithi104 2024-06-06T03:28:52.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:52 smithi040 ceph-mon[32220]: Reconfiguring mon.b (monmap changed)... 2024-06-06T03:28:52.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:52 smithi040 ceph-mon[32220]: Reconfiguring daemon mon.b on smithi104 2024-06-06T03:28:53.683 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:53 smithi131 ceph-mon[36687]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:53.683 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:53 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:53.683 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:53 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:53.683 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:53 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T03:28:53.684 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:53 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T03:28:53.684 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:53 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:53.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:53 smithi040 ceph-mon[32220]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:53.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:53 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:53.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:53 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:53.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:53 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T03:28:53.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:53 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T03:28:53.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:53 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:53.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:53 smithi104 ceph-mon[36569]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:53.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:53 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:53.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:53 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:53.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:53 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T03:28:53.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:53 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T03:28:53.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:53 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:54.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:54 smithi131 ceph-mon[36687]: Reconfiguring mon.c (monmap changed)... 2024-06-06T03:28:54.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:54 smithi131 ceph-mon[36687]: Reconfiguring daemon mon.c on smithi131 2024-06-06T03:28:54.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:54 smithi040 ceph-mon[32220]: Reconfiguring mon.c (monmap changed)... 2024-06-06T03:28:54.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:54 smithi040 ceph-mon[32220]: Reconfiguring daemon mon.c on smithi131 2024-06-06T03:28:54.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:54 smithi104 ceph-mon[36569]: Reconfiguring mon.c (monmap changed)... 2024-06-06T03:28:54.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:54 smithi104 ceph-mon[36569]: Reconfiguring daemon mon.c on smithi131 2024-06-06T03:28:55.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.137+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3148138785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c7c100ea0 msgr2=0x7f0c7c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:55.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.137+0000 7f0c82e7f700 1 --2- 172.21.15.131:0/3148138785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c7c100ea0 0x7f0c7c1012c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800b0a0 tx=0x7f0c7800b3b0 comp rx=0 tx=0).stop 2024-06-06T03:28:55.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.137+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3148138785 shutdown_connections 2024-06-06T03:28:55.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.137+0000 7f0c82e7f700 1 --2- 172.21.15.131:0/3148138785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c7c100ea0 0x7f0c7c1012c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:55.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.137+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3148138785 >> 172.21.15.131:0/3148138785 conn(0x7f0c7c0fc490 msgr2=0x7f0c7c0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:55.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.137+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3148138785 shutdown_connections 2024-06-06T03:28:55.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.137+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3148138785 wait complete. 2024-06-06T03:28:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c82e7f700 1 Processor -- start 2024-06-06T03:28:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c82e7f700 1 -- start start 2024-06-06T03:28:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c82e7f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c7c100ea0 0x7f0c7c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c82e7f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c7c197da0 0x7f0c7c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c82e7f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c7c198220 0x7f0c7c19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c82e7f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c106e90 con 0x7f0c7c197da0 2024-06-06T03:28:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c82e7f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c106d10 con 0x7f0c7c198220 2024-06-06T03:28:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c82e7f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c107010 con 0x7f0c7c100ea0 2024-06-06T03:28:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c80c1b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c7c100ea0 0x7f0c7c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c80c1b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c7c100ea0 0x7f0c7c197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59364/0 (socket says 172.21.15.131:59364) 2024-06-06T03:28:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c80c1b700 1 -- 172.21.15.131:0/3003368022 learned_addr learned my addr 172.21.15.131:0/3003368022 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-06-06T03:28:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c73fff700 1 --2- 172.21.15.131:0/3003368022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c7c197da0 0x7f0c7c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c8141c700 1 --2- 172.21.15.131:0/3003368022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c7c198220 0x7f0c7c19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c80c1b700 1 -- 172.21.15.131:0/3003368022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c7c198220 msgr2=0x7f0c7c19f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c80c1b700 1 --2- 172.21.15.131:0/3003368022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c7c198220 0x7f0c7c19f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c80c1b700 1 -- 172.21.15.131:0/3003368022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c7c197da0 msgr2=0x7f0c7c19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c80c1b700 1 --2- 172.21.15.131:0/3003368022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c7c197da0 0x7f0c7c19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.138+0000 7f0c80c1b700 1 -- 172.21.15.131:0/3003368022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7c19fc70 con 0x7f0c7c100ea0 2024-06-06T03:28:55.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.139+0000 7f0c80c1b700 1 --2- 172.21.15.131:0/3003368022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c7c100ea0 0x7f0c7c197860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0c78000f80 tx=0x7f0c78009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:55.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.139+0000 7f0c71ffb700 1 -- 172.21.15.131:0/3003368022 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78017040 con 0x7f0c7c100ea0 2024-06-06T03:28:55.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.139+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3003368022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c7c0fc8c0 con 0x7f0c7c100ea0 2024-06-06T03:28:55.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.139+0000 7f0c71ffb700 1 -- 172.21.15.131:0/3003368022 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c7800bde0 con 0x7f0c7c100ea0 2024-06-06T03:28:55.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.139+0000 7f0c71ffb700 1 -- 172.21.15.131:0/3003368022 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78012a70 con 0x7f0c7c100ea0 2024-06-06T03:28:55.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.139+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3003368022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c7c0fce80 con 0x7f0c7c100ea0 2024-06-06T03:28:55.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.140+0000 7f0c71ffb700 1 -- 172.21.15.131:0/3003368022 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0c78012c10 con 0x7f0c7c100ea0 2024-06-06T03:28:55.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.140+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3003368022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c7c063af0 con 0x7f0c7c100ea0 2024-06-06T03:28:55.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.140+0000 7f0c71ffb700 1 --2- 172.21.15.131:0/3003368022 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0c64039580 0x7f0c6403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:28:55.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.140+0000 7f0c71ffb700 1 -- 172.21.15.131:0/3003368022 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0c7800f020 con 0x7f0c7c100ea0 2024-06-06T03:28:55.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.140+0000 7f0c73fff700 1 --2- 172.21.15.131:0/3003368022 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0c64039580 0x7f0c6403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:28:55.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.141+0000 7f0c73fff700 1 --2- 172.21.15.131:0/3003368022 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0c64039580 0x7f0c6403ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009ad0 tx=0x7f0c6c009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:28:55.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.145+0000 7f0c71ffb700 1 -- 172.21.15.131:0/3003368022 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0c78029020 con 0x7f0c7c100ea0 2024-06-06T03:28:55.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.353+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3003368022 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi040=x;smithi104=y;smithi131=z", "target": ["mon-mgr", ""]}) -- 0x7f0c7c062960 con 0x7f0c64039580 2024-06-06T03:28:55.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.358+0000 7f0c71ffb700 1 -- 172.21.15.131:0/3003368022 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0c7c062960 con 0x7f0c64039580 2024-06-06T03:28:55.358 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled mgr update... 2024-06-06T03:28:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.361+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3003368022 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0c64039580 msgr2=0x7f0c6403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.361+0000 7f0c82e7f700 1 --2- 172.21.15.131:0/3003368022 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0c64039580 0x7f0c6403ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009ad0 tx=0x7f0c6c009210 comp rx=0 tx=0).stop 2024-06-06T03:28:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.361+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3003368022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c7c100ea0 msgr2=0x7f0c7c197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:28:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.361+0000 7f0c82e7f700 1 --2- 172.21.15.131:0/3003368022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c7c100ea0 0x7f0c7c197860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0c78000f80 tx=0x7f0c78009510 comp rx=0 tx=0).stop 2024-06-06T03:28:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.361+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3003368022 shutdown_connections 2024-06-06T03:28:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.361+0000 7f0c82e7f700 1 --2- 172.21.15.131:0/3003368022 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0c64039580 0x7f0c6403ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.361+0000 7f0c82e7f700 1 --2- 172.21.15.131:0/3003368022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c7c100ea0 0x7f0c7c197860 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.361+0000 7f0c82e7f700 1 --2- 172.21.15.131:0/3003368022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c7c198220 0x7f0c7c19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:55.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.361+0000 7f0c82e7f700 1 --2- 172.21.15.131:0/3003368022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c7c197da0 0x7f0c7c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:28:55.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.361+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3003368022 >> 172.21.15.131:0/3003368022 conn(0x7f0c7c0fc490 msgr2=0x7f0c7c18e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:28:55.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.361+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3003368022 shutdown_connections 2024-06-06T03:28:55.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:28:55.361+0000 7f0c82e7f700 1 -- 172.21.15.131:0/3003368022 wait complete. 2024-06-06T03:28:55.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:55 smithi131 ceph-mon[36687]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:55.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:55 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:55.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:55 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:55.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:55 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:28:55.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:55 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:55.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:55 smithi040 ceph-mon[32220]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:55.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:55 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:55.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:55 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:55.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:55 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:28:55.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:55 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:55.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:55 smithi104 ceph-mon[36569]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:55.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:55 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:55.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:55 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:55.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:55 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:28:55.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:55 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:55.956 DEBUG:teuthology.orchestra.run.smithi104:mgr.y> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-06T03:28:55.959 DEBUG:teuthology.orchestra.run.smithi131:mgr.z> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.z.service 2024-06-06T03:28:55.961 INFO:tasks.cephadm:Deploying OSDs... 2024-06-06T03:28:55.961 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:28:55.961 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T03:28:55.979 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T03:28:55.979 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2024-06-06T03:28:56.035 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T03:28:56.035 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T03:28:56.035 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-06-06T03:28:56.036 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T03:28:56.036 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T03:28:56.036 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-06-06 03:28:00.711940917 +0000 2024-06-06T03:28:56.036 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-06-06 03:23:11.808762478 +0000 2024-06-06T03:28:56.036 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-06-06 03:23:11.808762478 +0000 2024-06-06T03:28:56.036 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-06-06 03:23:11.808762478 +0000 2024-06-06T03:28:56.036 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T03:28:56.101 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-06-06T03:28:56.101 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-06-06T03:28:56.101 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.00021522 s, 2.4 MB/s 2024-06-06T03:28:56.102 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T03:28:56.160 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2024-06-06T03:28:56.216 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T03:28:56.217 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T03:28:56.217 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-06-06T03:28:56.217 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T03:28:56.217 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T03:28:56.217 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-06-06 03:28:00.750940464 +0000 2024-06-06T03:28:56.218 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-06-06 03:23:12.143758553 +0000 2024-06-06T03:28:56.218 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-06-06 03:23:12.143758553 +0000 2024-06-06T03:28:56.218 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-06-06 03:23:12.143758553 +0000 2024-06-06T03:28:56.218 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T03:28:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-06-06T03:28:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-06-06T03:28:56.281 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000181334 s, 2.8 MB/s 2024-06-06T03:28:56.282 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T03:28:56.338 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2024-06-06T03:28:56.395 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T03:28:56.395 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T03:28:56.395 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 710 Links: 1 2024-06-06T03:28:56.395 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T03:28:56.395 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T03:28:56.395 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-06-06 03:28:00.785940057 +0000 2024-06-06T03:28:56.395 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-06-06 03:23:12.484754557 +0000 2024-06-06T03:28:56.395 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-06-06 03:23:12.484754557 +0000 2024-06-06T03:28:56.396 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-06-06 03:23:12.484754557 +0000 2024-06-06T03:28:56.396 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T03:28:56.460 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-06-06T03:28:56.460 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-06-06T03:28:56.460 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000202653 s, 2.5 MB/s 2024-06-06T03:28:56.461 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T03:28:56.473 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:56 smithi131 ceph-mon[36687]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi040=x;smithi104=y;smithi131=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:28:56.473 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:56 smithi131 ceph-mon[36687]: Saving service mgr spec with placement smithi040=x;smithi104=y;smithi131=z;count:3 2024-06-06T03:28:56.473 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:56 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:56.473 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:56 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:28:56.473 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:56 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:56.473 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:56 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T03:28:56.473 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:56 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T03:28:56.473 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:56 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T03:28:56.473 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:56 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:56.519 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2024-06-06T03:28:56.576 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T03:28:56.576 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T03:28:56.576 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 728 Links: 1 2024-06-06T03:28:56.576 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T03:28:56.576 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T03:28:56.576 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-06-06 03:28:00.820939650 +0000 2024-06-06T03:28:56.576 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-06-06 03:23:12.821750609 +0000 2024-06-06T03:28:56.576 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-06-06 03:23:12.821750609 +0000 2024-06-06T03:28:56.576 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-06-06 03:23:12.821750609 +0000 2024-06-06T03:28:56.577 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T03:28:56.638 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:56 smithi040 ceph-mon[32220]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi040=x;smithi104=y;smithi131=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:28:56.639 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:56 smithi040 ceph-mon[32220]: Saving service mgr spec with placement smithi040=x;smithi104=y;smithi131=z;count:3 2024-06-06T03:28:56.639 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:56 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:56.639 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:56 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:28:56.639 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:56 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:56.639 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:56 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T03:28:56.639 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:56 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T03:28:56.639 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:56 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T03:28:56.640 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:56 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:56.641 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-06-06T03:28:56.641 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-06-06T03:28:56.641 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000195344 s, 2.6 MB/s 2024-06-06T03:28:56.642 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T03:28:56.699 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-06-06T03:28:56.699 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T03:28:56.718 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T03:28:56.718 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2024-06-06T03:28:56.775 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T03:28:56.776 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T03:28:56.776 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-06-06T03:28:56.776 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T03:28:56.776 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T03:28:56.776 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-06-06 03:28:41.085149278 +0000 2024-06-06T03:28:56.776 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-06-06 03:23:18.186703253 +0000 2024-06-06T03:28:56.776 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-06-06 03:23:18.186703253 +0000 2024-06-06T03:28:56.776 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-06-06 03:23:18.186703253 +0000 2024-06-06T03:28:56.776 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T03:28:56.839 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:56 smithi104 ceph-mon[36569]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi040=x;smithi104=y;smithi131=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:28:56.839 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:56 smithi104 ceph-mon[36569]: Saving service mgr spec with placement smithi040=x;smithi104=y;smithi131=z;count:3 2024-06-06T03:28:56.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:56 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:56.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:56 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:28:56.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:56 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:56.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:56 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T03:28:56.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:56 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T03:28:56.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:56 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T03:28:56.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:56 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:56.841 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-06-06T03:28:56.842 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-06-06T03:28:56.842 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000276258 s, 1.9 MB/s 2024-06-06T03:28:56.843 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T03:28:56.901 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2024-06-06T03:28:56.957 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T03:28:56.958 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T03:28:56.958 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-06-06T03:28:56.958 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T03:28:56.958 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T03:28:56.958 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-06-06 03:28:41.085149278 +0000 2024-06-06T03:28:56.958 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-06-06 03:23:18.525699622 +0000 2024-06-06T03:28:56.958 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-06-06 03:23:18.525699622 +0000 2024-06-06T03:28:56.958 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-06-06 03:23:18.525699622 +0000 2024-06-06T03:28:56.958 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T03:28:57.021 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-06-06T03:28:57.022 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-06-06T03:28:57.022 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.00020458 s, 2.5 MB/s 2024-06-06T03:28:57.023 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T03:28:57.080 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2024-06-06T03:28:57.136 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T03:28:57.136 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T03:28:57.136 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-06-06T03:28:57.136 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T03:28:57.136 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T03:28:57.136 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-06-06 03:28:41.085149278 +0000 2024-06-06T03:28:57.137 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-06-06 03:23:18.854696098 +0000 2024-06-06T03:28:57.137 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-06-06 03:23:18.854696098 +0000 2024-06-06T03:28:57.137 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-06-06 03:23:18.854696098 +0000 2024-06-06T03:28:57.137 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T03:28:57.199 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-06-06T03:28:57.199 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-06-06T03:28:57.199 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000226506 s, 2.3 MB/s 2024-06-06T03:28:57.201 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T03:28:57.258 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2024-06-06T03:28:57.314 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T03:28:57.314 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T03:28:57.314 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-06-06T03:28:57.314 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T03:28:57.314 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T03:28:57.314 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-06-06 03:28:41.086149268 +0000 2024-06-06T03:28:57.314 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-06-06 03:23:19.200692392 +0000 2024-06-06T03:28:57.314 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-06-06 03:23:19.200692392 +0000 2024-06-06T03:28:57.314 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-06-06 03:23:19.200692392 +0000 2024-06-06T03:28:57.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T03:28:57.378 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-06-06T03:28:57.379 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-06-06T03:28:57.379 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.00092893 s, 551 kB/s 2024-06-06T03:28:57.381 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T03:28:57.437 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-06-06T03:28:57.437 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T03:28:57.453 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T03:28:57.453 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2024-06-06T03:28:57.503 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:57 smithi131 ceph-mon[36687]: Deploying daemon mgr.z on smithi131 2024-06-06T03:28:57.503 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:57 smithi131 ceph-mon[36687]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:57.509 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T03:28:57.509 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T03:28:57.509 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-06-06T03:28:57.509 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T03:28:57.509 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T03:28:57.509 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-06-06 03:28:57.437306673 +0000 2024-06-06T03:28:57.509 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-06-06 03:23:18.047341361 +0000 2024-06-06T03:28:57.509 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-06-06 03:23:18.047341361 +0000 2024-06-06T03:28:57.509 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-06-06 03:23:18.047341361 +0000 2024-06-06T03:28:57.510 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T03:28:57.577 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-06-06T03:28:57.577 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-06-06T03:28:57.577 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000256752 s, 2.0 MB/s 2024-06-06T03:28:57.579 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T03:28:57.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:57 smithi104 ceph-mon[36569]: Deploying daemon mgr.z on smithi131 2024-06-06T03:28:57.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:57 smithi104 ceph-mon[36569]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:57.635 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2024-06-06T03:28:57.689 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T03:28:57.689 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T03:28:57.690 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-06T03:28:57.690 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T03:28:57.690 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T03:28:57.690 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-06-06 03:28:57.437306673 +0000 2024-06-06T03:28:57.690 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-06-06 03:23:18.373334506 +0000 2024-06-06T03:28:57.690 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-06-06 03:23:18.373334506 +0000 2024-06-06T03:28:57.690 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-06-06 03:23:18.373334506 +0000 2024-06-06T03:28:57.690 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T03:28:57.753 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-06-06T03:28:57.753 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-06-06T03:28:57.753 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000209054 s, 2.4 MB/s 2024-06-06T03:28:57.754 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T03:28:57.809 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2024-06-06T03:28:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:57 smithi040 ceph-mon[32220]: Deploying daemon mgr.z on smithi131 2024-06-06T03:28:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:57 smithi040 ceph-mon[32220]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:57.864 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T03:28:57.864 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T03:28:57.864 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-06-06T03:28:57.864 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T03:28:57.864 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T03:28:57.864 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-06-06 03:28:57.437306673 +0000 2024-06-06T03:28:57.864 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-06-06 03:23:18.714327336 +0000 2024-06-06T03:28:57.864 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-06-06 03:23:18.714327336 +0000 2024-06-06T03:28:57.865 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-06-06 03:23:18.714327336 +0000 2024-06-06T03:28:57.865 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T03:28:57.925 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-06-06T03:28:57.925 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-06-06T03:28:57.925 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000226021 s, 2.3 MB/s 2024-06-06T03:28:57.926 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T03:28:57.984 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2024-06-06T03:28:58.054 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T03:28:58.054 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T03:28:58.054 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-06-06T03:28:58.054 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T03:28:58.054 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T03:28:58.054 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-06-06 03:28:57.437306673 +0000 2024-06-06T03:28:58.054 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-06-06 03:23:19.035320586 +0000 2024-06-06T03:28:58.054 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-06-06 03:23:19.035320586 +0000 2024-06-06T03:28:58.055 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-06-06 03:23:19.035320586 +0000 2024-06-06T03:28:58.055 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T03:28:58.119 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-06-06T03:28:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-06-06T03:28:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000184357 s, 2.8 MB/s 2024-06-06T03:28:58.121 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T03:28:58.176 INFO:tasks.cephadm:Deploying osd.0 on smithi040 with /dev/vg_nvme/lv_4... 2024-06-06T03:28:58.176 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T03:28:58.306 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:28:58 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:28:58.253+0000 7f14c6d05200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T03:28:58.430 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:28:58.782 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:28:58 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:28:58.447+0000 7f14c6d05200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:28:58.782 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:28:58 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:28:58.554+0000 7f14c6d05200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T03:28:59.247 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:28:58 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:28:58.869+0000 7f14c6d05200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T03:28:59.247 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:28:58 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:28:58.955+0000 7f14c6d05200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T03:28:59.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:59 smithi131 ceph-mon[36687]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:59.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:59 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:59.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:59 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:59.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:59 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:59.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:59 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T03:28:59.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:59 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T03:28:59.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:59 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T03:28:59.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:59 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:59.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:28:59 smithi131 ceph-mon[36687]: Deploying daemon mgr.y on smithi104 2024-06-06T03:28:59.547 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:59 smithi040 ceph-mon[32220]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:59.547 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:59 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:59.547 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:59 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:59.548 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:59 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:59.548 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:59 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T03:28:59.548 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:59 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T03:28:59.548 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:59 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T03:28:59.548 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:59 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:59.548 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:28:59 smithi040 ceph-mon[32220]: Deploying daemon mgr.y on smithi104 2024-06-06T03:28:59.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:59 smithi104 ceph-mon[36569]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:28:59.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:59 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:59.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:59 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:59.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:59 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:28:59.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:59 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T03:28:59.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:59 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T03:28:59.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:59 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T03:28:59.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:59 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:28:59.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:28:59 smithi104 ceph-mon[36569]: Deploying daemon mgr.y on smithi104 2024-06-06T03:29:00.308 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T03:29:00.308 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T03:29:00.308 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-06-06T03:29:00.308 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-06-06T03:29:00.308 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011048 s, 949 MB/s 2024-06-06T03:29:00.308 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-06-06T03:29:00.340 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:00 smithi104 systemd[1]: Starting Ceph mgr.y for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:29:00.472 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:00 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:00.141+0000 7f14c6d05200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T03:29:00.473 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:00 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:00.305+0000 7f14c6d05200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T03:29:00.473 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:00 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:00.473+0000 7f14c6d05200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T03:29:00.703 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:00 smithi104 podman[37927]: 2024-06-06 03:29:00.318584382 +0000 UTC m=+0.017183705 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:29:00.703 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:00 smithi104 podman[37927]: 2024-06-06 03:29:00.43536696 +0000 UTC m=+0.133966328 container create 87df9e888403d5bb5c9cc4ea49d2b726571f1e2085a9822e3c326adca4a1f12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-06T03:29:00.703 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:00 smithi104 podman[37927]: 2024-06-06 03:29:00.628963474 +0000 UTC m=+0.327562783 container init 87df9e888403d5bb5c9cc4ea49d2b726571f1e2085a9822e3c326adca4a1f12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-06T03:29:00.703 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:00 smithi104 podman[37927]: 2024-06-06 03:29:00.632038541 +0000 UTC m=+0.330637851 container start 87df9e888403d5bb5c9cc4ea49d2b726571f1e2085a9822e3c326adca4a1f12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T03:29:00.703 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:00 smithi104 ceph-mgr[37947]: -- 172.21.15.104:0/1280965665 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eb210025a0 con 0x55eb20fd7000 2024-06-06T03:29:00.703 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:00 smithi104 bash[37927]: 87df9e888403d5bb5c9cc4ea49d2b726571f1e2085a9822e3c326adca4a1f12c 2024-06-06T03:29:00.943 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_4 2024-06-06T03:29:00.974 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:00 smithi104 systemd[1]: Started Ceph mgr.y for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:29:00.974 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:00 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:00.786+0000 7f564dcbd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T03:29:00.974 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:00 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:00.974+0000 7f564dcbd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:29:01.032 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:00 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:00.754+0000 7f14c6d05200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T03:29:01.032 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:00 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:00.845+0000 7f14c6d05200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T03:29:01.189 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:29:01.254 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:01 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:01.085+0000 7f564dcbd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T03:29:01.524 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:01 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:01.421+0000 7f564dcbd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T03:29:01.525 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:01 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:01.524+0000 7f564dcbd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T03:29:01.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:01 smithi104 ceph-mon[36569]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:01.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:01 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:01.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:01 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:01.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:01 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:01.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:01 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:01.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:01 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:01.532 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:01 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:01.053+0000 7f14c6d05200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T03:29:01.532 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:01 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:01.248+0000 7f14c6d05200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:29:01.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:01 smithi131 ceph-mon[36687]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:01.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:01 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:01.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:01 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:01.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:01 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:01.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:01 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:01.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:01 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:01.576 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:01 smithi040 ceph-mon[32220]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:01.576 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:01 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:01.576 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:01 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:01.576 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:01 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:01.576 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:01 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:01.576 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:01 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:02.032 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:01 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:01.889+0000 7f14c6d05200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T03:29:02.619 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:02 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:02.321+0000 7f14c6d05200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T03:29:02.619 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:02 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:02.418+0000 7f14c6d05200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T03:29:02.619 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:02 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:02.620+0000 7f14c6d05200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T03:29:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.687+0000 7f2f382c3700 1 -- 172.21.15.40:0/2252191065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f301013f0 msgr2=0x7f2f30101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:29:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.687+0000 7f2f382c3700 1 --2- 172.21.15.40:0/2252191065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f301013f0 0x7f2f30101810 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0099c0 tx=0x7f2f2c009cd0 comp rx=0 tx=0).stop 2024-06-06T03:29:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.687+0000 7f2f382c3700 1 -- 172.21.15.40:0/2252191065 shutdown_connections 2024-06-06T03:29:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.687+0000 7f2f382c3700 1 --2- 172.21.15.40:0/2252191065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f30102f90 0x7f2f30109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.687+0000 7f2f382c3700 1 --2- 172.21.15.40:0/2252191065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f301013f0 0x7f2f30101810 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.687+0000 7f2f382c3700 1 --2- 172.21.15.40:0/2252191065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f301025f0 0x7f2f30102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.687+0000 7f2f382c3700 1 -- 172.21.15.40:0/2252191065 >> 172.21.15.40:0/2252191065 conn(0x7f2f300fc990 msgr2=0x7f2f300fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:29:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.687+0000 7f2f382c3700 1 -- 172.21.15.40:0/2252191065 shutdown_connections 2024-06-06T03:29:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.687+0000 7f2f382c3700 1 -- 172.21.15.40:0/2252191065 wait complete. 2024-06-06T03:29:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.688+0000 7f2f382c3700 1 Processor -- start 2024-06-06T03:29:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.688+0000 7f2f382c3700 1 -- start start 2024-06-06T03:29:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.688+0000 7f2f382c3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f301013f0 0x7f2f301975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:29:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.688+0000 7f2f382c3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f301025f0 0x7f2f30197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:29:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.688+0000 7f2f382c3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f30102f90 0x7f2f3019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:29:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.688+0000 7f2f382c3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3010cfb0 con 0x7f2f30102f90 2024-06-06T03:29:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.688+0000 7f2f382c3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3010ce30 con 0x7f2f301025f0 2024-06-06T03:29:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.688+0000 7f2f382c3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3010d130 con 0x7f2f301013f0 2024-06-06T03:29:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.688+0000 7f2f36860700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f30102f90 0x7f2f3019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:29:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.688+0000 7f2f36860700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f30102f90 0x7f2f3019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33384/0 (socket says 172.21.15.40:33384) 2024-06-06T03:29:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.688+0000 7f2f3585e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f301025f0 0x7f2f30197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:29:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.688+0000 7f2f36860700 1 -- 172.21.15.40:0/5243043 learned_addr learned my addr 172.21.15.40:0/5243043 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:29:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.688+0000 7f2f3605f700 1 --2- 172.21.15.40:0/5243043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f301013f0 0x7f2f301975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:29:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.689+0000 7f2f36860700 1 -- 172.21.15.40:0/5243043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f301013f0 msgr2=0x7f2f301975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:29:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.689+0000 7f2f36860700 1 --2- 172.21.15.40:0/5243043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f301013f0 0x7f2f301975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.689+0000 7f2f36860700 1 -- 172.21.15.40:0/5243043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f301025f0 msgr2=0x7f2f30197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:29:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.689+0000 7f2f36860700 1 --2- 172.21.15.40:0/5243043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f301025f0 0x7f2f30197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.689+0000 7f2f36860700 1 -- 172.21.15.40:0/5243043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3019f250 con 0x7f2f30102f90 2024-06-06T03:29:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.689+0000 7f2f3585e700 1 --2- 172.21.15.40:0/5243043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f301025f0 0x7f2f30197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:29:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.689+0000 7f2f36860700 1 --2- 172.21.15.40:0/5243043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f30102f90 0x7f2f3019ebb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800e5b0 tx=0x7f2f2800e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:29:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.689+0000 7f2f277fe700 1 -- 172.21.15.40:0/5243043 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28004db0 con 0x7f2f30102f90 2024-06-06T03:29:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.689+0000 7f2f382c3700 1 -- 172.21.15.40:0/5243043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3006a1d0 con 0x7f2f30102f90 2024-06-06T03:29:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.689+0000 7f2f277fe700 1 -- 172.21.15.40:0/5243043 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f280090d0 con 0x7f2f30102f90 2024-06-06T03:29:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.689+0000 7f2f277fe700 1 -- 172.21.15.40:0/5243043 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f280046f0 con 0x7f2f30102f90 2024-06-06T03:29:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.689+0000 7f2f382c3700 1 -- 172.21.15.40:0/5243043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3006a710 con 0x7f2f30102f90 2024-06-06T03:29:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.690+0000 7f2f277fe700 1 -- 172.21.15.40:0/5243043 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2f28007500 con 0x7f2f30102f90 2024-06-06T03:29:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.690+0000 7f2f382c3700 1 -- 172.21.15.40:0/5243043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f30069180 con 0x7f2f30102f90 2024-06-06T03:29:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.690+0000 7f2f277fe700 1 --2- 172.21.15.40:0/5243043 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f2f1c03d740 0x7f2f1c03fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:29:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.691+0000 7f2f277fe700 1 -- 172.21.15.40:0/5243043 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2f2804ba60 con 0x7f2f30102f90 2024-06-06T03:29:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.691+0000 7f2f3605f700 1 --2- 172.21.15.40:0/5243043 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f2f1c03d740 0x7f2f1c03fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:29:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.691+0000 7f2f3605f700 1 --2- 172.21.15.40:0/5243043 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f2f1c03d740 0x7f2f1c03fc00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c000c00 tx=0x7f2f2c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:29:02.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.696+0000 7f2f277fe700 1 -- 172.21.15.40:0/5243043 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2f28009280 con 0x7f2f30102f90 2024-06-06T03:29:02.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:02.917+0000 7f2f382c3700 1 -- 172.21.15.40:0/5243043 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2f3006a970 con 0x7f2f1c03d740 2024-06-06T03:29:03.090 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:02 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:02.757+0000 7f564dcbd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T03:29:03.090 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:02 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:02.926+0000 7f564dcbd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T03:29:03.381 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:03.094+0000 7f564dcbd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T03:29:03.382 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:03.382 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.382 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.382 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.382 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.382 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:03.382 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:03.382 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.382 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T03:29:03.383 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T03:29:03.383 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T03:29:03.383 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:03.383 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: Reconfiguring daemon mgr.x on smithi040 2024-06-06T03:29:03.383 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.383 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:29:03.383 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:29:03.383 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:03.474 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:03.139+0000 7f14c6d05200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T03:29:03.475 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:03.232+0000 7f14c6d05200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T03:29:03.475 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:03.475 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.475 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.475 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.475 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.475 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:03.475 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:03.475 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.475 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T03:29:03.475 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T03:29:03.476 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T03:29:03.476 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:03.476 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: Reconfiguring daemon mgr.x on smithi040 2024-06-06T03:29:03.476 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.476 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:29:03.476 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:29:03.476 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:03.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:03.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:03.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:03.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T03:29:03.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T03:29:03.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T03:29:03.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:03.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: Reconfiguring daemon mgr.x on smithi040 2024-06-06T03:29:03.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:03.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:29:03.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:29:03.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:03.674 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:03.381+0000 7f564dcbd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T03:29:03.674 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:03.473+0000 7f564dcbd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T03:29:03.674 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:03.674+0000 7f564dcbd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T03:29:03.782 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:03.475+0000 7f14c6d05200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T03:29:03.782 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:03.608+0000 7f14c6d05200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T03:29:03.782 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:03.705+0000 7f14c6d05200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T03:29:04.090 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:03 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:03.864+0000 7f564dcbd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:29:04.157 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:03 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:03.822+0000 7f14c6d05200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T03:29:04.157 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:04 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:04.124+0000 7f14c6d05200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T03:29:04.460 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:04 smithi104 ceph-mon[36569]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:29:04.460 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:04 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:04.460 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:04 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:04.460 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:04 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:04.460 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:04 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:04.461 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:04 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:04.461 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:04 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:04.492 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:04 smithi040 ceph-mon[32220]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:29:04.492 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:04 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:04.492 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:04 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:04.492 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:04 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:04.492 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:04 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:04.492 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:04 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:04.492 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:04 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:04.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:04 smithi131 ceph-mon[36687]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:29:04.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:04 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:04.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:04 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:04.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:04 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:04.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:04 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:04.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:04 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:04.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:04 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:04.840 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:04 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:04.460+0000 7f564dcbd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T03:29:05.032 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:04 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:04.724+0000 7f14c6d05200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T03:29:05.032 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:29:04 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:29:04.940+0000 7f14c6d05200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T03:29:05.169 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:04 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:04.876+0000 7f564dcbd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T03:29:05.169 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:04 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:04.972+0000 7f564dcbd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T03:29:05.169 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:05 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:05.169+0000 7f564dcbd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T03:29:05.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:05.207+0000 7f2f277fe700 1 -- 172.21.15.40:0/5243043 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f2f28011ca0 con 0x7f2f30102f90 2024-06-06T03:29:05.498 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:05 smithi040 ceph-mon[32220]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:05.498 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:05 smithi040 ceph-mon[32220]: Standby manager daemon z started 2024-06-06T03:29:05.498 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:05 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.131:0/279546738' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T03:29:05.498 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:05 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.131:0/279546738' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T03:29:05.498 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:05 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.131:0/279546738' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T03:29:05.498 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:05 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.131:0/279546738' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T03:29:05.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:05 smithi131 ceph-mon[36687]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:05.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:05 smithi131 ceph-mon[36687]: Standby manager daemon z started 2024-06-06T03:29:05.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:05 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.131:0/279546738' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T03:29:05.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:05 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.131:0/279546738' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T03:29:05.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:05 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.131:0/279546738' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T03:29:05.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:05 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.131:0/279546738' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T03:29:05.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:05 smithi104 ceph-mon[36569]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:05.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:05 smithi104 ceph-mon[36569]: Standby manager daemon z started 2024-06-06T03:29:05.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:05 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.131:0/279546738' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T03:29:05.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:05 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.131:0/279546738' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T03:29:05.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:05 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.131:0/279546738' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T03:29:05.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:05 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.131:0/279546738' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T03:29:06.012 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:05 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:05.684+0000 7f564dcbd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T03:29:06.013 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:05 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:05.774+0000 7f564dcbd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T03:29:06.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:06 smithi104 ceph-mon[36569]: mgrmap e14: x(active, since 77s), standbys: z 2024-06-06T03:29:06.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:06 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T03:29:06.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:06 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3038958841' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63c0a320-a2f2-4b84-87e7-68501d335285"}]: dispatch 2024-06-06T03:29:06.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:06 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3038958841' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63c0a320-a2f2-4b84-87e7-68501d335285"}]': finished 2024-06-06T03:29:06.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:06 smithi104 ceph-mon[36569]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T03:29:06.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:06 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:06.341 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:06.013+0000 7f564dcbd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T03:29:06.341 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:06.146+0000 7f564dcbd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T03:29:06.341 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:06.241+0000 7f564dcbd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T03:29:06.515 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:06 smithi040 ceph-mon[32220]: mgrmap e14: x(active, since 77s), standbys: z 2024-06-06T03:29:06.515 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:06 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T03:29:06.515 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:06 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3038958841' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63c0a320-a2f2-4b84-87e7-68501d335285"}]: dispatch 2024-06-06T03:29:06.515 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:06 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3038958841' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63c0a320-a2f2-4b84-87e7-68501d335285"}]': finished 2024-06-06T03:29:06.515 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:06 smithi040 ceph-mon[32220]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T03:29:06.515 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:06 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:06.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:06 smithi131 ceph-mon[36687]: mgrmap e14: x(active, since 77s), standbys: z 2024-06-06T03:29:06.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:06 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T03:29:06.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:06 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3038958841' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63c0a320-a2f2-4b84-87e7-68501d335285"}]: dispatch 2024-06-06T03:29:06.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:06 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3038958841' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63c0a320-a2f2-4b84-87e7-68501d335285"}]': finished 2024-06-06T03:29:06.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:06 smithi131 ceph-mon[36687]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T03:29:06.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:06 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:06.652 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:06.357+0000 7f564dcbd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T03:29:07.090 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:06.652+0000 7f564dcbd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T03:29:07.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:07 smithi131 ceph-mon[36687]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:07.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:07 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3709826449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:29:07.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:07 smithi040 ceph-mon[32220]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:07.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:07 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3709826449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:29:07.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:07 smithi104 ceph-mon[36569]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:07.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:07 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3709826449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:29:07.590 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:07 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:07.245+0000 7f564dcbd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T03:29:07.590 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:29:07 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:29:07.458+0000 7f564dcbd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T03:29:08.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:08.224+0000 7f2f277fe700 1 -- 172.21.15.40:0/5243043 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f2f2801ec10 con 0x7f2f30102f90 2024-06-06T03:29:08.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:08 smithi131 ceph-mon[36687]: Standby manager daemon y started 2024-06-06T03:29:08.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:08 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.104:0/1099274016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T03:29:08.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:08 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.104:0/1099274016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T03:29:08.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:08 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.104:0/1099274016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T03:29:08.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:08 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.104:0/1099274016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T03:29:08.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:08 smithi040 ceph-mon[32220]: Standby manager daemon y started 2024-06-06T03:29:08.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:08 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.104:0/1099274016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T03:29:08.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:08 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.104:0/1099274016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T03:29:08.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:08 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.104:0/1099274016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T03:29:08.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:08 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.104:0/1099274016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T03:29:08.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:08 smithi104 ceph-mon[36569]: Standby manager daemon y started 2024-06-06T03:29:08.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:08 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.104:0/1099274016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T03:29:08.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:08 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.104:0/1099274016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T03:29:08.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:08 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.104:0/1099274016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T03:29:08.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:08 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.104:0/1099274016' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T03:29:09.528 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:09 smithi131 ceph-mon[36687]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:09.528 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:09 smithi131 ceph-mon[36687]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-06-06T03:29:09.528 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:09 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T03:29:09.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:09 smithi040 ceph-mon[32220]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:09.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:09 smithi040 ceph-mon[32220]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-06-06T03:29:09.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:09 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T03:29:09.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:09 smithi104 ceph-mon[36569]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:09.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:09 smithi104 ceph-mon[36569]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-06-06T03:29:09.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:09 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T03:29:11.422 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:11 smithi040 ceph-mon[32220]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:11.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:11 smithi131 ceph-mon[36687]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:11.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:11 smithi104 ceph-mon[36569]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:13.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:13 smithi131 ceph-mon[36687]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:13.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:13 smithi040 ceph-mon[32220]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:13.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:13 smithi104 ceph-mon[36569]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:14.462 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:14 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T03:29:14.463 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:14 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:14.463 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:14 smithi040 ceph-mon[32220]: Deploying daemon osd.0 on smithi040 2024-06-06T03:29:14.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:14 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T03:29:14.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:14 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:14.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:14 smithi131 ceph-mon[36687]: Deploying daemon osd.0 on smithi040 2024-06-06T03:29:14.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:14 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T03:29:14.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:14 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:14.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:14 smithi104 ceph-mon[36569]: Deploying daemon osd.0 on smithi040 2024-06-06T03:29:15.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:15 smithi040 ceph-mon[32220]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:15.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:15 smithi131 ceph-mon[36687]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:15.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:15 smithi104 ceph-mon[36569]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:17.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:17 smithi131 ceph-mon[36687]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:17 smithi040 ceph-mon[32220]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:17.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:17 smithi104 ceph-mon[36569]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:19 smithi040 ceph-mon[32220]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:19.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:19 smithi131 ceph-mon[36687]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:19.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:19.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:19.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:19.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:19 smithi104 ceph-mon[36569]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:19.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:19.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:19.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:21.676 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:21 smithi040 ceph-mon[32220]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:21.676 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:21 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:21.676 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:21 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:21.676 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:21 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:21.676 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:21 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:21.676 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:21 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:21.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:21 smithi131 ceph-mon[36687]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:21.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:21 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:21.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:21 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:21.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:21 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:21.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:21 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:21.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:21 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:21.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:21 smithi104 ceph-mon[36569]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:21.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:21 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:21.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:21 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:21.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:21 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:21.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:21 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:21.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:21 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:22.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:22.082+0000 7f2f277fe700 1 -- 172.21.15.40:0/5243043 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2f3006a970 con 0x7f2f1c03d740 2024-06-06T03:29:22.085 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 0 on host 'smithi040' 2024-06-06T03:29:22.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:22.084+0000 7f2f382c3700 1 -- 172.21.15.40:0/5243043 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f2f1c03d740 msgr2=0x7f2f1c03fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:29:22.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:22.084+0000 7f2f382c3700 1 --2- 172.21.15.40:0/5243043 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f2f1c03d740 0x7f2f1c03fc00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c000c00 tx=0x7f2f2c005fb0 comp rx=0 tx=0).stop 2024-06-06T03:29:22.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:22.084+0000 7f2f382c3700 1 -- 172.21.15.40:0/5243043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f30102f90 msgr2=0x7f2f3019ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:29:22.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:22.084+0000 7f2f382c3700 1 --2- 172.21.15.40:0/5243043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f30102f90 0x7f2f3019ebb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800e5b0 tx=0x7f2f2800e8c0 comp rx=0 tx=0).stop 2024-06-06T03:29:22.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:22.085+0000 7f2f382c3700 1 -- 172.21.15.40:0/5243043 shutdown_connections 2024-06-06T03:29:22.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:22.085+0000 7f2f382c3700 1 --2- 172.21.15.40:0/5243043 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f2f1c03d740 0x7f2f1c03fc00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:22.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:22.085+0000 7f2f382c3700 1 --2- 172.21.15.40:0/5243043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f301013f0 0x7f2f301975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:22.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:22.085+0000 7f2f382c3700 1 --2- 172.21.15.40:0/5243043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f30102f90 0x7f2f3019ebb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:22.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:22.085+0000 7f2f382c3700 1 --2- 172.21.15.40:0/5243043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f301025f0 0x7f2f30197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:22.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:22.085+0000 7f2f382c3700 1 -- 172.21.15.40:0/5243043 >> 172.21.15.40:0/5243043 conn(0x7f2f300fc990 msgr2=0x7f2f30105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:29:22.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:22.085+0000 7f2f382c3700 1 -- 172.21.15.40:0/5243043 shutdown_connections 2024-06-06T03:29:22.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:29:22.085+0000 7f2f382c3700 1 -- 172.21.15.40:0/5243043 wait complete. 2024-06-06T03:29:22.840 DEBUG:teuthology.orchestra.run.smithi040:osd.0> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.0.service 2024-06-06T03:29:22.842 INFO:tasks.cephadm:Deploying osd.1 on smithi104 with /dev/vg_nvme/lv_4... 2024-06-06T03:29:22.842 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T03:29:23.084 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T03:29:23.205 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:23 smithi104 ceph-mon[36569]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:23.206 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:23.206 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:23.206 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:23.206 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:23 smithi104 ceph-mon[36569]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T03:29:23.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:23 smithi040 ceph-mon[32220]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:23.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:23.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:23.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:23.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:23 smithi040 ceph-mon[32220]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T03:29:23.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:23 smithi131 ceph-mon[36687]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:23.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:23.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:23.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:23.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:23 smithi131 ceph-mon[36687]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T03:29:24.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:24 smithi040 ceph-mon[32220]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T03:29:24.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:24 smithi040 ceph-mon[32220]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T03:29:24.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:24 smithi040 ceph-mon[32220]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-06-06T03:29:24.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:24 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:24.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:24 smithi104 ceph-mon[36569]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T03:29:24.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:24 smithi104 ceph-mon[36569]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T03:29:24.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:24 smithi104 ceph-mon[36569]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-06-06T03:29:24.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:24 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:24.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:24 smithi131 ceph-mon[36687]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T03:29:24.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:24 smithi131 ceph-mon[36687]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T03:29:24.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:24 smithi131 ceph-mon[36687]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-06-06T03:29:24.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:24 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:24.833 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:29:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-0[46886]: 2024-06-06T03:29:24.519+0000 7fbc7b656700 -1 osd.0 0 waiting for initial osdmap 2024-06-06T03:29:24.833 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:29:24 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-0[46886]: 2024-06-06T03:29:24.523+0000 7fbc73441700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T03:29:24.929 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T03:29:24.930 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T03:29:24.930 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-06-06T03:29:24.930 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-06-06T03:29:24.930 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0166493 s, 630 MB/s 2024-06-06T03:29:24.930 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-06-06T03:29:25.231 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:25 smithi104 ceph-mon[36569]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:25.231 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:25 smithi104 ceph-mon[36569]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-06-06T03:29:25.231 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:25 smithi104 ceph-mon[36569]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T03:29:25.231 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:25 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:25.231 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:25 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:25.231 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:25 smithi104 ceph-mon[36569]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' 2024-06-06T03:29:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:25 smithi131 ceph-mon[36687]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:25 smithi131 ceph-mon[36687]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-06-06T03:29:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:25 smithi131 ceph-mon[36687]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T03:29:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:25 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:25 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:25 smithi131 ceph-mon[36687]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' 2024-06-06T03:29:25.534 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_4 2024-06-06T03:29:25.540 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:25 smithi040 ceph-mon[32220]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:25.540 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:25 smithi040 ceph-mon[32220]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-06-06T03:29:25.540 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:25 smithi040 ceph-mon[32220]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T03:29:25.540 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:25 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:25.540 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:25 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:25.540 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:25 smithi040 ceph-mon[32220]: from='osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793]' entity='osd.0' 2024-06-06T03:29:25.771 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T03:29:26.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:26 smithi040 ceph-mon[32220]: purged_snaps scrub starts 2024-06-06T03:29:26.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:26 smithi040 ceph-mon[32220]: purged_snaps scrub ok 2024-06-06T03:29:26.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:26.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:26 smithi040 ceph-mon[32220]: osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793] boot 2024-06-06T03:29:26.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:26 smithi040 ceph-mon[32220]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T03:29:26.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:26.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:26.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:26.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:29:26.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:26.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:26.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:26.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:26.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:26 smithi104 ceph-mon[36569]: purged_snaps scrub starts 2024-06-06T03:29:26.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:26 smithi104 ceph-mon[36569]: purged_snaps scrub ok 2024-06-06T03:29:26.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:26 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:26.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:26 smithi104 ceph-mon[36569]: osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793] boot 2024-06-06T03:29:26.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:26 smithi104 ceph-mon[36569]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T03:29:26.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:26 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:26.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:26 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:26.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:26 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:26.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:26 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:29:26.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:26 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:26.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:26 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:26.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:26 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:26.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:26 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:26.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:26 smithi131 ceph-mon[36687]: purged_snaps scrub starts 2024-06-06T03:29:26.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:26 smithi131 ceph-mon[36687]: purged_snaps scrub ok 2024-06-06T03:29:26.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:26 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:26.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:26 smithi131 ceph-mon[36687]: osd.0 [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793] boot 2024-06-06T03:29:26.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:26 smithi131 ceph-mon[36687]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T03:29:26.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:26 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:29:26.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:26 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:26.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:26 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:26.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:26 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:29:26.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:26 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:26.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:26 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:26.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:26 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:26.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:26 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:27.190 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.185+0000 7f1cbdfee700 1 -- 172.21.15.104:0/713503838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cb8102f90 msgr2=0x7f1cb8105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:29:27.191 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.185+0000 7f1cbdfee700 1 --2- 172.21.15.104:0/713503838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cb8102f90 0x7f1cb8105380 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009990 tx=0x7f1ca8009ca0 comp rx=0 tx=0).stop 2024-06-06T03:29:27.191 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.185+0000 7f1cbdfee700 1 -- 172.21.15.104:0/713503838 shutdown_connections 2024-06-06T03:29:27.191 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.185+0000 7f1cbdfee700 1 --2- 172.21.15.104:0/713503838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb81081f0 0x7f1cb810a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:27.191 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.185+0000 7f1cbdfee700 1 --2- 172.21.15.104:0/713503838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cb8102f90 0x7f1cb8105380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:27.191 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.185+0000 7f1cbdfee700 1 --2- 172.21.15.104:0/713503838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cb81058c0 0x7f1cb8107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:27.191 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.185+0000 7f1cbdfee700 1 -- 172.21.15.104:0/713503838 >> 172.21.15.104:0/713503838 conn(0x7f1cb80fc9c0 msgr2=0x7f1cb80fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:29:27.191 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.185+0000 7f1cbdfee700 1 -- 172.21.15.104:0/713503838 shutdown_connections 2024-06-06T03:29:27.191 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.185+0000 7f1cbdfee700 1 -- 172.21.15.104:0/713503838 wait complete. 2024-06-06T03:29:27.191 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.186+0000 7f1cbdfee700 1 Processor -- start 2024-06-06T03:29:27.191 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.186+0000 7f1cbdfee700 1 -- start start 2024-06-06T03:29:27.191 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.186+0000 7f1cbdfee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cb8102f90 0x7f1cb81987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:29:27.192 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.186+0000 7f1cbdfee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb81058c0 0x7f1cb8198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:29:27.192 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.186+0000 7f1cbdfee700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cb81081f0 0x7f1cb819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:29:27.192 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.186+0000 7f1cbdfee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb810e170 con 0x7f1cb81058c0 2024-06-06T03:29:27.192 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.186+0000 7f1cbdfee700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb810dff0 con 0x7f1cb81081f0 2024-06-06T03:29:27.192 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.186+0000 7f1cbdfee700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb810e2f0 con 0x7f1cb8102f90 2024-06-06T03:29:27.192 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cb81081f0 0x7f1cb819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:29:27.192 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cb81081f0 0x7f1cb819fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:37864/0 (socket says 172.21.15.104:37864) 2024-06-06T03:29:27.192 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb7fff700 1 -- 172.21.15.104:0/701298480 learned_addr learned my addr 172.21.15.104:0/701298480 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-06-06T03:29:27.192 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb77fe700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cb8102f90 0x7f1cb81987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:29:27.192 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb6ffd700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb81058c0 0x7f1cb8198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:29:27.192 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb7fff700 1 -- 172.21.15.104:0/701298480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cb8102f90 msgr2=0x7f1cb81987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:29:27.192 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb7fff700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cb8102f90 0x7f1cb81987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:27.193 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb7fff700 1 -- 172.21.15.104:0/701298480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb81058c0 msgr2=0x7f1cb8198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:29:27.193 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb7fff700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb81058c0 0x7f1cb8198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:27.193 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb7fff700 1 -- 172.21.15.104:0/701298480 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb81a0410 con 0x7f1cb81081f0 2024-06-06T03:29:27.193 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb77fe700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cb8102f90 0x7f1cb81987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:29:27.193 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb7fff700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cb81081f0 0x7f1cb819fd70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000b9e0 tx=0x7f1cb000bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:29:27.193 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb4ff9700 1 -- 172.21.15.104:0/701298480 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0019070 con 0x7f1cb81081f0 2024-06-06T03:29:27.193 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb4ff9700 1 -- 172.21.15.104:0/701298480 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb0014440 con 0x7f1cb81081f0 2024-06-06T03:29:27.193 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cbdfee700 1 -- 172.21.15.104:0/701298480 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb81a0700 con 0x7f1cb81081f0 2024-06-06T03:29:27.193 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cb4ff9700 1 -- 172.21.15.104:0/701298480 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00125a0 con 0x7f1cb81081f0 2024-06-06T03:29:27.193 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.187+0000 7f1cbdfee700 1 -- 172.21.15.104:0/701298480 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb81a0c70 con 0x7f1cb81081f0 2024-06-06T03:29:27.193 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.188+0000 7f1cbdfee700 1 -- 172.21.15.104:0/701298480 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb8069180 con 0x7f1cb81081f0 2024-06-06T03:29:27.193 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.189+0000 7f1cb4ff9700 1 -- 172.21.15.104:0/701298480 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f1cb001c070 con 0x7f1cb81081f0 2024-06-06T03:29:27.193 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.190+0000 7f1cb4ff9700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1ca00a2b90 0x7f1ca00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:29:27.194 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.190+0000 7f1cb4ff9700 1 -- 172.21.15.104:0/701298480 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f1cb00cca20 con 0x7f1cb81081f0 2024-06-06T03:29:27.194 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.190+0000 7f1cb77fe700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1ca00a2b90 0x7f1ca00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:29:27.194 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.191+0000 7f1cb77fe700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1ca00a2b90 0x7f1ca00a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800b5c0 tx=0x7f1ca8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:29:27.194 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.195+0000 7f1cb4ff9700 1 -- 172.21.15.104:0/701298480 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1cb009b3f0 con 0x7f1cb81081f0 2024-06-06T03:29:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:27 smithi104 ceph-mon[36569]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:27 smithi104 ceph-mon[36569]: Detected new or changed devices on smithi040 2024-06-06T03:29:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:27 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi040 to 17092M 2024-06-06T03:29:27.396 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:27.396+0000 7f1cbdfee700 1 -- 172.21.15.104:0/701298480 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1cb80628d0 con 0x7f1ca00a2b90 2024-06-06T03:29:27.433 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:27 smithi131 ceph-mon[36687]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:27.433 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:27 smithi131 ceph-mon[36687]: Detected new or changed devices on smithi040 2024-06-06T03:29:27.433 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:27 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi040 to 17092M 2024-06-06T03:29:27.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:27 smithi040 ceph-mon[32220]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T03:29:27.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:27 smithi040 ceph-mon[32220]: Detected new or changed devices on smithi040 2024-06-06T03:29:27.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:27 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi040 to 17092M 2024-06-06T03:29:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:28 smithi104 ceph-mon[36569]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T03:29:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:28 smithi104 ceph-mon[36569]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:29:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:28 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:29:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:28 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:29:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:28 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:28.453 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:28 smithi131 ceph-mon[36687]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T03:29:28.453 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:28 smithi131 ceph-mon[36687]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:29:28.453 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:28 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:29:28.453 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:28 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:29:28.453 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:28 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:28.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:28 smithi040 ceph-mon[32220]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T03:29:28.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:28 smithi040 ceph-mon[32220]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:29:28.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:28 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:29:28.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:28 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:29:28.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:28 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:29.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:29 smithi104 ceph-mon[36569]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:29.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:29 smithi131 ceph-mon[36687]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:29.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:29 smithi040 ceph-mon[32220]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:30.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:30 smithi131 ceph-mon[36687]: from='client.? 172.21.15.104:0/61671295' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a31534-06a1-4ffa-a54d-577d45bccf98"}]: dispatch 2024-06-06T03:29:30.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:30 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a31534-06a1-4ffa-a54d-577d45bccf98"}]: dispatch 2024-06-06T03:29:30.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:30 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70a31534-06a1-4ffa-a54d-577d45bccf98"}]': finished 2024-06-06T03:29:30.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:30 smithi131 ceph-mon[36687]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T03:29:30.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:30 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:30.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:30 smithi040 ceph-mon[32220]: from='client.? 172.21.15.104:0/61671295' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a31534-06a1-4ffa-a54d-577d45bccf98"}]: dispatch 2024-06-06T03:29:30.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:30 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a31534-06a1-4ffa-a54d-577d45bccf98"}]: dispatch 2024-06-06T03:29:30.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:30 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70a31534-06a1-4ffa-a54d-577d45bccf98"}]': finished 2024-06-06T03:29:30.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:30 smithi040 ceph-mon[32220]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T03:29:30.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:30 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:30.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:30 smithi104 ceph-mon[36569]: from='client.? 172.21.15.104:0/61671295' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a31534-06a1-4ffa-a54d-577d45bccf98"}]: dispatch 2024-06-06T03:29:30.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:30 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a31534-06a1-4ffa-a54d-577d45bccf98"}]: dispatch 2024-06-06T03:29:30.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:30 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70a31534-06a1-4ffa-a54d-577d45bccf98"}]': finished 2024-06-06T03:29:30.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:30 smithi104 ceph-mon[36569]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T03:29:30.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:30 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:31.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:31 smithi131 ceph-mon[36687]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:31.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:31 smithi131 ceph-mon[36687]: from='client.? 172.21.15.104:0/4004945704' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:29:31.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:31 smithi040 ceph-mon[32220]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:31.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:31 smithi040 ceph-mon[32220]: from='client.? 172.21.15.104:0/4004945704' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:29:31.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:31 smithi104 ceph-mon[36569]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:31.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:31 smithi104 ceph-mon[36569]: from='client.? 172.21.15.104:0/4004945704' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:29:33.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:33 smithi104 ceph-mon[36569]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:33.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:33 smithi131 ceph-mon[36687]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:33.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:33 smithi040 ceph-mon[32220]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:35.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:35 smithi104 ceph-mon[36569]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:35 smithi131 ceph-mon[36687]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:35.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:35 smithi040 ceph-mon[32220]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:37.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:37 smithi131 ceph-mon[36687]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:37.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T03:29:37.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:37.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:37 smithi040 ceph-mon[32220]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:37.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T03:29:37.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:37.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:37 smithi104 ceph-mon[36569]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:37.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:37 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T03:29:37.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:37 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:38.255 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:38 smithi104 ceph-mon[36569]: Deploying daemon osd.1 on smithi104 2024-06-06T03:29:38.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:38 smithi131 ceph-mon[36687]: Deploying daemon osd.1 on smithi104 2024-06-06T03:29:38.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:38 smithi040 ceph-mon[32220]: Deploying daemon osd.1 on smithi104 2024-06-06T03:29:39.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:39 smithi104 ceph-mon[36569]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:39.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:39 smithi131 ceph-mon[36687]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:39.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:39 smithi040 ceph-mon[32220]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:41.479 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:41 smithi104 ceph-mon[36569]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:41.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:41 smithi131 ceph-mon[36687]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:41.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:41 smithi040 ceph-mon[32220]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:43.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:43 smithi131 ceph-mon[36687]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:43.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:43 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:43.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:43 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:43.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:43 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:43.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:43 smithi040 ceph-mon[32220]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:43.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:43 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:43.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:43 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:43.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:43 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:43.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:43 smithi104 ceph-mon[36569]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:43.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:43 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:43.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:43 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:43.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:43 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:45.723 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:45 smithi104 ceph-mon[36569]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:45.723 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:45 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:45.723 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:45 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:45.723 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:45 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:45.723 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:45 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:45.723 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:45 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:45.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:45 smithi131 ceph-mon[36687]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:45.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:45 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:45.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:45 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:45.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:45 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:45.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:45 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:45.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:45 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:45.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:45 smithi040 ceph-mon[32220]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:45.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:45 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:45.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:45 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:45.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:45 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:45.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:45 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:45.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:45 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:46.063 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:46.064+0000 7f1cb4ff9700 1 -- 172.21.15.104:0/701298480 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1cb80628d0 con 0x7f1ca00a2b90 2024-06-06T03:29:46.064 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 1 on host 'smithi104' 2024-06-06T03:29:46.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:46.066+0000 7f1cbdfee700 1 -- 172.21.15.104:0/701298480 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1ca00a2b90 msgr2=0x7f1ca00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:29:46.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:46.066+0000 7f1cbdfee700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1ca00a2b90 0x7f1ca00a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800b5c0 tx=0x7f1ca8005fb0 comp rx=0 tx=0).stop 2024-06-06T03:29:46.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:46.066+0000 7f1cbdfee700 1 -- 172.21.15.104:0/701298480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cb81081f0 msgr2=0x7f1cb819fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:29:46.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:46.066+0000 7f1cbdfee700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cb81081f0 0x7f1cb819fd70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000b9e0 tx=0x7f1cb000bda0 comp rx=0 tx=0).stop 2024-06-06T03:29:46.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:46.066+0000 7f1cbdfee700 1 -- 172.21.15.104:0/701298480 shutdown_connections 2024-06-06T03:29:46.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:46.066+0000 7f1cbdfee700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1ca00a2b90 0x7f1ca00a5050 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:46.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:46.066+0000 7f1cbdfee700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cb8102f90 0x7f1cb81987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:46.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:46.066+0000 7f1cbdfee700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cb81081f0 0x7f1cb819fd70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:46.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:46.066+0000 7f1cbdfee700 1 --2- 172.21.15.104:0/701298480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb81058c0 0x7f1cb8198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:46.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:46.066+0000 7f1cbdfee700 1 -- 172.21.15.104:0/701298480 >> 172.21.15.104:0/701298480 conn(0x7f1cb80fc9c0 msgr2=0x7f1cb81038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:29:46.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:46.067+0000 7f1cbdfee700 1 -- 172.21.15.104:0/701298480 shutdown_connections 2024-06-06T03:29:46.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:29:46.067+0000 7f1cbdfee700 1 -- 172.21.15.104:0/701298480 wait complete. 2024-06-06T03:29:46.627 DEBUG:teuthology.orchestra.run.smithi104:osd.1> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.1.service 2024-06-06T03:29:46.629 INFO:tasks.cephadm:Deploying osd.2 on smithi131 with /dev/vg_nvme/lv_4... 2024-06-06T03:29:46.629 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T03:29:46.873 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:29:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:47 smithi131 ceph-mon[36687]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:47 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:47 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:47 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:47 smithi131 ceph-mon[36687]: from='osd.1 [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T03:29:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:47 smithi131 ceph-mon[36687]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T03:29:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:47 smithi040 ceph-mon[32220]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:47 smithi040 ceph-mon[32220]: from='osd.1 [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T03:29:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:47 smithi040 ceph-mon[32220]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T03:29:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:47 smithi104 ceph-mon[36569]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:47 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:47 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:47 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:29:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:47 smithi104 ceph-mon[36569]: from='osd.1 [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T03:29:47.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:47 smithi104 ceph-mon[36569]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T03:29:48.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:48 smithi104 ceph-mon[36569]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T03:29:48.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:48 smithi104 ceph-mon[36569]: from='osd.1 [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:29:48.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:48 smithi104 ceph-mon[36569]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T03:29:48.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:48.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:48 smithi104 ceph-mon[36569]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:29:48.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:48 smithi040 ceph-mon[32220]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T03:29:48.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:48 smithi040 ceph-mon[32220]: from='osd.1 [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:29:48.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:48 smithi040 ceph-mon[32220]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T03:29:48.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:48.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:48 smithi040 ceph-mon[32220]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:29:48.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:48 smithi131 ceph-mon[36687]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T03:29:48.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:48 smithi131 ceph-mon[36687]: from='osd.1 [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:29:48.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:48 smithi131 ceph-mon[36687]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T03:29:48.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:48.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:48 smithi131 ceph-mon[36687]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:29:48.617 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T03:29:48.617 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T03:29:48.617 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-06-06T03:29:48.617 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-06-06T03:29:48.617 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111708 s, 939 MB/s 2024-06-06T03:29:48.617 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-06-06T03:29:48.840 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:29:48 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-1[41142]: 2024-06-06T03:29:48.509+0000 7ff53d651700 -1 osd.1 0 waiting for initial osdmap 2024-06-06T03:29:48.841 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:29:48 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-1[41142]: 2024-06-06T03:29:48.512+0000 7ff53543c700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T03:29:49.194 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_4 2024-06-06T03:29:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:49 smithi131 ceph-mon[36687]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:49 smithi131 ceph-mon[36687]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-06-06T03:29:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:49 smithi131 ceph-mon[36687]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T03:29:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:49 smithi131 ceph-mon[36687]: from='osd.1 ' entity='osd.1' 2024-06-06T03:29:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:49 smithi040 ceph-mon[32220]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:49 smithi040 ceph-mon[32220]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-06-06T03:29:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:49 smithi040 ceph-mon[32220]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T03:29:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:49 smithi040 ceph-mon[32220]: from='osd.1 ' entity='osd.1' 2024-06-06T03:29:49.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:49 smithi104 ceph-mon[36569]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:49.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:49 smithi104 ceph-mon[36569]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-06-06T03:29:49.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:49 smithi104 ceph-mon[36569]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T03:29:49.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:49.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:49.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:49 smithi104 ceph-mon[36569]: from='osd.1 ' entity='osd.1' 2024-06-06T03:29:49.431 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:29:50.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:50 smithi040 ceph-mon[32220]: purged_snaps scrub starts 2024-06-06T03:29:50.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:50 smithi040 ceph-mon[32220]: purged_snaps scrub ok 2024-06-06T03:29:50.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:50.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:50 smithi040 ceph-mon[32220]: osd.1 [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252] boot 2024-06-06T03:29:50.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:50 smithi040 ceph-mon[32220]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T03:29:50.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:50.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:50.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:50.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:29:50.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:50.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:50.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:50.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:50.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:50 smithi104 ceph-mon[36569]: purged_snaps scrub starts 2024-06-06T03:29:50.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:50 smithi104 ceph-mon[36569]: purged_snaps scrub ok 2024-06-06T03:29:50.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:50.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:50 smithi104 ceph-mon[36569]: osd.1 [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252] boot 2024-06-06T03:29:50.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:50 smithi104 ceph-mon[36569]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T03:29:50.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:50.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:50.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:50.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:29:50.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:50.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:50.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:50.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:50.426 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:50 smithi131 ceph-mon[36687]: purged_snaps scrub starts 2024-06-06T03:29:50.426 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:50 smithi131 ceph-mon[36687]: purged_snaps scrub ok 2024-06-06T03:29:50.426 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:50.427 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:50 smithi131 ceph-mon[36687]: osd.1 [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252] boot 2024-06-06T03:29:50.427 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:50 smithi131 ceph-mon[36687]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T03:29:50.427 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:29:50.427 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:50.427 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:50.427 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:29:50.427 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:50.427 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:50.427 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:29:50.428 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:29:50.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.957+0000 7fc908e01700 1 -- 172.21.15.131:0/9318074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9040764c0 msgr2=0x7fc904074920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:29:50.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.957+0000 7fc908e01700 1 --2- 172.21.15.131:0/9318074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9040764c0 0x7fc904074920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009990 tx=0x7fc8f4009ca0 comp rx=0 tx=0).stop 2024-06-06T03:29:50.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.957+0000 7fc908e01700 1 -- 172.21.15.131:0/9318074 shutdown_connections 2024-06-06T03:29:50.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.957+0000 7fc908e01700 1 --2- 172.21.15.131:0/9318074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9040764c0 0x7fc904074920 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:50.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.957+0000 7fc908e01700 1 --2- 172.21.15.131:0/9318074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc90410a3b0 0x7fc90410c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:50.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.957+0000 7fc908e01700 1 --2- 172.21.15.131:0/9318074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc904074ef0 0x7fc904075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:50.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.957+0000 7fc908e01700 1 -- 172.21.15.131:0/9318074 >> 172.21.15.131:0/9318074 conn(0x7fc9040fda80 msgr2=0x7fc9040ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:29:50.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.957+0000 7fc908e01700 1 -- 172.21.15.131:0/9318074 shutdown_connections 2024-06-06T03:29:50.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.957+0000 7fc908e01700 1 -- 172.21.15.131:0/9318074 wait complete. 2024-06-06T03:29:50.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc908e01700 1 Processor -- start 2024-06-06T03:29:50.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc908e01700 1 -- start start 2024-06-06T03:29:50.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc908e01700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc904074ef0 0x7fc90419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:29:50.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc908e01700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9040764c0 0x7fc90419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:29:50.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc908e01700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc90410a3b0 0x7fc9041a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:29:50.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc908e01700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc9041102b0 con 0x7fc90410a3b0 2024-06-06T03:29:50.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc908e01700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc904110130 con 0x7fc904074ef0 2024-06-06T03:29:50.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc908e01700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc904110430 con 0x7fc9040764c0 2024-06-06T03:29:50.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc901d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9040764c0 0x7fc90419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:29:50.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc90259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc904074ef0 0x7fc90419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:29:50.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc901d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9040764c0 0x7fc90419ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49954/0 (socket says 172.21.15.131:49954) 2024-06-06T03:29:50.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc90259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc904074ef0 0x7fc90419a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.131:52800/0 (socket says 172.21.15.131:52800) 2024-06-06T03:29:50.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc901d9b700 1 -- 172.21.15.131:0/1708960023 learned_addr learned my addr 172.21.15.131:0/1708960023 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-06-06T03:29:50.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc902d9d700 1 --2- 172.21.15.131:0/1708960023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc90410a3b0 0x7fc9041a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:29:50.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc901d9b700 1 -- 172.21.15.131:0/1708960023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc904074ef0 msgr2=0x7fc90419a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:29:50.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc901d9b700 1 --2- 172.21.15.131:0/1708960023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc904074ef0 0x7fc90419a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:50.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc901d9b700 1 -- 172.21.15.131:0/1708960023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc90410a3b0 msgr2=0x7fc9041a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:29:50.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.958+0000 7fc901d9b700 1 --2- 172.21.15.131:0/1708960023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc90410a3b0 0x7fc9041a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:29:50.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.959+0000 7fc901d9b700 1 -- 172.21.15.131:0/1708960023 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9041a2510 con 0x7fc9040764c0 2024-06-06T03:29:50.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.959+0000 7fc901d9b700 1 --2- 172.21.15.131:0/1708960023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9040764c0 0x7fc90419ae40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800d9b0 tx=0x7fc8f800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:29:50.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.959+0000 7fc8f37fe700 1 -- 172.21.15.131:0/1708960023 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8017030 con 0x7fc9040764c0 2024-06-06T03:29:50.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.959+0000 7fc908e01700 1 -- 172.21.15.131:0/1708960023 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9041a2800 con 0x7fc9040764c0 2024-06-06T03:29:50.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.959+0000 7fc8f37fe700 1 -- 172.21.15.131:0/1708960023 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f8004500 con 0x7fc9040764c0 2024-06-06T03:29:50.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.959+0000 7fc8f37fe700 1 -- 172.21.15.131:0/1708960023 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8003970 con 0x7fc9040764c0 2024-06-06T03:29:50.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.959+0000 7fc908e01700 1 -- 172.21.15.131:0/1708960023 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9041a2d40 con 0x7fc9040764c0 2024-06-06T03:29:50.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.960+0000 7fc908e01700 1 -- 172.21.15.131:0/1708960023 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc904069180 con 0x7fc9040764c0 2024-06-06T03:29:50.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.961+0000 7fc8f37fe700 1 -- 172.21.15.131:0/1708960023 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fc8f801a070 con 0x7fc9040764c0 2024-06-06T03:29:50.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.962+0000 7fc8f37fe700 1 --2- 172.21.15.131:0/1708960023 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc8ec0a2b90 0x7fc8ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:29:50.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.962+0000 7fc8f37fe700 1 -- 172.21.15.131:0/1708960023 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7fc8f80cd3c0 con 0x7fc9040764c0 2024-06-06T03:29:50.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.962+0000 7fc90259c700 1 --2- 172.21.15.131:0/1708960023 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc8ec0a2b90 0x7fc8ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:29:50.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.963+0000 7fc90259c700 1 --2- 172.21.15.131:0/1708960023 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc8ec0a2b90 0x7fc8ec0a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4003890 tx=0x7fc8f4003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:29:50.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:50.966+0000 7fc8f37fe700 1 -- 172.21.15.131:0/1708960023 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc8f809bb70 con 0x7fc9040764c0 2024-06-06T03:29:51.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:29:51.169+0000 7fc908e01700 1 -- 172.21.15.131:0/1708960023 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc9040628d0 con 0x7fc8ec0a2b90 2024-06-06T03:29:51.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:51 smithi131 ceph-mon[36687]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:51.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:51 smithi131 ceph-mon[36687]: Detected new or changed devices on smithi104 2024-06-06T03:29:51.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:51 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi104 to 17088M 2024-06-06T03:29:51.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:51 smithi040 ceph-mon[32220]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:51.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:51 smithi040 ceph-mon[32220]: Detected new or changed devices on smithi104 2024-06-06T03:29:51.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:51 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi104 to 17088M 2024-06-06T03:29:51.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:51 smithi104 ceph-mon[36569]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T03:29:51.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:51 smithi104 ceph-mon[36569]: Detected new or changed devices on smithi104 2024-06-06T03:29:51.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:51 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi104 to 17088M 2024-06-06T03:29:52.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:52 smithi040 ceph-mon[32220]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T03:29:52.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:52 smithi040 ceph-mon[32220]: from='client.24139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:29:52.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:52 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:29:52.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:52 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:29:52.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:52 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:52.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:52 smithi104 ceph-mon[36569]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T03:29:52.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:52 smithi104 ceph-mon[36569]: from='client.24139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:29:52.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:52 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:29:52.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:52 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:29:52.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:52 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:52.408 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:52 smithi131 ceph-mon[36687]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T03:29:52.408 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:52 smithi131 ceph-mon[36687]: from='client.24139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:29:52.408 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:52 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:29:52.408 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:52 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:29:52.408 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:52 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:29:53.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:53 smithi131 ceph-mon[36687]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:29:53.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:53 smithi040 ceph-mon[32220]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:29:53.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:53 smithi104 ceph-mon[36569]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:29:54.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:54 smithi104 ceph-mon[36569]: from='client.? 172.21.15.131:0/107932742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5a89f43-e533-45f0-a777-bd3e9bfa8072"}]: dispatch 2024-06-06T03:29:54.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:54 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5a89f43-e533-45f0-a777-bd3e9bfa8072"}]: dispatch 2024-06-06T03:29:54.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:54 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5a89f43-e533-45f0-a777-bd3e9bfa8072"}]': finished 2024-06-06T03:29:54.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:54 smithi104 ceph-mon[36569]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T03:29:54.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:54 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:29:54.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:54 smithi131 ceph-mon[36687]: from='client.? 172.21.15.131:0/107932742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5a89f43-e533-45f0-a777-bd3e9bfa8072"}]: dispatch 2024-06-06T03:29:54.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:54 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5a89f43-e533-45f0-a777-bd3e9bfa8072"}]: dispatch 2024-06-06T03:29:54.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:54 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5a89f43-e533-45f0-a777-bd3e9bfa8072"}]': finished 2024-06-06T03:29:54.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:54 smithi131 ceph-mon[36687]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T03:29:54.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:54 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:29:54.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:54 smithi040 ceph-mon[32220]: from='client.? 172.21.15.131:0/107932742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5a89f43-e533-45f0-a777-bd3e9bfa8072"}]: dispatch 2024-06-06T03:29:54.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:54 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5a89f43-e533-45f0-a777-bd3e9bfa8072"}]: dispatch 2024-06-06T03:29:54.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:54 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5a89f43-e533-45f0-a777-bd3e9bfa8072"}]': finished 2024-06-06T03:29:54.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:54 smithi040 ceph-mon[32220]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T03:29:54.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:54 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:29:55.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:55 smithi104 ceph-mon[36569]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:29:55.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:55 smithi104 ceph-mon[36569]: from='client.? 172.21.15.131:0/1104982426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:29:55.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:55 smithi131 ceph-mon[36687]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:29:55.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:55 smithi131 ceph-mon[36687]: from='client.? 172.21.15.131:0/1104982426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:29:55.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:55 smithi040 ceph-mon[32220]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:29:55.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:55 smithi040 ceph-mon[32220]: from='client.? 172.21.15.131:0/1104982426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:29:57.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:57 smithi104 ceph-mon[36569]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:29:57.345 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:57 smithi131 ceph-mon[36687]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:29:57.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:57 smithi040 ceph-mon[32220]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:29:59.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:29:59 smithi131 ceph-mon[36687]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:29:59.414 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:29:59 smithi104 ceph-mon[36569]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:29:59.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:29:59 smithi040 ceph-mon[32220]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:00.167 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:00 smithi131 ceph-mon[36687]: overall HEALTH_OK 2024-06-06T03:30:00.352 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:00 smithi104 ceph-mon[36569]: overall HEALTH_OK 2024-06-06T03:30:00.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:00 smithi040 ceph-mon[32220]: overall HEALTH_OK 2024-06-06T03:30:01.396 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:01 smithi131 ceph-mon[36687]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:01.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:01 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T03:30:01.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:01 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:01.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:01 smithi040 ceph-mon[32220]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:01.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:01 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T03:30:01.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:01 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:01.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:01 smithi104 ceph-mon[36569]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:01.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:01 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T03:30:01.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:01 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:02.278 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:02 smithi131 ceph-mon[36687]: Deploying daemon osd.2 on smithi131 2024-06-06T03:30:02.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:02 smithi040 ceph-mon[32220]: Deploying daemon osd.2 on smithi131 2024-06-06T03:30:02.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:02 smithi104 ceph-mon[36569]: Deploying daemon osd.2 on smithi131 2024-06-06T03:30:03.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:03 smithi131 ceph-mon[36687]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:03.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:03 smithi040 ceph-mon[32220]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:03.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:03 smithi104 ceph-mon[36569]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:05.425 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:05 smithi131 ceph-mon[36687]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:05.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:05 smithi040 ceph-mon[32220]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:06.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:05 smithi104 ceph-mon[36569]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:07.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:06 smithi131 ceph-mon[36687]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:07.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:06 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:07.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:06 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:07.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:06 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:07.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:06 smithi040 ceph-mon[32220]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:07.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:06 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:07.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:06 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:07.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:06 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:06 smithi104 ceph-mon[36569]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:06 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:06 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:06 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:09.251 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:08 smithi131 ceph-mon[36687]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:09.251 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:09.251 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:09.251 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:09.251 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:30:09.252 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:09.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:08 smithi040 ceph-mon[32220]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:09.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:09.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:09.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:09.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:30:09.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:08 smithi104 ceph-mon[36569]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:30:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:09.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:30:09.585+0000 7fc8f37fe700 1 -- 172.21.15.131:0/1708960023 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc9040628d0 con 0x7fc8ec0a2b90 2024-06-06T03:30:09.585 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 2 on host 'smithi131' 2024-06-06T03:30:09.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:30:09.587+0000 7fc908e01700 1 -- 172.21.15.131:0/1708960023 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc8ec0a2b90 msgr2=0x7fc8ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:30:09.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:30:09.587+0000 7fc908e01700 1 --2- 172.21.15.131:0/1708960023 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc8ec0a2b90 0x7fc8ec0a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4003890 tx=0x7fc8f4003960 comp rx=0 tx=0).stop 2024-06-06T03:30:09.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:30:09.587+0000 7fc908e01700 1 -- 172.21.15.131:0/1708960023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9040764c0 msgr2=0x7fc90419ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:30:09.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:30:09.587+0000 7fc908e01700 1 --2- 172.21.15.131:0/1708960023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9040764c0 0x7fc90419ae40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800d9b0 tx=0x7fc8f800dcc0 comp rx=0 tx=0).stop 2024-06-06T03:30:09.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:30:09.587+0000 7fc908e01700 1 -- 172.21.15.131:0/1708960023 shutdown_connections 2024-06-06T03:30:09.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:30:09.587+0000 7fc908e01700 1 --2- 172.21.15.131:0/1708960023 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc8ec0a2b90 0x7fc8ec0a5050 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:09.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:30:09.587+0000 7fc908e01700 1 --2- 172.21.15.131:0/1708960023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc90410a3b0 0x7fc9041a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:09.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:30:09.587+0000 7fc908e01700 1 --2- 172.21.15.131:0/1708960023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc904074ef0 0x7fc90419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:09.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:30:09.587+0000 7fc908e01700 1 --2- 172.21.15.131:0/1708960023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9040764c0 0x7fc90419ae40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:09.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:30:09.587+0000 7fc908e01700 1 -- 172.21.15.131:0/1708960023 >> 172.21.15.131:0/1708960023 conn(0x7fc9040fda80 msgr2=0x7fc9041062b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:30:09.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:30:09.587+0000 7fc908e01700 1 -- 172.21.15.131:0/1708960023 shutdown_connections 2024-06-06T03:30:09.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:30:09.587+0000 7fc908e01700 1 -- 172.21.15.131:0/1708960023 wait complete. 2024-06-06T03:30:10.178 DEBUG:teuthology.orchestra.run.smithi131:osd.2> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.2.service 2024-06-06T03:30:10.181 INFO:tasks.cephadm:Deploying osd.3 on smithi040 with /dev/vg_nvme/lv_3... 2024-06-06T03:30:10.181 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T03:30:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:30:10.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:10 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:10.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:10 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:10.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:10 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:10.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:10 smithi131 ceph-mon[36687]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:10.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:10 smithi131 ceph-mon[36687]: from='osd.2 [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T03:30:10.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:10 smithi131 ceph-mon[36687]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T03:30:10.816 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:10 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:10.816 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:10 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:10.816 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:10 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:10.816 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:10 smithi040 ceph-mon[32220]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:10.816 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:10 smithi040 ceph-mon[32220]: from='osd.2 [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T03:30:10.816 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:10 smithi040 ceph-mon[32220]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T03:30:10.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:10 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:10.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:10 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:10.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:10 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:10.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:10 smithi104 ceph-mon[36569]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:10.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:10 smithi104 ceph-mon[36569]: from='osd.2 [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T03:30:10.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:10 smithi104 ceph-mon[36569]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T03:30:11.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:11 smithi131 ceph-mon[36687]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T03:30:11.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:11 smithi131 ceph-mon[36687]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T03:30:11.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:11 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:11.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:11 smithi131 ceph-mon[36687]: from='osd.2 [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:30:11.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:11 smithi131 ceph-mon[36687]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:30:11.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:11 smithi040 ceph-mon[32220]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T03:30:11.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:11 smithi040 ceph-mon[32220]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T03:30:11.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:11 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:11.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:11 smithi040 ceph-mon[32220]: from='osd.2 [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:30:11.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:11 smithi040 ceph-mon[32220]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:30:11.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:11 smithi104 ceph-mon[36569]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T03:30:11.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:11 smithi104 ceph-mon[36569]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T03:30:11.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:11 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:11.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:11 smithi104 ceph-mon[36569]: from='osd.2 [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:30:11.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:11 smithi104 ceph-mon[36569]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:30:12.282 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:30:11 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-2[41779]: 2024-06-06T03:30:11.987+0000 7fa293254700 -1 osd.2 0 waiting for initial osdmap 2024-06-06T03:30:12.282 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:30:11 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-2[41779]: 2024-06-06T03:30:11.992+0000 7fa28a83e700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T03:30:12.602 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T03:30:12.603 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T03:30:12.603 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-06-06T03:30:12.603 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-06-06T03:30:12.603 INFO:teuthology.orchestra.run.smithi040.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124753 s, 841 MB/s 2024-06-06T03:30:12.603 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 2024-06-06T03:30:12.603 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-06-06T03:30:12.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:12 smithi131 ceph-mon[36687]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-06-06T03:30:12.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:12 smithi131 ceph-mon[36687]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T03:30:12.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:12 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:12.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:12 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:12.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:12 smithi131 ceph-mon[36687]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:12.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:12 smithi131 ceph-mon[36687]: from='osd.2 ' entity='osd.2' 2024-06-06T03:30:12.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:12 smithi040 ceph-mon[32220]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-06-06T03:30:12.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:12 smithi040 ceph-mon[32220]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T03:30:12.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:12 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:12.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:12 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:12.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:12 smithi040 ceph-mon[32220]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:12.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:12 smithi040 ceph-mon[32220]: from='osd.2 ' entity='osd.2' 2024-06-06T03:30:12.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:12 smithi104 ceph-mon[36569]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-06-06T03:30:12.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:12 smithi104 ceph-mon[36569]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T03:30:12.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:12 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:12.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:12 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:12.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:12 smithi104 ceph-mon[36569]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T03:30:12.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:12 smithi104 ceph-mon[36569]: from='osd.2 ' entity='osd.2' 2024-06-06T03:30:13.196 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_3 2024-06-06T03:30:13.457 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:30:13.786 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: purged_snaps scrub starts 2024-06-06T03:30:13.786 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: purged_snaps scrub ok 2024-06-06T03:30:13.787 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:13.787 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: osd.2 [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141] boot 2024-06-06T03:30:13.787 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T03:30:13.787 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:13.787 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: Detected new or changed devices on smithi131 2024-06-06T03:30:13.787 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:13.787 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:13.787 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:30:13.787 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi131 to 17088M 2024-06-06T03:30:13.787 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:13.787 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:13.787 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:30:13.787 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:13 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:13.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: purged_snaps scrub starts 2024-06-06T03:30:13.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: purged_snaps scrub ok 2024-06-06T03:30:13.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:13.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: osd.2 [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141] boot 2024-06-06T03:30:13.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T03:30:13.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:13.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: Detected new or changed devices on smithi131 2024-06-06T03:30:13.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:13.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:13.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:30:13.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi131 to 17088M 2024-06-06T03:30:13.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:13.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:13.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:30:13.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:13 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:14.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: purged_snaps scrub starts 2024-06-06T03:30:14.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: purged_snaps scrub ok 2024-06-06T03:30:14.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:14.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: osd.2 [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141] boot 2024-06-06T03:30:14.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T03:30:14.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:30:14.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: Detected new or changed devices on smithi131 2024-06-06T03:30:14.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:14.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:14.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:30:14.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi131 to 17088M 2024-06-06T03:30:14.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:14.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:14.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:30:14.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:13 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:14.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:14 smithi040 ceph-mon[32220]: pgmap v81: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:14.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:14 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T03:30:15.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:14 smithi131 ceph-mon[36687]: pgmap v81: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:15.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:14 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T03:30:15.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.080+0000 7f6d9f21f700 1 -- 172.21.15.40:0/3417586101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d98103020 msgr2=0x7f6d98105410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:30:15.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.080+0000 7f6d9f21f700 1 --2- 172.21.15.40:0/3417586101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d98103020 0x7f6d98105410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009990 tx=0x7f6d8c009ca0 comp rx=0 tx=0).stop 2024-06-06T03:30:15.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.080+0000 7f6d9f21f700 1 -- 172.21.15.40:0/3417586101 shutdown_connections 2024-06-06T03:30:15.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.080+0000 7f6d9f21f700 1 --2- 172.21.15.40:0/3417586101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d98108280 0x7f6d9810a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:15.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.080+0000 7f6d9f21f700 1 --2- 172.21.15.40:0/3417586101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d98103020 0x7f6d98105410 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:15.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.080+0000 7f6d9f21f700 1 --2- 172.21.15.40:0/3417586101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d98105950 0x7f6d98107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:15.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.080+0000 7f6d9f21f700 1 -- 172.21.15.40:0/3417586101 >> 172.21.15.40:0/3417586101 conn(0x7f6d980fc990 msgr2=0x7f6d980fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:30:15.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.080+0000 7f6d9f21f700 1 -- 172.21.15.40:0/3417586101 shutdown_connections 2024-06-06T03:30:15.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.080+0000 7f6d9f21f700 1 -- 172.21.15.40:0/3417586101 wait complete. 2024-06-06T03:30:15.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.080+0000 7f6d9f21f700 1 Processor -- start 2024-06-06T03:30:15.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.080+0000 7f6d9f21f700 1 -- start start 2024-06-06T03:30:15.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d9f21f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d98103020 0x7f6d98102980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:30:15.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d9f21f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d98105950 0x7f6d98100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:30:15.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d9f21f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d98108280 0x7f6d98101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:30:15.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d9f21f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9810e120 con 0x7f6d98105950 2024-06-06T03:30:15.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d9f21f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9810dfa0 con 0x7f6d98103020 2024-06-06T03:30:15.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d9f21f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9810e2a0 con 0x7f6d98108280 2024-06-06T03:30:15.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d98105950 0x7f6d98100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:30:15.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d98105950 0x7f6d98100fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34580/0 (socket says 172.21.15.40:34580) 2024-06-06T03:30:15.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d97fff700 1 -- 172.21.15.40:0/1473463948 learned_addr learned my addr 172.21.15.40:0/1473463948 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:30:15.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d9cfbb700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d98103020 0x7f6d98102980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:30:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d9d7bc700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d98108280 0x7f6d98101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:30:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d97fff700 1 -- 172.21.15.40:0/1473463948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d98108280 msgr2=0x7f6d98101510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:30:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d97fff700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d98108280 0x7f6d98101510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d97fff700 1 -- 172.21.15.40:0/1473463948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d98103020 msgr2=0x7f6d98102980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:30:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d97fff700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d98103020 0x7f6d98102980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d97fff700 1 -- 172.21.15.40:0/1473463948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d98101dc0 con 0x7f6d98105950 2024-06-06T03:30:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d9d7bc700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d98108280 0x7f6d98101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:30:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d97fff700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d98105950 0x7f6d98100fd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400b530 tx=0x7f6d8400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:30:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d95ffb700 1 -- 172.21.15.40:0/1473463948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84015020 con 0x7f6d98105950 2024-06-06T03:30:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d95ffb700 1 -- 172.21.15.40:0/1473463948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d84011d10 con 0x7f6d98105950 2024-06-06T03:30:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d9f21f700 1 -- 172.21.15.40:0/1473463948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d981a28d0 con 0x7f6d98105950 2024-06-06T03:30:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.081+0000 7f6d95ffb700 1 -- 172.21.15.40:0/1473463948 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8400f2e0 con 0x7f6d98105950 2024-06-06T03:30:15.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.082+0000 7f6d9f21f700 1 -- 172.21.15.40:0/1473463948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d981a2e90 con 0x7f6d98105950 2024-06-06T03:30:15.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.083+0000 7f6d9f21f700 1 -- 172.21.15.40:0/1473463948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d98069180 con 0x7f6d98105950 2024-06-06T03:30:15.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.084+0000 7f6d95ffb700 1 -- 172.21.15.40:0/1473463948 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f6d84018020 con 0x7f6d98105950 2024-06-06T03:30:15.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.084+0000 7f6d95ffb700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6d880a2c10 0x7f6d880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:30:15.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.085+0000 7f6d9cfbb700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6d880a2c10 0x7f6d880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:30:15.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.085+0000 7f6d95ffb700 1 -- 172.21.15.40:0/1473463948 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f6d840cd3c0 con 0x7f6d98105950 2024-06-06T03:30:15.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.085+0000 7f6d9cfbb700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6d880a2c10 0x7f6d880a50d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009750 tx=0x7f6d8c003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:30:15.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.090+0000 7f6d95ffb700 1 -- 172.21.15.40:0/1473463948 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6d8401f020 con 0x7f6d98105950 2024-06-06T03:30:15.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:14 smithi104 ceph-mon[36569]: pgmap v81: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:15.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:14 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T03:30:15.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:15.303+0000 7f6d9f21f700 1 -- 172.21.15.40:0/1473463948 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6d980628d0 con 0x7f6d880a2c10 2024-06-06T03:30:15.957 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:15 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T03:30:15.957 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:15 smithi040 ceph-mon[32220]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T03:30:15.957 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:15 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T03:30:15.957 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:15 smithi040 ceph-mon[32220]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:30:15.957 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:15 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:30:15.957 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:15 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:30:15.957 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:15 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:16.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:15 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T03:30:16.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:15 smithi131 ceph-mon[36687]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T03:30:16.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:15 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T03:30:16.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:15 smithi131 ceph-mon[36687]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:30:16.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:15 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:30:16.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:15 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:30:16.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:15 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:16.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:15 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T03:30:16.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:15 smithi104 ceph-mon[36569]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T03:30:16.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:15 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T03:30:16.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:15 smithi104 ceph-mon[36569]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:30:16.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:15 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:30:16.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:15 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:30:16.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:15 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:16.214 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:30:16 smithi040 sudo[51409]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T03:30:16.214 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:30:16 smithi040 sudo[51409]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T03:30:16.215 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:30:16 smithi040 sudo[51409]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T03:30:16.215 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:30:16 smithi040 sudo[51409]: pam_unix(sudo:session): session closed for user root 2024-06-06T03:30:16.865 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:16 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T03:30:16.865 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:16 smithi104 ceph-mon[36569]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T03:30:16.865 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:16 smithi104 ceph-mon[36569]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:16.909 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:16 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T03:30:16.910 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:16 smithi040 ceph-mon[32220]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T03:30:16.910 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:16 smithi040 ceph-mon[32220]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:16.910 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:30:16 smithi040 sudo[51426]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T03:30:16.910 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:30:16 smithi040 sudo[51426]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T03:30:16.910 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:30:16 smithi040 sudo[51426]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T03:30:16.910 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:30:16 smithi040 sudo[51426]: pam_unix(sudo:session): session closed for user root 2024-06-06T03:30:17.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:16 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T03:30:17.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:16 smithi131 ceph-mon[36687]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T03:30:17.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:16 smithi131 ceph-mon[36687]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:17.117 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:30:16 smithi104 sudo[45246]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T03:30:17.118 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:30:16 smithi104 sudo[45246]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T03:30:17.118 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:30:16 smithi104 sudo[45246]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T03:30:17.118 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:30:16 smithi104 sudo[45246]: pam_unix(sudo:session): session closed for user root 2024-06-06T03:30:17.590 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:30:17 smithi104 sudo[45249]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T03:30:17.591 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:30:17 smithi104 sudo[45249]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T03:30:17.591 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:30:17 smithi104 sudo[45249]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T03:30:17.591 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:30:17 smithi104 sudo[45249]: pam_unix(sudo:session): session closed for user root 2024-06-06T03:30:17.636 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:17 smithi131 ceph-mon[36687]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T03:30:17.636 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:17 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:17.636 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:17 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:17.636 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:30:17 smithi131 sudo[45839]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T03:30:17.636 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:30:17 smithi131 sudo[45839]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T03:30:17.637 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:30:17 smithi131 sudo[45839]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T03:30:17.637 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:30:17 smithi131 sudo[45839]: pam_unix(sudo:session): session closed for user root 2024-06-06T03:30:18.032 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:30:17 smithi131 sudo[45842]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T03:30:18.032 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:30:17 smithi131 sudo[45842]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T03:30:18.032 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:30:17 smithi131 sudo[45842]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T03:30:18.032 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:30:17 smithi131 sudo[45842]: pam_unix(sudo:session): session closed for user root 2024-06-06T03:30:18.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:17 smithi040 ceph-mon[32220]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T03:30:18.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:18.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:18.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:17 smithi104 ceph-mon[36569]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T03:30:18.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:17 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:18.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:17 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:18.559 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:18 smithi040 sudo[51504]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T03:30:18.559 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:18 smithi040 sudo[51504]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T03:30:18.559 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:18 smithi040 sudo[51504]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T03:30:18.559 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:18 smithi040 sudo[51504]: pam_unix(sudo:session): session closed for user root 2024-06-06T03:30:18.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:18 smithi040 sudo[51508]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T03:30:18.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:18 smithi040 sudo[51508]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T03:30:18.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:18 smithi040 sudo[51508]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T03:30:18.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:18 smithi040 sudo[51508]: pam_unix(sudo:session): session closed for user root 2024-06-06T03:30:19.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:18 smithi131 sudo[45845]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T03:30:19.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:18 smithi131 sudo[45845]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T03:30:19.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:18 smithi131 sudo[45845]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T03:30:19.056 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:18 smithi131 sudo[45845]: pam_unix(sudo:session): session closed for user root 2024-06-06T03:30:19.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 sudo[45848]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T03:30:19.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 sudo[45848]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T03:30:19.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 sudo[45848]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T03:30:19.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 sudo[45848]: pam_unix(sudo:session): session closed for user root 2024-06-06T03:30:19.549 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 sudo[45252]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T03:30:19.549 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 sudo[45252]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T03:30:19.549 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 sudo[45252]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T03:30:19.550 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 sudo[45252]: pam_unix(sudo:session): session closed for user root 2024-06-06T03:30:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:19.562+0000 7f6d95ffb700 1 -- 172.21.15.40:0/1473463948 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6d840976b0 con 0x7f6d98105950 2024-06-06T03:30:19.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:19.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:19.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T03:30:19.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3722748205' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df19390-6378-4346-a2be-29bce1dd2380"}]: dispatch 2024-06-06T03:30:19.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T03:30:19.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:30:19.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df19390-6378-4346-a2be-29bce1dd2380"}]: dispatch 2024-06-06T03:30:19.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:30:19.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:30:19.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4df19390-6378-4346-a2be-29bce1dd2380"}]': finished 2024-06-06T03:30:19.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: osdmap e22: 4 total, 3 up, 4 in 2024-06-06T03:30:19.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:19.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:30:19.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:30:19.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:30:19.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 sudo[45255]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T03:30:19.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 sudo[45255]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T03:30:19.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 sudo[45255]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T03:30:19.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 sudo[45255]: pam_unix(sudo:session): session closed for user root 2024-06-06T03:30:19.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:19.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:19.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T03:30:19.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3722748205' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df19390-6378-4346-a2be-29bce1dd2380"}]: dispatch 2024-06-06T03:30:19.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T03:30:19.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:30:19.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df19390-6378-4346-a2be-29bce1dd2380"}]: dispatch 2024-06-06T03:30:19.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:30:19.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:30:19.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4df19390-6378-4346-a2be-29bce1dd2380"}]': finished 2024-06-06T03:30:19.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: osdmap e22: 4 total, 3 up, 4 in 2024-06-06T03:30:19.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:19.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:30:19.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:30:19.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:30:20.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:20.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:20.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T03:30:20.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3722748205' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df19390-6378-4346-a2be-29bce1dd2380"}]: dispatch 2024-06-06T03:30:20.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T03:30:20.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:30:20.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df19390-6378-4346-a2be-29bce1dd2380"}]: dispatch 2024-06-06T03:30:20.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:30:20.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:30:20.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4df19390-6378-4346-a2be-29bce1dd2380"}]': finished 2024-06-06T03:30:20.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: osdmap e22: 4 total, 3 up, 4 in 2024-06-06T03:30:20.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:20.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:30:20.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:30:20.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:30:20.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:20 smithi040 ceph-mon[32220]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T03:30:20.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:20 smithi040 ceph-mon[32220]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T03:30:20.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:20 smithi040 ceph-mon[32220]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T03:30:20.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:30:20.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:30:20.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:30:20.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:20 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1527530879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:30:20.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:20 smithi040 ceph-mon[32220]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T03:30:20.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:20 smithi040 ceph-mon[32220]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T03:30:20.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:20 smithi040 ceph-mon[32220]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:20.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:20 smithi104 ceph-mon[36569]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T03:30:20.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:20 smithi104 ceph-mon[36569]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T03:30:20.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:20 smithi104 ceph-mon[36569]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T03:30:20.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:30:20.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:30:20.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:30:20.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:20 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1527530879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:30:20.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:20 smithi104 ceph-mon[36569]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T03:30:20.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:20 smithi104 ceph-mon[36569]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T03:30:20.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:20 smithi104 ceph-mon[36569]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:20 smithi131 ceph-mon[36687]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T03:30:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:20 smithi131 ceph-mon[36687]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T03:30:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:20 smithi131 ceph-mon[36687]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T03:30:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:30:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:30:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:30:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:20 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1527530879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:30:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:20 smithi131 ceph-mon[36687]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T03:30:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:20 smithi131 ceph-mon[36687]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T03:30:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:20 smithi131 ceph-mon[36687]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:22.920 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:22 smithi040 ceph-mon[32220]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:23.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:22 smithi131 ceph-mon[36687]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:23.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:22 smithi104 ceph-mon[36569]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:24.967 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:24 smithi040 ceph-mon[32220]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:25.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:24 smithi131 ceph-mon[36687]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:25.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:24 smithi104 ceph-mon[36569]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:27.005 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:26 smithi040 ceph-mon[32220]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:27.005 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T03:30:27.006 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:27.006 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:26 smithi040 ceph-mon[32220]: Deploying daemon osd.3 on smithi040 2024-06-06T03:30:27.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:26 smithi131 ceph-mon[36687]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:27.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:26 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T03:30:27.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:26 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:27.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:26 smithi131 ceph-mon[36687]: Deploying daemon osd.3 on smithi040 2024-06-06T03:30:27.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:26 smithi104 ceph-mon[36569]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:27.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:26 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T03:30:27.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:26 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:27.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:26 smithi104 ceph-mon[36569]: Deploying daemon osd.3 on smithi040 2024-06-06T03:30:28.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:28 smithi040 ceph-mon[32220]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:29.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:28 smithi131 ceph-mon[36687]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:29.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:28 smithi104 ceph-mon[36569]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:30.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:30 smithi040 ceph-mon[32220]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:30.920 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:30 smithi104 ceph-mon[36569]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:31.015 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:30 smithi131 ceph-mon[36687]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:32 smithi040 ceph-mon[32220]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:32 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:32 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:32 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:33.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:32 smithi104 ceph-mon[36569]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:33.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:32 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:33.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:32 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:33.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:32 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:32 smithi131 ceph-mon[36687]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:35.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:34 smithi040 ceph-mon[32220]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:35.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:34 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:35.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:34 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:35.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:34 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:35.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:34 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:30:35.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:34 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:35.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:34 smithi104 ceph-mon[36569]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:35.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:34 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:35.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:34 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:35.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:34 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:35.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:34 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:30:35.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:34 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:34 smithi131 ceph-mon[36687]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:34 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:34 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:34 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:34 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:30:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:34 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:35.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:35.617+0000 7f6d95ffb700 1 -- 172.21.15.40:0/1473463948 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6d980628d0 con 0x7f6d880a2c10 2024-06-06T03:30:35.617 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 3 on host 'smithi040' 2024-06-06T03:30:35.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:35.620+0000 7f6d9f21f700 1 -- 172.21.15.40:0/1473463948 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6d880a2c10 msgr2=0x7f6d880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:30:35.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:35.620+0000 7f6d9f21f700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6d880a2c10 0x7f6d880a50d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009750 tx=0x7f6d8c003960 comp rx=0 tx=0).stop 2024-06-06T03:30:35.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:35.620+0000 7f6d9f21f700 1 -- 172.21.15.40:0/1473463948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d98105950 msgr2=0x7f6d98100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:30:35.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:35.620+0000 7f6d9f21f700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d98105950 0x7f6d98100fd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400b530 tx=0x7f6d8400b840 comp rx=0 tx=0).stop 2024-06-06T03:30:35.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:35.620+0000 7f6d9f21f700 1 -- 172.21.15.40:0/1473463948 shutdown_connections 2024-06-06T03:30:35.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:35.620+0000 7f6d9f21f700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6d880a2c10 0x7f6d880a50d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:35.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:35.620+0000 7f6d9f21f700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d98108280 0x7f6d98101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:35.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:35.620+0000 7f6d9f21f700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d98103020 0x7f6d98102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:35.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:35.620+0000 7f6d9f21f700 1 --2- 172.21.15.40:0/1473463948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d98105950 0x7f6d98100fd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:35.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:35.620+0000 7f6d9f21f700 1 -- 172.21.15.40:0/1473463948 >> 172.21.15.40:0/1473463948 conn(0x7f6d980fc990 msgr2=0x7f6d98103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:30:35.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:35.620+0000 7f6d9f21f700 1 -- 172.21.15.40:0/1473463948 shutdown_connections 2024-06-06T03:30:35.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:30:35.620+0000 7f6d9f21f700 1 -- 172.21.15.40:0/1473463948 wait complete. 2024-06-06T03:30:36.283 DEBUG:teuthology.orchestra.run.smithi040:osd.3> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.3.service 2024-06-06T03:30:36.285 INFO:tasks.cephadm:Deploying osd.4 on smithi104 with /dev/vg_nvme/lv_3... 2024-06-06T03:30:36.285 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T03:30:36.526 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T03:30:36.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:36 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:36.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:36 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:36.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:36 smithi040 ceph-mon[32220]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:36.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:36 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:36.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:36 smithi040 ceph-mon[32220]: from='osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T03:30:36.871 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:36 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:36.871 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:36 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:36.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:36 smithi104 ceph-mon[36569]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:36.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:36 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:36.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:36 smithi104 ceph-mon[36569]: from='osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T03:30:37.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:36 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:37.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:36 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:37.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:36 smithi131 ceph-mon[36687]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:37.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:36 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:37.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:36 smithi131 ceph-mon[36687]: from='osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T03:30:37.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:37 smithi040 ceph-mon[32220]: from='osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T03:30:37.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:37 smithi040 ceph-mon[32220]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T03:30:37.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:37 smithi040 ceph-mon[32220]: from='osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-06-06T03:30:37.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:37.830 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:30:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-3[53559]: 2024-06-06T03:30:37.630+0000 7fd96997e700 -1 osd.3 0 waiting for initial osdmap 2024-06-06T03:30:37.831 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:30:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-3[53559]: 2024-06-06T03:30:37.636+0000 7fd965771700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T03:30:37.945 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:37 smithi104 ceph-mon[36569]: from='osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T03:30:37.945 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:37 smithi104 ceph-mon[36569]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T03:30:37.945 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:37 smithi104 ceph-mon[36569]: from='osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-06-06T03:30:37.945 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:37 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:38.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:37 smithi131 ceph-mon[36687]: from='osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T03:30:38.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:37 smithi131 ceph-mon[36687]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T03:30:38.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:37 smithi131 ceph-mon[36687]: from='osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-06-06T03:30:38.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:38.508 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T03:30:38.509 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T03:30:38.509 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-06-06T03:30:38.509 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-06-06T03:30:38.509 INFO:teuthology.orchestra.run.smithi104.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113943 s, 920 MB/s 2024-06-06T03:30:38.509 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-06-06T03:30:38.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:38 smithi040 ceph-mon[32220]: from='osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-06-06T03:30:38.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:38 smithi040 ceph-mon[32220]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:38.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:38 smithi040 ceph-mon[32220]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T03:30:38.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:38 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:38.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:38 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:38.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:38 smithi040 ceph-mon[32220]: osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522] boot 2024-06-06T03:30:38.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:38 smithi040 ceph-mon[32220]: osdmap e25: 4 total, 4 up, 4 in 2024-06-06T03:30:38.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:38 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:38.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:38 smithi104 ceph-mon[36569]: from='osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-06-06T03:30:38.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:38 smithi104 ceph-mon[36569]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:38.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:38 smithi104 ceph-mon[36569]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T03:30:38.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:38 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:38.882 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:38 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:38.882 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:38 smithi104 ceph-mon[36569]: osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522] boot 2024-06-06T03:30:38.882 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:38 smithi104 ceph-mon[36569]: osdmap e25: 4 total, 4 up, 4 in 2024-06-06T03:30:38.882 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:38 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:39.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:38 smithi131 ceph-mon[36687]: from='osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-06-06T03:30:39.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:38 smithi131 ceph-mon[36687]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-06T03:30:39.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:38 smithi131 ceph-mon[36687]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T03:30:39.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:38 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:39.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:38 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:39.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:38 smithi131 ceph-mon[36687]: osd.3 [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522] boot 2024-06-06T03:30:39.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:38 smithi131 ceph-mon[36687]: osdmap e25: 4 total, 4 up, 4 in 2024-06-06T03:30:39.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:38 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:30:39.178 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_3 2024-06-06T03:30:39.413 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T03:30:39.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:39 smithi104 ceph-mon[36569]: purged_snaps scrub starts 2024-06-06T03:30:39.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:39 smithi104 ceph-mon[36569]: purged_snaps scrub ok 2024-06-06T03:30:39.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:39 smithi104 ceph-mon[36569]: Detected new or changed devices on smithi040 2024-06-06T03:30:39.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:39 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:39.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:39 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:39.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:39 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:30:39.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:39 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:30:39.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:39 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi040 to 8546M 2024-06-06T03:30:39.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:39 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:39.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:39 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:39.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:39 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:30:39.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:39 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:39.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:39 smithi104 ceph-mon[36569]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T03:30:40.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:39 smithi131 ceph-mon[36687]: purged_snaps scrub starts 2024-06-06T03:30:40.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:39 smithi131 ceph-mon[36687]: purged_snaps scrub ok 2024-06-06T03:30:40.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:39 smithi131 ceph-mon[36687]: Detected new or changed devices on smithi040 2024-06-06T03:30:40.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:39 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:40.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:39 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:40.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:39 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:30:40.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:39 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:30:40.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:39 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi040 to 8546M 2024-06-06T03:30:40.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:39 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:40.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:39 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:40.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:39 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:30:40.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:39 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:40.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:39 smithi131 ceph-mon[36687]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T03:30:40.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:39 smithi040 ceph-mon[32220]: purged_snaps scrub starts 2024-06-06T03:30:40.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:39 smithi040 ceph-mon[32220]: purged_snaps scrub ok 2024-06-06T03:30:40.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:39 smithi040 ceph-mon[32220]: Detected new or changed devices on smithi040 2024-06-06T03:30:40.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:39 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:40.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:39 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:40.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:39 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:30:40.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:39 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:30:40.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:39 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi040 to 8546M 2024-06-06T03:30:40.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:39 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:40.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:39 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:40.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:39 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:30:40.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:39 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:40.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:39 smithi040 ceph-mon[32220]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T03:30:40.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:40 smithi131 ceph-mon[36687]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:30:40.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:40 smithi131 ceph-mon[36687]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T03:30:40.900 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:40 smithi104 ceph-mon[36569]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:30:40.900 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:40 smithi104 ceph-mon[36569]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T03:30:41.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:40 smithi040 ceph-mon[32220]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:30:41.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:40 smithi040 ceph-mon[32220]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T03:30:41.143 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.128+0000 7f8c06767700 1 -- 172.21.15.104:0/3076962541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c00102f80 msgr2=0x7f8c00109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:30:41.143 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.128+0000 7f8c06767700 1 --2- 172.21.15.104:0/3076962541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c00102f80 0x7f8c00109830 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800b0d0 tx=0x7f8bf800b3e0 comp rx=0 tx=0).stop 2024-06-06T03:30:41.143 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.128+0000 7f8c06767700 1 -- 172.21.15.104:0/3076962541 shutdown_connections 2024-06-06T03:30:41.144 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.128+0000 7f8c06767700 1 --2- 172.21.15.104:0/3076962541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c001013e0 0x7f8c00101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:41.144 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.128+0000 7f8c06767700 1 --2- 172.21.15.104:0/3076962541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c00102f80 0x7f8c00109830 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:41.144 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.128+0000 7f8c06767700 1 --2- 172.21.15.104:0/3076962541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c001025e0 0x7f8c00102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:41.144 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.128+0000 7f8c06767700 1 -- 172.21.15.104:0/3076962541 >> 172.21.15.104:0/3076962541 conn(0x7f8c000fc980 msgr2=0x7f8c000fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:30:41.144 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.128+0000 7f8c06767700 1 -- 172.21.15.104:0/3076962541 shutdown_connections 2024-06-06T03:30:41.144 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.129+0000 7f8c06767700 1 -- 172.21.15.104:0/3076962541 wait complete. 2024-06-06T03:30:41.144 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.129+0000 7f8c06767700 1 Processor -- start 2024-06-06T03:30:41.144 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.129+0000 7f8c06767700 1 -- start start 2024-06-06T03:30:41.144 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c06767700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c001013e0 0x7f8c00197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:30:41.144 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c06767700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c001025e0 0x7f8c00197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:30:41.144 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c06767700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c00102f80 0x7f8c0019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:30:41.144 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c06767700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0010d000 con 0x7f8c001025e0 2024-06-06T03:30:41.145 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c06767700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0010ce80 con 0x7f8c00102f80 2024-06-06T03:30:41.145 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c06767700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0010d180 con 0x7f8c001013e0 2024-06-06T03:30:41.145 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c04d04700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c00102f80 0x7f8c0019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:30:41.145 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c04d04700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c00102f80 0x7f8c0019eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:45780/0 (socket says 172.21.15.104:45780) 2024-06-06T03:30:41.145 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c04d04700 1 -- 172.21.15.104:0/3571571316 learned_addr learned my addr 172.21.15.104:0/3571571316 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-06-06T03:30:41.145 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8bff7fe700 1 --2- 172.21.15.104:0/3571571316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c001025e0 0x7f8c00197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:30:41.145 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8bfffff700 1 --2- 172.21.15.104:0/3571571316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c001013e0 0x7f8c00197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:30:41.145 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c04d04700 1 -- 172.21.15.104:0/3571571316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c001013e0 msgr2=0x7f8c00197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:30:41.145 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c04d04700 1 --2- 172.21.15.104:0/3571571316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c001013e0 0x7f8c00197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:41.145 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c04d04700 1 -- 172.21.15.104:0/3571571316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c001025e0 msgr2=0x7f8c00197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:30:41.145 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c04d04700 1 --2- 172.21.15.104:0/3571571316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c001025e0 0x7f8c00197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:30:41.145 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c04d04700 1 -- 172.21.15.104:0/3571571316 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0019f180 con 0x7f8c00102f80 2024-06-06T03:30:41.146 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c04d04700 1 --2- 172.21.15.104:0/3571571316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c00102f80 0x7f8c0019eae0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8007fa0 tx=0x7f8bf8007d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:30:41.146 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8bfd7fa700 1 -- 172.21.15.104:0/3571571316 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf801f020 con 0x7f8c00102f80 2024-06-06T03:30:41.146 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8bfd7fa700 1 -- 172.21.15.104:0/3571571316 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf8003e10 con 0x7f8c00102f80 2024-06-06T03:30:41.146 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c06767700 1 -- 172.21.15.104:0/3571571316 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0019f410 con 0x7f8c00102f80 2024-06-06T03:30:41.146 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8bfd7fa700 1 -- 172.21.15.104:0/3571571316 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf801b7e0 con 0x7f8c00102f80 2024-06-06T03:30:41.146 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.130+0000 7f8c06767700 1 -- 172.21.15.104:0/3571571316 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0019f8f0 con 0x7f8c00102f80 2024-06-06T03:30:41.146 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.131+0000 7f8c06767700 1 -- 172.21.15.104:0/3571571316 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c00069180 con 0x7f8c00102f80 2024-06-06T03:30:41.146 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.133+0000 7f8bfd7fa700 1 -- 172.21.15.104:0/3571571316 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8bf8022020 con 0x7f8c00102f80 2024-06-06T03:30:41.146 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.133+0000 7f8bfd7fa700 1 --2- 172.21.15.104:0/3571571316 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8be80a2c20 0x7f8be80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:30:41.146 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.133+0000 7f8bfd7fa700 1 -- 172.21.15.104:0/3571571316 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f8bf800f020 con 0x7f8c00102f80 2024-06-06T03:30:41.146 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.133+0000 7f8bfffff700 1 --2- 172.21.15.104:0/3571571316 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8be80a2c20 0x7f8be80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:30:41.146 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.134+0000 7f8bfffff700 1 --2- 172.21.15.104:0/3571571316 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8be80a2c20 0x7f8be80a50e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0005fd0 tx=0x7f8bf0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:30:41.146 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.137+0000 7f8bfd7fa700 1 -- 172.21.15.104:0/3571571316 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8bf809d720 con 0x7f8c00102f80 2024-06-06T03:30:41.340 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:30:41.341+0000 7f8c06767700 1 -- 172.21.15.104:0/3571571316 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8c00062990 con 0x7f8be80a2c20 2024-06-06T03:30:41.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:41 smithi104 ceph-mon[36569]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:30:41.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:41 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:30:41.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:41 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:30:41.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:41 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:42.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:41 smithi131 ceph-mon[36687]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:30:42.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:41 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:30:42.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:41 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:30:42.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:41 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:42.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:41 smithi040 ceph-mon[32220]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:30:42.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:41 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:30:42.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:41 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:30:42.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:41 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:43.007 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:42 smithi104 ceph-mon[36569]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:30:43.007 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:42 smithi104 ceph-mon[36569]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T03:30:43.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:42 smithi131 ceph-mon[36687]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:30:43.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:42 smithi131 ceph-mon[36687]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T03:30:43.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:42 smithi040 ceph-mon[32220]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:30:43.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:42 smithi040 ceph-mon[32220]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T03:30:43.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:43 smithi104 ceph-mon[36569]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T03:30:44.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:43 smithi131 ceph-mon[36687]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T03:30:44.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:43 smithi040 ceph-mon[32220]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T03:30:44.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:44 smithi104 ceph-mon[36569]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:30:44.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:44 smithi104 ceph-mon[36569]: from='client.? 172.21.15.104:0/2459391517' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "508de408-1cb1-4423-8199-db6bde9034f3"}]: dispatch 2024-06-06T03:30:44.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:44 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "508de408-1cb1-4423-8199-db6bde9034f3"}]: dispatch 2024-06-06T03:30:44.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:44 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "508de408-1cb1-4423-8199-db6bde9034f3"}]': finished 2024-06-06T03:30:44.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:44 smithi104 ceph-mon[36569]: osdmap e30: 5 total, 4 up, 5 in 2024-06-06T03:30:44.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:44 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:30:45.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:44 smithi131 ceph-mon[36687]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:30:45.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:44 smithi131 ceph-mon[36687]: from='client.? 172.21.15.104:0/2459391517' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "508de408-1cb1-4423-8199-db6bde9034f3"}]: dispatch 2024-06-06T03:30:45.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:44 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "508de408-1cb1-4423-8199-db6bde9034f3"}]: dispatch 2024-06-06T03:30:45.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:44 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "508de408-1cb1-4423-8199-db6bde9034f3"}]': finished 2024-06-06T03:30:45.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:44 smithi131 ceph-mon[36687]: osdmap e30: 5 total, 4 up, 5 in 2024-06-06T03:30:45.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:44 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:30:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:44 smithi040 ceph-mon[32220]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:30:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:44 smithi040 ceph-mon[32220]: from='client.? 172.21.15.104:0/2459391517' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "508de408-1cb1-4423-8199-db6bde9034f3"}]: dispatch 2024-06-06T03:30:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:44 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "508de408-1cb1-4423-8199-db6bde9034f3"}]: dispatch 2024-06-06T03:30:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:44 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "508de408-1cb1-4423-8199-db6bde9034f3"}]': finished 2024-06-06T03:30:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:44 smithi040 ceph-mon[32220]: osdmap e30: 5 total, 4 up, 5 in 2024-06-06T03:30:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:44 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:30:46.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:45 smithi131 ceph-mon[36687]: from='client.? 172.21.15.104:0/4016879587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:30:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:45 smithi040 ceph-mon[32220]: from='client.? 172.21.15.104:0/4016879587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:30:46.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:45 smithi104 ceph-mon[36569]: from='client.? 172.21.15.104:0/4016879587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:30:47.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:46 smithi131 ceph-mon[36687]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-06-06T03:30:47.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:46 smithi040 ceph-mon[32220]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-06-06T03:30:47.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:46 smithi104 ceph-mon[36569]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-06-06T03:30:49.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:48 smithi131 ceph-mon[36687]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T03:30:49.058 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:48 smithi104 ceph-mon[36569]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T03:30:49.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:48 smithi040 ceph-mon[32220]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T03:30:51.013 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:50 smithi040 ceph-mon[32220]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T03:30:51.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:50 smithi131 ceph-mon[36687]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T03:30:51.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:50 smithi104 ceph-mon[36569]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T03:30:52.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:51 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T03:30:52.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:51 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:52.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:51 smithi131 ceph-mon[36687]: Deploying daemon osd.4 on smithi104 2024-06-06T03:30:52.046 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:51 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T03:30:52.046 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:51 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:52.046 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:51 smithi104 ceph-mon[36569]: Deploying daemon osd.4 on smithi104 2024-06-06T03:30:52.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:51 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T03:30:52.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:51 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:30:52.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:51 smithi040 ceph-mon[32220]: Deploying daemon osd.4 on smithi104 2024-06-06T03:30:53.009 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:52 smithi104 ceph-mon[36569]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-06T03:30:53.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:52 smithi131 ceph-mon[36687]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-06T03:30:53.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:52 smithi040 ceph-mon[32220]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-06T03:30:55.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:54 smithi131 ceph-mon[36687]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T03:30:55.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:54 smithi040 ceph-mon[32220]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T03:30:55.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:54 smithi104 ceph-mon[36569]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T03:30:56.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:56 smithi104 ceph-mon[36569]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-06-06T03:30:57.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:56 smithi131 ceph-mon[36687]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-06-06T03:30:57.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:56 smithi040 ceph-mon[32220]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-06-06T03:30:58.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:58 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:58.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:58 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:58.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:58 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:58.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:58 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:58.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:58 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:58.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:58 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:58.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:58 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:58.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:58 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:30:58.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:58 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:30:59.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:30:59 smithi104 ceph-mon[36569]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:30:59.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:30:59 smithi131 ceph-mon[36687]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:30:59.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:30:59 smithi040 ceph-mon[32220]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:31:00.657 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:00.658+0000 7f8bfd7fa700 1 -- 172.21.15.104:0/3571571316 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8c00062990 con 0x7f8be80a2c20 2024-06-06T03:31:00.658 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 4 on host 'smithi104' 2024-06-06T03:31:00.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:00.660+0000 7f8c06767700 1 -- 172.21.15.104:0/3571571316 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8be80a2c20 msgr2=0x7f8be80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:00.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:00.660+0000 7f8c06767700 1 --2- 172.21.15.104:0/3571571316 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8be80a2c20 0x7f8be80a50e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0005fd0 tx=0x7f8bf0005e50 comp rx=0 tx=0).stop 2024-06-06T03:31:00.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:00.660+0000 7f8c06767700 1 -- 172.21.15.104:0/3571571316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c00102f80 msgr2=0x7f8c0019eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:00.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:00.660+0000 7f8c06767700 1 --2- 172.21.15.104:0/3571571316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c00102f80 0x7f8c0019eae0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8007fa0 tx=0x7f8bf8007d10 comp rx=0 tx=0).stop 2024-06-06T03:31:00.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:00.660+0000 7f8c06767700 1 -- 172.21.15.104:0/3571571316 shutdown_connections 2024-06-06T03:31:00.661 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:00.660+0000 7f8c06767700 1 --2- 172.21.15.104:0/3571571316 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8be80a2c20 0x7f8be80a50e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:00.661 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:00.660+0000 7f8c06767700 1 --2- 172.21.15.104:0/3571571316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c001013e0 0x7f8c00197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:00.661 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:00.660+0000 7f8c06767700 1 --2- 172.21.15.104:0/3571571316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c00102f80 0x7f8c0019eae0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:00.661 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:00.660+0000 7f8c06767700 1 --2- 172.21.15.104:0/3571571316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c001025e0 0x7f8c00197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:00.661 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:00.660+0000 7f8c06767700 1 -- 172.21.15.104:0/3571571316 >> 172.21.15.104:0/3571571316 conn(0x7f8c000fc980 msgr2=0x7f8c00105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:31:00.661 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:00.660+0000 7f8c06767700 1 -- 172.21.15.104:0/3571571316 shutdown_connections 2024-06-06T03:31:00.661 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:00.661+0000 7f8c06767700 1 -- 172.21.15.104:0/3571571316 wait complete. 2024-06-06T03:31:01.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:01 smithi104 ceph-mon[36569]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:31:01.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:01 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:01.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:01 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:01.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:01 smithi104 ceph-mon[36569]: from='osd.4 [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T03:31:01.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:01 smithi104 ceph-mon[36569]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T03:31:01.367 DEBUG:teuthology.orchestra.run.smithi104:osd.4> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.4.service 2024-06-06T03:31:01.369 INFO:tasks.cephadm:Deploying osd.5 on smithi131 with /dev/vg_nvme/lv_3... 2024-06-06T03:31:01.369 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T03:31:01.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:01 smithi131 ceph-mon[36687]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:31:01.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:01 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:01.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:01 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:01.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:01 smithi131 ceph-mon[36687]: from='osd.4 [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T03:31:01.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:01 smithi131 ceph-mon[36687]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T03:31:01.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:01 smithi040 ceph-mon[32220]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:31:01.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:01 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:01.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:01 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:01.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:01 smithi040 ceph-mon[32220]: from='osd.4 [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T03:31:01.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:01 smithi040 ceph-mon[32220]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T03:31:01.614 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:31:02.657 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:02 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:02.657 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:02 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:02.657 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:02 smithi104 ceph-mon[36569]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T03:31:02.657 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:02 smithi104 ceph-mon[36569]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T03:31:02.657 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:02 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:31:02.657 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:02 smithi104 ceph-mon[36569]: from='osd.4 [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:31:02.657 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:02 smithi104 ceph-mon[36569]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:31:02.658 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:31:02 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4[48033]: 2024-06-06T03:31:02.652+0000 7f9fd9db3700 -1 osd.4 0 waiting for initial osdmap 2024-06-06T03:31:02.658 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:31:02 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4[48033]: 2024-06-06T03:31:02.657+0000 7f9fd5ba6700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T03:31:02.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:02 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:02.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:02 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:02.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:02 smithi131 ceph-mon[36687]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T03:31:02.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:02 smithi131 ceph-mon[36687]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T03:31:02.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:02 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:31:02.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:02 smithi131 ceph-mon[36687]: from='osd.4 [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:31:02.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:02 smithi131 ceph-mon[36687]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:31:02.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:02 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:02.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:02 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:02.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:02 smithi040 ceph-mon[32220]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T03:31:02.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:02 smithi040 ceph-mon[32220]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T03:31:02.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:02 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:31:02.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:02 smithi040 ceph-mon[32220]: from='osd.4 [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:31:02.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:02 smithi040 ceph-mon[32220]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:31:03.617 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T03:31:03.617 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T03:31:03.617 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-06-06T03:31:03.617 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-06-06T03:31:03.617 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113939 s, 920 MB/s 2024-06-06T03:31:03.617 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-06-06T03:31:03.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:03 smithi131 ceph-mon[36687]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:31:03.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:03 smithi131 ceph-mon[36687]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-06-06T03:31:03.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:03 smithi131 ceph-mon[36687]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T03:31:03.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:31:03.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:31:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:03 smithi040 ceph-mon[32220]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:31:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:03 smithi040 ceph-mon[32220]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-06-06T03:31:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:03 smithi040 ceph-mon[32220]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T03:31:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:31:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:31:03.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:03 smithi104 ceph-mon[36569]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:31:03.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:03 smithi104 ceph-mon[36569]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-06-06T03:31:03.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:03 smithi104 ceph-mon[36569]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T03:31:03.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:31:03.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:31:04.334 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_3 2024-06-06T03:31:04.614 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:31:04.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:04 smithi104 ceph-mon[36569]: purged_snaps scrub starts 2024-06-06T03:31:04.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:04 smithi104 ceph-mon[36569]: purged_snaps scrub ok 2024-06-06T03:31:04.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:04 smithi104 ceph-mon[36569]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:31:04.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:04 smithi104 ceph-mon[36569]: osd.4 [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847] boot 2024-06-06T03:31:04.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:04 smithi104 ceph-mon[36569]: osdmap e33: 5 total, 5 up, 5 in 2024-06-06T03:31:04.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:04 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:31:04.864 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:04 smithi131 ceph-mon[36687]: purged_snaps scrub starts 2024-06-06T03:31:04.864 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:04 smithi131 ceph-mon[36687]: purged_snaps scrub ok 2024-06-06T03:31:04.864 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:04 smithi131 ceph-mon[36687]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:31:04.865 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:04 smithi131 ceph-mon[36687]: osd.4 [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847] boot 2024-06-06T03:31:04.865 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:04 smithi131 ceph-mon[36687]: osdmap e33: 5 total, 5 up, 5 in 2024-06-06T03:31:04.865 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:04 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:31:05.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:04 smithi040 ceph-mon[32220]: purged_snaps scrub starts 2024-06-06T03:31:05.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:04 smithi040 ceph-mon[32220]: purged_snaps scrub ok 2024-06-06T03:31:05.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:04 smithi040 ceph-mon[32220]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T03:31:05.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:04 smithi040 ceph-mon[32220]: osd.4 [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847] boot 2024-06-06T03:31:05.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:04 smithi040 ceph-mon[32220]: osdmap e33: 5 total, 5 up, 5 in 2024-06-06T03:31:05.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:04 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:31:05.727 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T03:31:05.728 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: Detected new or changed devices on smithi104 2024-06-06T03:31:05.728 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:05.728 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:05.728 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:05.728 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:05.728 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi104 to 8544M 2024-06-06T03:31:05.728 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:05.728 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:05.728 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:05.728 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:05.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:05.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:05.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:05.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:05 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:06.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T03:31:06.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: Detected new or changed devices on smithi104 2024-06-06T03:31:06.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:06.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:06.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:06.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:06.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi104 to 8544M 2024-06-06T03:31:06.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:06.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:06.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:06.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:06.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:06.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:06.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:06.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:05 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:06.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T03:31:06.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: Detected new or changed devices on smithi104 2024-06-06T03:31:06.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:06.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:06.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:06.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:06.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi104 to 8544M 2024-06-06T03:31:06.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:06.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:06.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:06.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:06.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:06.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:06.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:06.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:05 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:06.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.266+0000 7fd590612700 1 -- 172.21.15.131:0/1257629580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd588104080 msgr2=0x7fd58810a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:06.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.266+0000 7fd590612700 1 --2- 172.21.15.131:0/1257629580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd588104080 0x7fd58810a930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd58000b0a0 tx=0x7fd58000b3b0 comp rx=0 tx=0).stop 2024-06-06T03:31:06.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.266+0000 7fd590612700 1 -- 172.21.15.131:0/1257629580 shutdown_connections 2024-06-06T03:31:06.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.266+0000 7fd590612700 1 --2- 172.21.15.131:0/1257629580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd588104080 0x7fd58810a930 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:06.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.266+0000 7fd590612700 1 --2- 172.21.15.131:0/1257629580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5881024e0 0x7fd588102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:06.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.266+0000 7fd590612700 1 --2- 172.21.15.131:0/1257629580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5881036e0 0x7fd588103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:06.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.266+0000 7fd590612700 1 -- 172.21.15.131:0/1257629580 >> 172.21.15.131:0/1257629580 conn(0x7fd5880fda80 msgr2=0x7fd5880ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:31:06.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.266+0000 7fd590612700 1 -- 172.21.15.131:0/1257629580 shutdown_connections 2024-06-06T03:31:06.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.266+0000 7fd590612700 1 -- 172.21.15.131:0/1257629580 wait complete. 2024-06-06T03:31:06.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.267+0000 7fd590612700 1 Processor -- start 2024-06-06T03:31:06.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.267+0000 7fd590612700 1 -- start start 2024-06-06T03:31:06.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.267+0000 7fd590612700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5881024e0 0x7fd5881986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:31:06.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.267+0000 7fd590612700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5881036e0 0x7fd588198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:31:06.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.267+0000 7fd590612700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd588104080 0x7fd58819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:31:06.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.267+0000 7fd590612700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd58810e0d0 con 0x7fd5881036e0 2024-06-06T03:31:06.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.267+0000 7fd590612700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd58810df50 con 0x7fd588104080 2024-06-06T03:31:06.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.267+0000 7fd590612700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd58810e250 con 0x7fd5881024e0 2024-06-06T03:31:06.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.267+0000 7fd58e3ae700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5881024e0 0x7fd5881986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:31:06.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.267+0000 7fd58e3ae700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5881024e0 0x7fd5881986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52220/0 (socket says 172.21.15.131:52220) 2024-06-06T03:31:06.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.267+0000 7fd58e3ae700 1 -- 172.21.15.131:0/2571518081 learned_addr learned my addr 172.21.15.131:0/2571518081 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-06-06T03:31:06.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.267+0000 7fd58ebaf700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd588104080 0x7fd58819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:31:06.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.267+0000 7fd58dbad700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5881036e0 0x7fd588198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:31:06.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.267+0000 7fd58e3ae700 1 -- 172.21.15.131:0/2571518081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd588104080 msgr2=0x7fd58819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:06.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.268+0000 7fd58e3ae700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd588104080 0x7fd58819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:06.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.268+0000 7fd58e3ae700 1 -- 172.21.15.131:0/2571518081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5881036e0 msgr2=0x7fd588198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:06.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.268+0000 7fd58e3ae700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5881036e0 0x7fd588198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:06.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.268+0000 7fd58e3ae700 1 -- 172.21.15.131:0/2571518081 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5881a0310 con 0x7fd5881024e0 2024-06-06T03:31:06.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.268+0000 7fd58ebaf700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd588104080 0x7fd58819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:31:06.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.268+0000 7fd58e3ae700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5881024e0 0x7fd5881986b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd57800e980 tx=0x7fd57800ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:31:06.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.268+0000 7fd57f7fe700 1 -- 172.21.15.131:0/2571518081 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57800cbd0 con 0x7fd5881024e0 2024-06-06T03:31:06.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.268+0000 7fd590612700 1 -- 172.21.15.131:0/2571518081 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5881a0600 con 0x7fd5881024e0 2024-06-06T03:31:06.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.268+0000 7fd57f7fe700 1 -- 172.21.15.131:0/2571518081 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd578004510 con 0x7fd5881024e0 2024-06-06T03:31:06.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.268+0000 7fd57f7fe700 1 -- 172.21.15.131:0/2571518081 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578010430 con 0x7fd5881024e0 2024-06-06T03:31:06.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.268+0000 7fd590612700 1 -- 172.21.15.131:0/2571518081 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5881a0b40 con 0x7fd5881024e0 2024-06-06T03:31:06.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.269+0000 7fd590612700 1 -- 172.21.15.131:0/2571518081 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd588069180 con 0x7fd5881024e0 2024-06-06T03:31:06.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.270+0000 7fd57f7fe700 1 -- 172.21.15.131:0/2571518081 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd578003fe0 con 0x7fd5881024e0 2024-06-06T03:31:06.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.270+0000 7fd57f7fe700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd5740a2ca0 0x7fd5740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:31:06.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.271+0000 7fd57f7fe700 1 -- 172.21.15.131:0/2571518081 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fd5780ce3b0 con 0x7fd5881024e0 2024-06-06T03:31:06.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.271+0000 7fd58dbad700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd5740a2ca0 0x7fd5740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:31:06.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.272+0000 7fd58dbad700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd5740a2ca0 0x7fd5740a5160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd584005950 tx=0x7fd5840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:31:06.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.274+0000 7fd57f7fe700 1 -- 172.21.15.131:0/2571518081 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd57809c380 con 0x7fd5881024e0 2024-06-06T03:31:06.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:06.475+0000 7fd590612700 1 -- 172.21.15.131:0/2571518081 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd5880628d0 con 0x7fd5740a2ca0 2024-06-06T03:31:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:07 smithi131 ceph-mon[36687]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:07 smithi131 ceph-mon[36687]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:31:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:07 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:31:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:07 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:31:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:07 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:07.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:07 smithi040 ceph-mon[32220]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:07.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:07 smithi040 ceph-mon[32220]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:31:07.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:07 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:31:07.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:07 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:31:07.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:07 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:07.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:07 smithi104 ceph-mon[36569]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:07.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:07 smithi104 ceph-mon[36569]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:31:07.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:07 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:31:07.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:07 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:31:07.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:07 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:09.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:09 smithi131 ceph-mon[36687]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:09.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:09 smithi040 ceph-mon[32220]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:09.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:09 smithi104 ceph-mon[36569]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:10.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:10 smithi131 ceph-mon[36687]: from='client.? 172.21.15.131:0/1000332299' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c0cec9-a8ad-4a5d-800b-8b891ca6d836"}]: dispatch 2024-06-06T03:31:10.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:10 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c0cec9-a8ad-4a5d-800b-8b891ca6d836"}]: dispatch 2024-06-06T03:31:10.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:10 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8c0cec9-a8ad-4a5d-800b-8b891ca6d836"}]': finished 2024-06-06T03:31:10.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:10 smithi131 ceph-mon[36687]: osdmap e35: 6 total, 5 up, 6 in 2024-06-06T03:31:10.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:10 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:10.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:10 smithi131 ceph-mon[36687]: from='client.? 172.21.15.131:0/1517314861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:31:10.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:10 smithi040 ceph-mon[32220]: from='client.? 172.21.15.131:0/1000332299' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c0cec9-a8ad-4a5d-800b-8b891ca6d836"}]: dispatch 2024-06-06T03:31:10.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:10 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c0cec9-a8ad-4a5d-800b-8b891ca6d836"}]: dispatch 2024-06-06T03:31:10.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:10 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8c0cec9-a8ad-4a5d-800b-8b891ca6d836"}]': finished 2024-06-06T03:31:10.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:10 smithi040 ceph-mon[32220]: osdmap e35: 6 total, 5 up, 6 in 2024-06-06T03:31:10.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:10 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:10.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:10 smithi040 ceph-mon[32220]: from='client.? 172.21.15.131:0/1517314861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:31:10.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:10 smithi104 ceph-mon[36569]: from='client.? 172.21.15.131:0/1000332299' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c0cec9-a8ad-4a5d-800b-8b891ca6d836"}]: dispatch 2024-06-06T03:31:10.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:10 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c0cec9-a8ad-4a5d-800b-8b891ca6d836"}]: dispatch 2024-06-06T03:31:10.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:10 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8c0cec9-a8ad-4a5d-800b-8b891ca6d836"}]': finished 2024-06-06T03:31:10.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:10 smithi104 ceph-mon[36569]: osdmap e35: 6 total, 5 up, 6 in 2024-06-06T03:31:10.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:10 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:10.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:10 smithi104 ceph-mon[36569]: from='client.? 172.21.15.131:0/1517314861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:31:11.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:11 smithi131 ceph-mon[36687]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:11.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:11 smithi040 ceph-mon[32220]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:11.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:11 smithi104 ceph-mon[36569]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:13.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:13 smithi131 ceph-mon[36687]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:13.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:13 smithi040 ceph-mon[32220]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:13.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:13 smithi104 ceph-mon[36569]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:15.307 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:15 smithi131 ceph-mon[36687]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:15.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:15 smithi040 ceph-mon[32220]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:15.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:15 smithi104 ceph-mon[36569]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:17.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:17 smithi131 ceph-mon[36687]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:17.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:17 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T03:31:17.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:17 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:17 smithi040 ceph-mon[32220]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T03:31:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:17.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:17 smithi104 ceph-mon[36569]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:17.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:17 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T03:31:17.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:17 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:18.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:18 smithi131 ceph-mon[36687]: Deploying daemon osd.5 on smithi131 2024-06-06T03:31:18.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:18 smithi040 ceph-mon[32220]: Deploying daemon osd.5 on smithi131 2024-06-06T03:31:18.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:18 smithi104 ceph-mon[36569]: Deploying daemon osd.5 on smithi131 2024-06-06T03:31:19.507 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:19 smithi131 ceph-mon[36687]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:19 smithi040 ceph-mon[32220]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:19.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:19 smithi104 ceph-mon[36569]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:21.493 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:21 smithi131 ceph-mon[36687]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:21.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:21 smithi040 ceph-mon[32220]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:21.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:21 smithi104 ceph-mon[36569]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:22.481 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:22 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:22.481 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:22 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:22.481 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:22 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:22.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:22 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:22.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:22 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:22.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:22 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:22.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:22 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:22.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:22 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:22.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:22 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:23.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:23 smithi131 ceph-mon[36687]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:23.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:23 smithi040 ceph-mon[32220]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:23.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:23 smithi104 ceph-mon[36569]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:25 smithi131 ceph-mon[36687]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:25 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:25 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:25 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:25 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:25 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:25.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:25 smithi040 ceph-mon[32220]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:25.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:25 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:25.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:25 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:25.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:25 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:25.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:25 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:25.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:25 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:25.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:25 smithi104 ceph-mon[36569]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:25.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:25 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:25.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:25 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:25.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:25 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:25.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:25 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:25.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:25 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:25.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:25.948+0000 7fd57f7fe700 1 -- 172.21.15.131:0/2571518081 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd5880628d0 con 0x7fd5740a2ca0 2024-06-06T03:31:25.948 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 5 on host 'smithi131' 2024-06-06T03:31:25.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:25.952+0000 7fd590612700 1 -- 172.21.15.131:0/2571518081 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd5740a2ca0 msgr2=0x7fd5740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:25.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:25.952+0000 7fd590612700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd5740a2ca0 0x7fd5740a5160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd584005950 tx=0x7fd5840058e0 comp rx=0 tx=0).stop 2024-06-06T03:31:25.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:25.952+0000 7fd590612700 1 -- 172.21.15.131:0/2571518081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5881024e0 msgr2=0x7fd5881986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:25.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:25.952+0000 7fd590612700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5881024e0 0x7fd5881986b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd57800e980 tx=0x7fd57800ed40 comp rx=0 tx=0).stop 2024-06-06T03:31:25.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:25.952+0000 7fd590612700 1 -- 172.21.15.131:0/2571518081 shutdown_connections 2024-06-06T03:31:25.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:25.952+0000 7fd590612700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd5740a2ca0 0x7fd5740a5160 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:25.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:25.952+0000 7fd590612700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5881024e0 0x7fd5881986b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:25.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:25.952+0000 7fd590612700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd588104080 0x7fd58819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:25.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:25.952+0000 7fd590612700 1 --2- 172.21.15.131:0/2571518081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5881036e0 0x7fd588198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:25.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:25.952+0000 7fd590612700 1 -- 172.21.15.131:0/2571518081 >> 172.21.15.131:0/2571518081 conn(0x7fd5880fda80 msgr2=0x7fd588106980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:31:25.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:25.952+0000 7fd590612700 1 -- 172.21.15.131:0/2571518081 shutdown_connections 2024-06-06T03:31:25.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:31:25.952+0000 7fd590612700 1 -- 172.21.15.131:0/2571518081 wait complete. 2024-06-06T03:31:26.614 DEBUG:teuthology.orchestra.run.smithi131:osd.5> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.5.service 2024-06-06T03:31:26.617 INFO:tasks.cephadm:Deploying osd.6 on smithi040 with /dev/vg_nvme/lv_2... 2024-06-06T03:31:26.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T03:31:26.873 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:31:27.221 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:26 smithi040 ceph-mon[32220]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:27.221 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:27.221 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:27.221 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:26 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:27.221 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:26 smithi040 ceph-mon[32220]: from='osd.5 [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T03:31:27.221 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:26 smithi040 ceph-mon[32220]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T03:31:27.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:26 smithi131 ceph-mon[36687]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:27.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:26 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:27.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:26 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:27.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:26 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:27.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:26 smithi131 ceph-mon[36687]: from='osd.5 [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T03:31:27.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:26 smithi131 ceph-mon[36687]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T03:31:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:26 smithi104 ceph-mon[36569]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:26 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:26 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:26 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:26 smithi104 ceph-mon[36569]: from='osd.5 [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T03:31:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:26 smithi104 ceph-mon[36569]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T03:31:28.113 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:27 smithi131 ceph-mon[36687]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T03:31:28.113 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:27 smithi131 ceph-mon[36687]: from='osd.5 [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:31:28.113 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:27 smithi131 ceph-mon[36687]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T03:31:28.113 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:27 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:28.113 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:27 smithi131 ceph-mon[36687]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:31:28.205 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:27 smithi040 ceph-mon[32220]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T03:31:28.205 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:27 smithi040 ceph-mon[32220]: from='osd.5 [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:31:28.205 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:27 smithi040 ceph-mon[32220]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T03:31:28.205 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:27 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:28.205 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:27 smithi040 ceph-mon[32220]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:31:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:27 smithi104 ceph-mon[36569]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T03:31:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:27 smithi104 ceph-mon[36569]: from='osd.5 [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:31:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:27 smithi104 ceph-mon[36569]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T03:31:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:27 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:27 smithi104 ceph-mon[36569]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:31:28.370 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:31:28 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:31:28.361+0000 7f3a2e640700 -1 osd.5 0 waiting for initial osdmap 2024-06-06T03:31:28.370 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:31:28 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:31:28.371+0000 7f3a25c2a700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T03:31:28.965 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T03:31:28.965 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T03:31:28.965 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-06-06T03:31:28.965 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-06-06T03:31:28.965 INFO:teuthology.orchestra.run.smithi040.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109985 s, 953 MB/s 2024-06-06T03:31:28.965 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-06-06T03:31:29.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:28 smithi131 ceph-mon[36687]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:29.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:28 smithi131 ceph-mon[36687]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-06-06T03:31:29.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:28 smithi131 ceph-mon[36687]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T03:31:29.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:28 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:29.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:28 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:29.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:28 smithi131 ceph-mon[36687]: from='osd.5 ' entity='osd.5' 2024-06-06T03:31:29.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:28 smithi131 ceph-mon[36687]: osd.5 [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303] boot 2024-06-06T03:31:29.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:28 smithi131 ceph-mon[36687]: osdmap e38: 6 total, 6 up, 6 in 2024-06-06T03:31:29.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:28 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:29.299 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:28 smithi040 ceph-mon[32220]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:29.299 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:28 smithi040 ceph-mon[32220]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-06-06T03:31:29.299 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:28 smithi040 ceph-mon[32220]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T03:31:29.299 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:28 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:29.299 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:28 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:29.299 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:28 smithi040 ceph-mon[32220]: from='osd.5 ' entity='osd.5' 2024-06-06T03:31:29.299 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:28 smithi040 ceph-mon[32220]: osd.5 [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303] boot 2024-06-06T03:31:29.300 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:28 smithi040 ceph-mon[32220]: osdmap e38: 6 total, 6 up, 6 in 2024-06-06T03:31:29.300 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:28 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:29.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:28 smithi104 ceph-mon[36569]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T03:31:29.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:28 smithi104 ceph-mon[36569]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-06-06T03:31:29.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:28 smithi104 ceph-mon[36569]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T03:31:29.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:28 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:29.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:28 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:29.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:28 smithi104 ceph-mon[36569]: from='osd.5 ' entity='osd.5' 2024-06-06T03:31:29.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:28 smithi104 ceph-mon[36569]: osd.5 [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303] boot 2024-06-06T03:31:29.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:28 smithi104 ceph-mon[36569]: osdmap e38: 6 total, 6 up, 6 in 2024-06-06T03:31:29.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:28 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:31:29.623 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_2 2024-06-06T03:31:29.882 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:31:30.009 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:29 smithi131 ceph-mon[36687]: purged_snaps scrub starts 2024-06-06T03:31:30.009 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:29 smithi131 ceph-mon[36687]: purged_snaps scrub ok 2024-06-06T03:31:30.009 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:29 smithi131 ceph-mon[36687]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T03:31:30.009 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:29 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:30.009 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:29 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:30.009 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:29 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:30.009 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:29 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:30.010 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:29 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:30.010 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:29 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:30.010 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:29 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:30.010 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:29 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:30.242 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:29 smithi104 ceph-mon[36569]: purged_snaps scrub starts 2024-06-06T03:31:30.242 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:29 smithi104 ceph-mon[36569]: purged_snaps scrub ok 2024-06-06T03:31:30.242 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:29 smithi104 ceph-mon[36569]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T03:31:30.242 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:29 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:30.242 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:29 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:30.242 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:29 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:30.242 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:29 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:30.242 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:29 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:30.242 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:29 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:30.242 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:29 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:30.242 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:29 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:30.262 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:29 smithi040 ceph-mon[32220]: purged_snaps scrub starts 2024-06-06T03:31:30.262 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:29 smithi040 ceph-mon[32220]: purged_snaps scrub ok 2024-06-06T03:31:30.262 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:29 smithi040 ceph-mon[32220]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T03:31:30.263 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:29 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:30.263 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:29 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:30.263 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:29 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:30.263 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:29 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:30.263 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:29 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:30.263 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:29 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:30.263 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:29 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:30.263 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:29 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:31.008 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:30 smithi040 ceph-mon[32220]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:31.008 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:30 smithi040 ceph-mon[32220]: Detected new or changed devices on smithi131 2024-06-06T03:31:31.008 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:30 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi131 to 8544M 2024-06-06T03:31:31.008 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:30 smithi040 ceph-mon[32220]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T03:31:31.253 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:30 smithi104 ceph-mon[36569]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:31.253 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:30 smithi104 ceph-mon[36569]: Detected new or changed devices on smithi131 2024-06-06T03:31:31.253 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:30 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi131 to 8544M 2024-06-06T03:31:31.253 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:30 smithi104 ceph-mon[36569]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T03:31:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:30 smithi131 ceph-mon[36687]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:30 smithi131 ceph-mon[36687]: Detected new or changed devices on smithi131 2024-06-06T03:31:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:30 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi131 to 8544M 2024-06-06T03:31:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:30 smithi131 ceph-mon[36687]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T03:31:31.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.579+0000 7f0affbb5700 1 -- 172.21.15.40:0/3201072374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af81013f0 msgr2=0x7f0af8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:31.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.579+0000 7f0affbb5700 1 --2- 172.21.15.40:0/3201072374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af81013f0 0x7f0af8101810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0af40099c0 tx=0x7f0af4009cd0 comp rx=0 tx=0).stop 2024-06-06T03:31:31.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.579+0000 7f0affbb5700 1 -- 172.21.15.40:0/3201072374 shutdown_connections 2024-06-06T03:31:31.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.579+0000 7f0affbb5700 1 --2- 172.21.15.40:0/3201072374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0af8102f90 0x7f0af8109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:31.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.579+0000 7f0affbb5700 1 --2- 172.21.15.40:0/3201072374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af81013f0 0x7f0af8101810 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:31.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.579+0000 7f0affbb5700 1 --2- 172.21.15.40:0/3201072374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af81025f0 0x7f0af8102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:31.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.579+0000 7f0affbb5700 1 -- 172.21.15.40:0/3201072374 >> 172.21.15.40:0/3201072374 conn(0x7f0af80fc990 msgr2=0x7f0af80fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:31:31.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.579+0000 7f0affbb5700 1 -- 172.21.15.40:0/3201072374 shutdown_connections 2024-06-06T03:31:31.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.579+0000 7f0affbb5700 1 -- 172.21.15.40:0/3201072374 wait complete. 2024-06-06T03:31:31.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.579+0000 7f0affbb5700 1 Processor -- start 2024-06-06T03:31:31.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0affbb5700 1 -- start start 2024-06-06T03:31:31.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0affbb5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0af81013f0 0x7f0af806acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:31:31.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0affbb5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af81025f0 0x7f0af819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:31:31.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0affbb5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af8102f90 0x7f0af819fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:31:31.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0affbb5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0af810cfd0 con 0x7f0af8102f90 2024-06-06T03:31:31.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0affbb5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0af810ce50 con 0x7f0af81025f0 2024-06-06T03:31:31.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0affbb5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0af810d150 con 0x7f0af81013f0 2024-06-06T03:31:31.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0afe152700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af8102f90 0x7f0af819fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:31:31.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0afd951700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0af81013f0 0x7f0af806acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:31:31.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0afd150700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af81025f0 0x7f0af819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:31:31.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0afd951700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0af81013f0 0x7f0af806acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.40:59198/0 (socket says 172.21.15.40:59198) 2024-06-06T03:31:31.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0afe152700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af8102f90 0x7f0af819fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38028/0 (socket says 172.21.15.40:38028) 2024-06-06T03:31:31.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0afd951700 1 -- 172.21.15.40:0/508207705 learned_addr learned my addr 172.21.15.40:0/508207705 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:31:31.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0afe152700 1 -- 172.21.15.40:0/508207705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0af81013f0 msgr2=0x7f0af806acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:31.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0afe152700 1 --2- 172.21.15.40:0/508207705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0af81013f0 0x7f0af806acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:31.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0afe152700 1 -- 172.21.15.40:0/508207705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af81025f0 msgr2=0x7f0af819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:31.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0afe152700 1 --2- 172.21.15.40:0/508207705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af81025f0 0x7f0af819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:31.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0afe152700 1 -- 172.21.15.40:0/508207705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af81a03e0 con 0x7f0af8102f90 2024-06-06T03:31:31.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0afd150700 1 --2- 172.21.15.40:0/508207705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af81025f0 0x7f0af819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:31:31.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.580+0000 7f0afe152700 1 --2- 172.21.15.40:0/508207705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af8102f90 0x7f0af819fb10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0af000efc0 tx=0x7f0af000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:31:31.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.581+0000 7f0aeeffd700 1 -- 172.21.15.40:0/508207705 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af00092e0 con 0x7f0af8102f90 2024-06-06T03:31:31.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.581+0000 7f0aeeffd700 1 -- 172.21.15.40:0/508207705 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af0004bb0 con 0x7f0af8102f90 2024-06-06T03:31:31.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.581+0000 7f0affbb5700 1 -- 172.21.15.40:0/508207705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af81a0670 con 0x7f0af8102f90 2024-06-06T03:31:31.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.581+0000 7f0aeeffd700 1 -- 172.21.15.40:0/508207705 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af0004280 con 0x7f0af8102f90 2024-06-06T03:31:31.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.581+0000 7f0affbb5700 1 -- 172.21.15.40:0/508207705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af81a0c00 con 0x7f0af8102f90 2024-06-06T03:31:31.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.582+0000 7f0affbb5700 1 -- 172.21.15.40:0/508207705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af8069180 con 0x7f0af8102f90 2024-06-06T03:31:31.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.583+0000 7f0aeeffd700 1 -- 172.21.15.40:0/508207705 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0af001b030 con 0x7f0af8102f90 2024-06-06T03:31:31.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.583+0000 7f0aeeffd700 1 --2- 172.21.15.40:0/508207705 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0ae40a2ca0 0x7f0ae40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:31:31.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.584+0000 7f0afd951700 1 --2- 172.21.15.40:0/508207705 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0ae40a2ca0 0x7f0ae40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:31:31.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.584+0000 7f0aeeffd700 1 -- 172.21.15.40:0/508207705 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f0af00cdc00 con 0x7f0af8102f90 2024-06-06T03:31:31.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.584+0000 7f0afd951700 1 --2- 172.21.15.40:0/508207705 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0ae40a2ca0 0x7f0ae40a5160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0af4000c00 tx=0x7f0af4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:31:31.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.588+0000 7f0aeeffd700 1 -- 172.21.15.40:0/508207705 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0af00d4080 con 0x7f0af8102f90 2024-06-06T03:31:31.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:31.811+0000 7f0affbb5700 1 -- 172.21.15.40:0/508207705 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0af80628d0 con 0x7f0ae40a2ca0 2024-06-06T03:31:32.696 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:32 smithi040 ceph-mon[32220]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T03:31:32.696 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:32 smithi040 ceph-mon[32220]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:32.697 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:32 smithi040 ceph-mon[32220]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:31:32.697 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:32 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:31:32.697 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:32 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:31:32.697 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:32 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:32.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:32 smithi104 ceph-mon[36569]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T03:31:32.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:32 smithi104 ceph-mon[36569]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:32.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:32 smithi104 ceph-mon[36569]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:31:32.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:32 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:31:32.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:32 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:31:32.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:32 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:33.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:32 smithi131 ceph-mon[36687]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T03:31:33.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:32 smithi131 ceph-mon[36687]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:33.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:32 smithi131 ceph-mon[36687]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:31:33.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:31:33.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:31:33.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:33.771 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:33 smithi040 ceph-mon[32220]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T03:31:34.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:33 smithi131 ceph-mon[36687]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T03:31:34.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:33 smithi104 ceph-mon[36569]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T03:31:34.957 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:34 smithi040 ceph-mon[32220]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:35.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:34 smithi131 ceph-mon[36687]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:35.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:34 smithi104 ceph-mon[36569]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:36.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:35 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1572106927' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75"}]: dispatch 2024-06-06T03:31:36.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:35 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1572106927' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75"}]': finished 2024-06-06T03:31:36.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:35 smithi131 ceph-mon[36687]: osdmap e43: 7 total, 6 up, 7 in 2024-06-06T03:31:36.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:35 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:36.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:35 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/72933718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:31:36.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:35 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1572106927' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75"}]: dispatch 2024-06-06T03:31:36.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:35 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1572106927' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75"}]': finished 2024-06-06T03:31:36.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:35 smithi040 ceph-mon[32220]: osdmap e43: 7 total, 6 up, 7 in 2024-06-06T03:31:36.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:35 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:36.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:35 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/72933718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:31:36.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:35 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1572106927' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75"}]: dispatch 2024-06-06T03:31:36.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:35 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1572106927' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75"}]': finished 2024-06-06T03:31:36.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:35 smithi104 ceph-mon[36569]: osdmap e43: 7 total, 6 up, 7 in 2024-06-06T03:31:36.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:35 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:36.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:35 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/72933718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:31:37.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:36 smithi131 ceph-mon[36687]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-06-06T03:31:37.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:36 smithi040 ceph-mon[32220]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-06-06T03:31:37.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:36 smithi104 ceph-mon[36569]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-06-06T03:31:39.014 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:38 smithi040 ceph-mon[32220]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-06-06T03:31:39.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:38 smithi131 ceph-mon[36687]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-06-06T03:31:39.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:38 smithi104 ceph-mon[36569]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-06-06T03:31:40.979 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:40 smithi040 ceph-mon[32220]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T03:31:41.003 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:40 smithi131 ceph-mon[36687]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T03:31:41.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:40 smithi104 ceph-mon[36569]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T03:31:42.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:42 smithi104 ceph-mon[36569]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-06T03:31:42.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T03:31:42.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:42.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:42 smithi104 ceph-mon[36569]: Deploying daemon osd.6 on smithi040 2024-06-06T03:31:42.978 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:42 smithi040 ceph-mon[32220]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-06T03:31:42.978 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T03:31:42.978 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:42.978 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:42 smithi040 ceph-mon[32220]: Deploying daemon osd.6 on smithi040 2024-06-06T03:31:43.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:42 smithi131 ceph-mon[36687]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-06T03:31:43.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T03:31:43.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:43.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:42 smithi131 ceph-mon[36687]: Deploying daemon osd.6 on smithi040 2024-06-06T03:31:44.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:44 smithi040 ceph-mon[32220]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T03:31:45.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:44 smithi131 ceph-mon[36687]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T03:31:45.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:44 smithi104 ceph-mon[36569]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T03:31:47.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:46 smithi131 ceph-mon[36687]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-06T03:31:47.066 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:46 smithi040 ceph-mon[32220]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-06T03:31:47.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:46 smithi104 ceph-mon[36569]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-06T03:31:49.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:49 smithi131 ceph-mon[36687]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:49.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:49.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:49.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:49 smithi040 ceph-mon[32220]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:49.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:49 smithi104 ceph-mon[36569]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:49.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:49.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:49.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:51.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:51 smithi040 ceph-mon[32220]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:51.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:51 smithi131 ceph-mon[36687]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:51.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:51 smithi104 ceph-mon[36569]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:52.324 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 6 on host 'smithi040' 2024-06-06T03:31:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:52.324+0000 7f0aeeffd700 1 -- 172.21.15.40:0/508207705 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0af80628d0 con 0x7f0ae40a2ca0 2024-06-06T03:31:52.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:52.327+0000 7f0affbb5700 1 -- 172.21.15.40:0/508207705 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0ae40a2ca0 msgr2=0x7f0ae40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:52.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:52.327+0000 7f0affbb5700 1 --2- 172.21.15.40:0/508207705 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0ae40a2ca0 0x7f0ae40a5160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0af4000c00 tx=0x7f0af4005fb0 comp rx=0 tx=0).stop 2024-06-06T03:31:52.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:52.327+0000 7f0affbb5700 1 -- 172.21.15.40:0/508207705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af8102f90 msgr2=0x7f0af819fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:52.327+0000 7f0affbb5700 1 --2- 172.21.15.40:0/508207705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af8102f90 0x7f0af819fb10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0af000efc0 tx=0x7f0af000c3a0 comp rx=0 tx=0).stop 2024-06-06T03:31:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:52.327+0000 7f0affbb5700 1 -- 172.21.15.40:0/508207705 shutdown_connections 2024-06-06T03:31:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:52.327+0000 7f0affbb5700 1 --2- 172.21.15.40:0/508207705 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0ae40a2ca0 0x7f0ae40a5160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:52.327+0000 7f0affbb5700 1 --2- 172.21.15.40:0/508207705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0af81013f0 0x7f0af806acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:52.327+0000 7f0affbb5700 1 --2- 172.21.15.40:0/508207705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af8102f90 0x7f0af819fb10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:52.327+0000 7f0affbb5700 1 --2- 172.21.15.40:0/508207705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af81025f0 0x7f0af819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:52.327+0000 7f0affbb5700 1 -- 172.21.15.40:0/508207705 >> 172.21.15.40:0/508207705 conn(0x7f0af80fc990 msgr2=0x7f0af8105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:31:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:52.327+0000 7f0affbb5700 1 -- 172.21.15.40:0/508207705 shutdown_connections 2024-06-06T03:31:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:31:52.327+0000 7f0affbb5700 1 -- 172.21.15.40:0/508207705 wait complete. 2024-06-06T03:31:52.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:52 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:52 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:52 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:52.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:52 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:52.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:52 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:52 smithi040 ceph-mon[32220]: from='osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T03:31:52.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:52 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:52 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:52 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:52.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:52 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:52 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:52 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:52.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:52 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:52.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:52 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:52 smithi104 ceph-mon[36569]: from='osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T03:31:52.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:52 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:52 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:52 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:52.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:52 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:52 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:52 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:52.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:52 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:52.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:52 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:52 smithi131 ceph-mon[36687]: from='osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T03:31:52.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:52 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:52 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:52.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:52 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:31:53.011 DEBUG:teuthology.orchestra.run.smithi040:osd.6> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.6.service 2024-06-06T03:31:53.013 INFO:tasks.cephadm:Deploying osd.7 on smithi104 with /dev/vg_nvme/lv_2... 2024-06-06T03:31:53.014 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T03:31:53.256 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T03:31:53.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:53 smithi040 ceph-mon[32220]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:53.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:53 smithi040 ceph-mon[32220]: from='osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T03:31:53.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:53 smithi040 ceph-mon[32220]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T03:31:53.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:53 smithi040 ceph-mon[32220]: from='osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-06-06T03:31:53.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:53 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:53.581 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:31:53 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:31:53.353+0000 7fcd34b52700 -1 osd.6 0 waiting for initial osdmap 2024-06-06T03:31:53.581 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:31:53 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:31:53.362+0000 7fcd30945700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T03:31:53.612 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:53 smithi104 ceph-mon[36569]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:53.612 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:53 smithi104 ceph-mon[36569]: from='osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T03:31:53.612 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:53 smithi104 ceph-mon[36569]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T03:31:53.612 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:53 smithi104 ceph-mon[36569]: from='osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-06-06T03:31:53.612 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:53 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:53.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:53 smithi131 ceph-mon[36687]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:53.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:53 smithi131 ceph-mon[36687]: from='osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T03:31:53.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:53 smithi131 ceph-mon[36687]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T03:31:53.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:53 smithi131 ceph-mon[36687]: from='osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-06-06T03:31:53.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:53 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:54.539 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:54 smithi040 ceph-mon[32220]: from='osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-06-06T03:31:54.539 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:54 smithi040 ceph-mon[32220]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T03:31:54.539 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:54 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:54.539 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:54 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:54.654 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:54 smithi104 ceph-mon[36569]: from='osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-06-06T03:31:54.655 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:54 smithi104 ceph-mon[36569]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T03:31:54.655 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:54 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:54.655 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:54 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:54.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:54 smithi131 ceph-mon[36687]: from='osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-06-06T03:31:54.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:54 smithi131 ceph-mon[36687]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T03:31:54.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:54 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:54.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:54 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:55.399 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T03:31:55.399 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T03:31:55.399 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-06-06T03:31:55.399 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-06-06T03:31:55.399 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115819 s, 905 MB/s 2024-06-06T03:31:55.399 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-06-06T03:31:55.622 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:55 smithi040 ceph-mon[32220]: purged_snaps scrub starts 2024-06-06T03:31:55.622 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:55 smithi040 ceph-mon[32220]: purged_snaps scrub ok 2024-06-06T03:31:55.622 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:55 smithi040 ceph-mon[32220]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:55.622 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:55 smithi040 ceph-mon[32220]: osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011] boot 2024-06-06T03:31:55.622 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:55 smithi040 ceph-mon[32220]: osdmap e46: 7 total, 7 up, 7 in 2024-06-06T03:31:55.623 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:55 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:55.700 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:55 smithi104 ceph-mon[36569]: purged_snaps scrub starts 2024-06-06T03:31:55.700 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:55 smithi104 ceph-mon[36569]: purged_snaps scrub ok 2024-06-06T03:31:55.700 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:55 smithi104 ceph-mon[36569]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:55.700 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:55 smithi104 ceph-mon[36569]: osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011] boot 2024-06-06T03:31:55.701 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:55 smithi104 ceph-mon[36569]: osdmap e46: 7 total, 7 up, 7 in 2024-06-06T03:31:55.701 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:55 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:55.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:55 smithi131 ceph-mon[36687]: purged_snaps scrub starts 2024-06-06T03:31:55.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:55 smithi131 ceph-mon[36687]: purged_snaps scrub ok 2024-06-06T03:31:55.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:55 smithi131 ceph-mon[36687]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T03:31:55.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:55 smithi131 ceph-mon[36687]: osd.6 [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011] boot 2024-06-06T03:31:55.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:55 smithi131 ceph-mon[36687]: osdmap e46: 7 total, 7 up, 7 in 2024-06-06T03:31:55.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:55 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:31:56.103 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_2 2024-06-06T03:31:56.339 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T03:31:56.653 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:56 smithi104 ceph-mon[36569]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T03:31:56.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:56 smithi131 ceph-mon[36687]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T03:31:56.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:56 smithi040 ceph-mon[32220]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T03:31:57.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:57 smithi131 ceph-mon[36687]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:31:57.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:57 smithi131 ceph-mon[36687]: Detected new or changed devices on smithi040 2024-06-06T03:31:57.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:57.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:57.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:57.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:57.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:57.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:57 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi040 to 5697M 2024-06-06T03:31:57.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:57.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:57.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:57.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:57.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:57 smithi104 ceph-mon[36569]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:31:57.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:57 smithi104 ceph-mon[36569]: Detected new or changed devices on smithi040 2024-06-06T03:31:57.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:57.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:57.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:57.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:57.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:57.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:57 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi040 to 5697M 2024-06-06T03:31:57.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:57.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:57.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:57.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:57 smithi040 ceph-mon[32220]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:31:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:57 smithi040 ceph-mon[32220]: Detected new or changed devices on smithi040 2024-06-06T03:31:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:31:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:57 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi040 to 5697M 2024-06-06T03:31:57.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:57.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:57.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:31:57.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:31:58.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.052+0000 7fc894f2f700 1 -- 172.21.15.104:0/3981948197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8901007e0 msgr2=0x7fc890100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:58.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.052+0000 7fc894f2f700 1 --2- 172.21.15.104:0/3981948197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8901007e0 0x7fc890100c00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc878009990 tx=0x7fc878009ca0 comp rx=0 tx=0).stop 2024-06-06T03:31:58.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.053+0000 7fc894f2f700 1 -- 172.21.15.104:0/3981948197 shutdown_connections 2024-06-06T03:31:58.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.053+0000 7fc894f2f700 1 --2- 172.21.15.104:0/3981948197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc890104160 0x7fc89010aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:58.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.053+0000 7fc894f2f700 1 --2- 172.21.15.104:0/3981948197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8901007e0 0x7fc890100c00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:58.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.053+0000 7fc894f2f700 1 --2- 172.21.15.104:0/3981948197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8901037c0 0x7fc890103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:58.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.053+0000 7fc894f2f700 1 -- 172.21.15.104:0/3981948197 >> 172.21.15.104:0/3981948197 conn(0x7fc8900fc140 msgr2=0x7fc8900fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:31:58.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.053+0000 7fc894f2f700 1 -- 172.21.15.104:0/3981948197 shutdown_connections 2024-06-06T03:31:58.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.053+0000 7fc894f2f700 1 -- 172.21.15.104:0/3981948197 wait complete. 2024-06-06T03:31:58.054 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.053+0000 7fc894f2f700 1 Processor -- start 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.053+0000 7fc894f2f700 1 -- start start 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc894f2f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8901007e0 0x7fc8901986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc894f2f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8901037c0 0x7fc890198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc894f2f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc890104160 0x7fc89019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc894f2f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc89010e150 con 0x7fc890104160 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc894f2f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc89010dfd0 con 0x7fc8901037c0 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc894f2f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc89010e2d0 con 0x7fc8901007e0 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc88dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8901037c0 0x7fc890198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc88dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8901037c0 0x7fc890198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35172/0 (socket says 172.21.15.104:35172) 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc88dd9b700 1 -- 172.21.15.104:0/2772085237 learned_addr learned my addr 172.21.15.104:0/2772085237 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc88ed9d700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc890104160 0x7fc89019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc88dd9b700 1 -- 172.21.15.104:0/2772085237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8901007e0 msgr2=0x7fc8901986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc88e59c700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8901007e0 0x7fc8901986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc88dd9b700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8901007e0 0x7fc8901986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc88dd9b700 1 -- 172.21.15.104:0/2772085237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc890104160 msgr2=0x7fc89019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:31:58.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc88dd9b700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc890104160 0x7fc89019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:31:58.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc88dd9b700 1 -- 172.21.15.104:0/2772085237 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8901a0310 con 0x7fc8901037c0 2024-06-06T03:31:58.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc88ed9d700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc890104160 0x7fc89019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:31:58.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc88dd9b700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8901037c0 0x7fc890198bf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc88000c9b0 tx=0x7fc88000cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:31:58.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc8877fe700 1 -- 172.21.15.104:0/2772085237 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880007b00 con 0x7fc8901037c0 2024-06-06T03:31:58.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc894f2f700 1 -- 172.21.15.104:0/2772085237 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8901a0600 con 0x7fc8901037c0 2024-06-06T03:31:58.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc8877fe700 1 -- 172.21.15.104:0/2772085237 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc880007ca0 con 0x7fc8901037c0 2024-06-06T03:31:58.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc8877fe700 1 -- 172.21.15.104:0/2772085237 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880018820 con 0x7fc8901037c0 2024-06-06T03:31:58.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.054+0000 7fc894f2f700 1 -- 172.21.15.104:0/2772085237 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8901a0b40 con 0x7fc8901037c0 2024-06-06T03:31:58.057 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.055+0000 7fc894f2f700 1 -- 172.21.15.104:0/2772085237 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc890069180 con 0x7fc8901037c0 2024-06-06T03:31:58.057 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.057+0000 7fc8877fe700 1 -- 172.21.15.104:0/2772085237 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc88000fa30 con 0x7fc8901037c0 2024-06-06T03:31:58.057 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.058+0000 7fc8877fe700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc87c0a2c20 0x7fc87c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:31:58.057 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.058+0000 7fc8877fe700 1 -- 172.21.15.104:0/2772085237 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7fc8800ce680 con 0x7fc8901037c0 2024-06-06T03:31:58.057 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.058+0000 7fc88e59c700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc87c0a2c20 0x7fc87c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:31:58.058 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.058+0000 7fc88e59c700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc87c0a2c20 0x7fc87c0a50e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc878006010 tx=0x7fc878005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:31:58.061 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.061+0000 7fc8877fe700 1 -- 172.21.15.104:0/2772085237 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc880014030 con 0x7fc8901037c0 2024-06-06T03:31:58.266 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:31:58.266+0000 7fc894f2f700 1 -- 172.21.15.104:0/2772085237 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc8900628d0 con 0x7fc87c0a2c20 2024-06-06T03:31:58.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:58 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:31:58.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:58 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:31:58.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:58 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:58.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:58 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:31:58.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:58 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:31:58.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:58 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:58.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:58 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:31:58.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:58 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:31:58.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:58 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:31:59.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:59 smithi131 ceph-mon[36687]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:31:59.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:31:59 smithi131 ceph-mon[36687]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:31:59.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:59 smithi040 ceph-mon[32220]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:31:59.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:31:59 smithi040 ceph-mon[32220]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:31:59.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:59 smithi104 ceph-mon[36569]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:31:59.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:31:59 smithi104 ceph-mon[36569]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:32:01.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:01 smithi131 ceph-mon[36687]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:01.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:01 smithi131 ceph-mon[36687]: from='client.? 172.21.15.104:0/1799168242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfc536bb-6ae4-459e-afd9-8197c7420bd1"}]: dispatch 2024-06-06T03:32:01.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:01 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfc536bb-6ae4-459e-afd9-8197c7420bd1"}]: dispatch 2024-06-06T03:32:01.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:01 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfc536bb-6ae4-459e-afd9-8197c7420bd1"}]': finished 2024-06-06T03:32:01.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:01 smithi131 ceph-mon[36687]: osdmap e48: 8 total, 7 up, 8 in 2024-06-06T03:32:01.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:01 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:01.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:01 smithi040 ceph-mon[32220]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:01.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:01 smithi040 ceph-mon[32220]: from='client.? 172.21.15.104:0/1799168242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfc536bb-6ae4-459e-afd9-8197c7420bd1"}]: dispatch 2024-06-06T03:32:01.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:01 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfc536bb-6ae4-459e-afd9-8197c7420bd1"}]: dispatch 2024-06-06T03:32:01.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:01 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfc536bb-6ae4-459e-afd9-8197c7420bd1"}]': finished 2024-06-06T03:32:01.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:01 smithi040 ceph-mon[32220]: osdmap e48: 8 total, 7 up, 8 in 2024-06-06T03:32:01.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:01 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:01.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:01 smithi104 ceph-mon[36569]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:01.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:01 smithi104 ceph-mon[36569]: from='client.? 172.21.15.104:0/1799168242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfc536bb-6ae4-459e-afd9-8197c7420bd1"}]: dispatch 2024-06-06T03:32:01.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:01 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfc536bb-6ae4-459e-afd9-8197c7420bd1"}]: dispatch 2024-06-06T03:32:01.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:01 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfc536bb-6ae4-459e-afd9-8197c7420bd1"}]': finished 2024-06-06T03:32:01.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:01 smithi104 ceph-mon[36569]: osdmap e48: 8 total, 7 up, 8 in 2024-06-06T03:32:01.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:01 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:02.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:02 smithi131 ceph-mon[36687]: from='client.? 172.21.15.104:0/3524195741' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:32:02.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:02 smithi040 ceph-mon[32220]: from='client.? 172.21.15.104:0/3524195741' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:32:02.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:02 smithi104 ceph-mon[36569]: from='client.? 172.21.15.104:0/3524195741' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:32:03.769 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:03 smithi040 ceph-mon[32220]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:03.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:03 smithi131 ceph-mon[36687]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:03.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:03 smithi104 ceph-mon[36569]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:05.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:05 smithi131 ceph-mon[36687]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:05.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:05 smithi040 ceph-mon[32220]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:05.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:05 smithi104 ceph-mon[36569]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:07.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:07 smithi131 ceph-mon[36687]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:07.793 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:07 smithi104 ceph-mon[36569]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:07.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:07 smithi040 ceph-mon[32220]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:09.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:09 smithi131 ceph-mon[36687]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:09.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:09 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T03:32:09.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:09 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:09.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:09 smithi040 ceph-mon[32220]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:09.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:09 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T03:32:09.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:09 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:09.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:09 smithi104 ceph-mon[36569]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:09.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:09 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T03:32:09.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:09 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:10.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:10 smithi104 ceph-mon[36569]: Deploying daemon osd.7 on smithi104 2024-06-06T03:32:10.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:10 smithi104 ceph-mon[36569]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:10.781 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:10 smithi131 ceph-mon[36687]: Deploying daemon osd.7 on smithi104 2024-06-06T03:32:10.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:10 smithi131 ceph-mon[36687]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:10.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:10 smithi040 ceph-mon[32220]: Deploying daemon osd.7 on smithi104 2024-06-06T03:32:10.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:10 smithi040 ceph-mon[32220]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:12.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:12 smithi104 ceph-mon[36569]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:13.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:12 smithi131 ceph-mon[36687]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:13.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:12 smithi040 ceph-mon[32220]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:15.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:15 smithi104 ceph-mon[36569]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:15.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:15 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:15.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:15 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:15.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:15 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:32:15.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:15 smithi040 ceph-mon[32220]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:15.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:15 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:15.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:15 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:15.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:15 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:32:15.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:15 smithi131 ceph-mon[36687]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:15.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:15 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:15.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:15 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:15.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:15 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:32:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:17 smithi040 ceph-mon[32220]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:32:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:17.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:17 smithi104 ceph-mon[36569]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:17.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:17 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:17.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:17 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:17.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:17 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:17.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:17 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:32:17.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:17 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:17.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:17 smithi131 ceph-mon[36687]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:17.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:17 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:17.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:17 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:17.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:17 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:17.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:17 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:32:17.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:17 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:18.337 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:32:18.337+0000 7fc8877fe700 1 -- 172.21.15.104:0/2772085237 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc8900628d0 con 0x7fc87c0a2c20 2024-06-06T03:32:18.337 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 7 on host 'smithi104' 2024-06-06T03:32:18.340 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:32:18.340+0000 7fc894f2f700 1 -- 172.21.15.104:0/2772085237 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc87c0a2c20 msgr2=0x7fc87c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:32:18.340 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:32:18.340+0000 7fc894f2f700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc87c0a2c20 0x7fc87c0a50e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc878006010 tx=0x7fc878005c20 comp rx=0 tx=0).stop 2024-06-06T03:32:18.340 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:32:18.340+0000 7fc894f2f700 1 -- 172.21.15.104:0/2772085237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8901037c0 msgr2=0x7fc890198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:32:18.340 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:32:18.340+0000 7fc894f2f700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8901037c0 0x7fc890198bf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc88000c9b0 tx=0x7fc88000cd70 comp rx=0 tx=0).stop 2024-06-06T03:32:18.340 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:32:18.340+0000 7fc894f2f700 1 -- 172.21.15.104:0/2772085237 shutdown_connections 2024-06-06T03:32:18.340 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:32:18.340+0000 7fc894f2f700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc87c0a2c20 0x7fc87c0a50e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:18.340 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:32:18.340+0000 7fc894f2f700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8901007e0 0x7fc8901986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:18.340 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:32:18.340+0000 7fc894f2f700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc890104160 0x7fc89019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:18.340 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:32:18.340+0000 7fc894f2f700 1 --2- 172.21.15.104:0/2772085237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8901037c0 0x7fc890198bf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:18.340 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:32:18.340+0000 7fc894f2f700 1 -- 172.21.15.104:0/2772085237 >> 172.21.15.104:0/2772085237 conn(0x7fc8900fc140 msgr2=0x7fc890106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:32:18.341 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:32:18.340+0000 7fc894f2f700 1 -- 172.21.15.104:0/2772085237 shutdown_connections 2024-06-06T03:32:18.341 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:32:18.340+0000 7fc894f2f700 1 -- 172.21.15.104:0/2772085237 wait complete. 2024-06-06T03:32:18.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:18 smithi040 ceph-mon[32220]: from='osd.7 [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T03:32:18.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:18 smithi040 ceph-mon[32220]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T03:32:18.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:18 smithi104 ceph-mon[36569]: from='osd.7 [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T03:32:18.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:18 smithi104 ceph-mon[36569]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T03:32:18.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:18 smithi131 ceph-mon[36687]: from='osd.7 [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T03:32:18.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:18 smithi131 ceph-mon[36687]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T03:32:19.031 DEBUG:teuthology.orchestra.run.smithi104:osd.7> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.7.service 2024-06-06T03:32:19.034 INFO:tasks.cephadm:Deploying osd.8 on smithi131 with /dev/vg_nvme/lv_2... 2024-06-06T03:32:19.034 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T03:32:19.279 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:32:19.401 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:19 smithi131 ceph-mon[36687]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:19.401 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:19 smithi131 ceph-mon[36687]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T03:32:19.401 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:19 smithi131 ceph-mon[36687]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T03:32:19.401 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:19.401 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:19 smithi131 ceph-mon[36687]: from='osd.7 [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:32:19.402 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:19 smithi131 ceph-mon[36687]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:32:19.402 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:19.402 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:32:19.402 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:19 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:19 smithi040 ceph-mon[32220]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:19 smithi040 ceph-mon[32220]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T03:32:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:19 smithi040 ceph-mon[32220]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T03:32:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:19 smithi040 ceph-mon[32220]: from='osd.7 [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:32:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:19 smithi040 ceph-mon[32220]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:32:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:32:19.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:19 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:19.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:19 smithi104 ceph-mon[36569]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:19.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:19 smithi104 ceph-mon[36569]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T03:32:19.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:19 smithi104 ceph-mon[36569]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T03:32:19.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:19.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:19 smithi104 ceph-mon[36569]: from='osd.7 [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:32:19.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:19 smithi104 ceph-mon[36569]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:32:19.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:19.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:32:19.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:19 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:19.979 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:32:19 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:32:19.726+0000 7f6f12eda700 -1 osd.7 0 waiting for initial osdmap 2024-06-06T03:32:19.979 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:32:19 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:32:19.736+0000 7f6f0acc5700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T03:32:20.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:20 smithi040 ceph-mon[32220]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-06-06T03:32:20.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:20 smithi040 ceph-mon[32220]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T03:32:20.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:20.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:20.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:20 smithi040 ceph-mon[32220]: from='osd.7 ' entity='osd.7' 2024-06-06T03:32:20.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:20 smithi104 ceph-mon[36569]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-06-06T03:32:20.594 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:20 smithi104 ceph-mon[36569]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T03:32:20.594 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:20.594 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:20.594 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:20 smithi104 ceph-mon[36569]: from='osd.7 ' entity='osd.7' 2024-06-06T03:32:20.719 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:20 smithi131 ceph-mon[36687]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-06-06T03:32:20.719 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:20 smithi131 ceph-mon[36687]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T03:32:20.719 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:20.720 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:20.720 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:20 smithi131 ceph-mon[36687]: from='osd.7 ' entity='osd.7' 2024-06-06T03:32:21.448 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T03:32:21.448 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T03:32:21.448 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-06-06T03:32:21.448 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-06-06T03:32:21.448 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115068 s, 911 MB/s 2024-06-06T03:32:21.448 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-06-06T03:32:21.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:21 smithi040 ceph-mon[32220]: purged_snaps scrub starts 2024-06-06T03:32:21.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:21 smithi040 ceph-mon[32220]: purged_snaps scrub ok 2024-06-06T03:32:21.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:21 smithi040 ceph-mon[32220]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:21.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:21 smithi040 ceph-mon[32220]: osd.7 [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051] boot 2024-06-06T03:32:21.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:21 smithi040 ceph-mon[32220]: osdmap e51: 8 total, 8 up, 8 in 2024-06-06T03:32:21.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:21 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:21.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:21 smithi040 ceph-mon[32220]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T03:32:21.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:21 smithi104 ceph-mon[36569]: purged_snaps scrub starts 2024-06-06T03:32:21.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:21 smithi104 ceph-mon[36569]: purged_snaps scrub ok 2024-06-06T03:32:21.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:21 smithi104 ceph-mon[36569]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:21.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:21 smithi104 ceph-mon[36569]: osd.7 [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051] boot 2024-06-06T03:32:21.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:21 smithi104 ceph-mon[36569]: osdmap e51: 8 total, 8 up, 8 in 2024-06-06T03:32:21.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:21 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:21.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:21 smithi104 ceph-mon[36569]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T03:32:21.748 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:21 smithi131 ceph-mon[36687]: purged_snaps scrub starts 2024-06-06T03:32:21.748 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:21 smithi131 ceph-mon[36687]: purged_snaps scrub ok 2024-06-06T03:32:21.748 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:21 smithi131 ceph-mon[36687]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-06T03:32:21.749 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:21 smithi131 ceph-mon[36687]: osd.7 [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051] boot 2024-06-06T03:32:21.749 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:21 smithi131 ceph-mon[36687]: osdmap e51: 8 total, 8 up, 8 in 2024-06-06T03:32:21.749 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:21 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:32:21.749 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:21 smithi131 ceph-mon[36687]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T03:32:22.092 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_2 2024-06-06T03:32:22.337 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:32:23.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:23 smithi131 ceph-mon[36687]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:23.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:23 smithi131 ceph-mon[36687]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T03:32:23.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:23 smithi131 ceph-mon[36687]: Detected new or changed devices on smithi104 2024-06-06T03:32:23.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:23.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:23.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:23.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:23.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:23.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:23 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi104 to 5696M 2024-06-06T03:32:23.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:23.407 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:23.407 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:32:23.407 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:23.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:23 smithi040 ceph-mon[32220]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:23.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:23 smithi040 ceph-mon[32220]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T03:32:23.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:23 smithi040 ceph-mon[32220]: Detected new or changed devices on smithi104 2024-06-06T03:32:23.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:23.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:23.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:23.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:23.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:23.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:23 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi104 to 5696M 2024-06-06T03:32:23.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:23.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:23.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:32:23.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:23.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:23 smithi104 ceph-mon[36569]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:23.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:23 smithi104 ceph-mon[36569]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T03:32:23.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:23 smithi104 ceph-mon[36569]: Detected new or changed devices on smithi104 2024-06-06T03:32:23.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:23.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:23.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:23.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:23.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:23.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:23 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi104 to 5696M 2024-06-06T03:32:23.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:23.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:23.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:32:23.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.888+0000 7f1de9613700 1 -- 172.21.15.131:0/2624459945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1de4103700 msgr2=0x7f1de4103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:32:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.888+0000 7f1de9613700 1 --2- 172.21.15.131:0/2624459945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1de4103700 0x7f1de4103b60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009990 tx=0x7f1dcc009ca0 comp rx=0 tx=0).stop 2024-06-06T03:32:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.888+0000 7f1de9613700 1 -- 172.21.15.131:0/2624459945 shutdown_connections 2024-06-06T03:32:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.888+0000 7f1de9613700 1 --2- 172.21.15.131:0/2624459945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1de41040a0 0x7f1de410a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.888+0000 7f1de9613700 1 --2- 172.21.15.131:0/2624459945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1de4102500 0x7f1de4102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.888+0000 7f1de9613700 1 --2- 172.21.15.131:0/2624459945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1de4103700 0x7f1de4103b60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.888+0000 7f1de9613700 1 -- 172.21.15.131:0/2624459945 >> 172.21.15.131:0/2624459945 conn(0x7f1de40fdaa0 msgr2=0x7f1de40ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:32:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.888+0000 7f1de9613700 1 -- 172.21.15.131:0/2624459945 shutdown_connections 2024-06-06T03:32:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.888+0000 7f1de9613700 1 -- 172.21.15.131:0/2624459945 wait complete. 2024-06-06T03:32:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.888+0000 7f1de9613700 1 Processor -- start 2024-06-06T03:32:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de9613700 1 -- start start 2024-06-06T03:32:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de9613700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1de4102500 0x7f1de41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:32:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de9613700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1de4103700 0x7f1de4198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:32:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de9613700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1de41040a0 0x7f1de419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:32:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de9613700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1de410e0f0 con 0x7f1de41040a0 2024-06-06T03:32:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de9613700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1de410df70 con 0x7f1de4103700 2024-06-06T03:32:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de9613700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1de410e270 con 0x7f1de4102500 2024-06-06T03:32:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1de4102500 0x7f1de41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:32:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1de4102500 0x7f1de41986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38422/0 (socket says 172.21.15.131:38422) 2024-06-06T03:32:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de2ffd700 1 -- 172.21.15.131:0/86859125 learned_addr learned my addr 172.21.15.131:0/86859125 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-06-06T03:32:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de37fe700 1 --2- 172.21.15.131:0/86859125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1de41040a0 0x7f1de419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:32:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1dda7fc700 1 --2- 172.21.15.131:0/86859125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1de4103700 0x7f1de4198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:32:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de2ffd700 1 -- 172.21.15.131:0/86859125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1de4103700 msgr2=0x7f1de4198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:32:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de2ffd700 1 --2- 172.21.15.131:0/86859125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1de4103700 0x7f1de4198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de2ffd700 1 -- 172.21.15.131:0/86859125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1de41040a0 msgr2=0x7f1de419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:32:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de2ffd700 1 --2- 172.21.15.131:0/86859125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1de41040a0 0x7f1de419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.889+0000 7f1de2ffd700 1 -- 172.21.15.131:0/86859125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de41a0330 con 0x7f1de4102500 2024-06-06T03:32:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.890+0000 7f1de2ffd700 1 --2- 172.21.15.131:0/86859125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1de4102500 0x7f1de41986d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400d9b0 tx=0x7f1dd400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:32:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.890+0000 7f1de0ff9700 1 -- 172.21.15.131:0/86859125 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4017070 con 0x7f1de4102500 2024-06-06T03:32:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.890+0000 7f1de0ff9700 1 -- 172.21.15.131:0/86859125 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dd4009e10 con 0x7f1de4102500 2024-06-06T03:32:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.890+0000 7f1de0ff9700 1 -- 172.21.15.131:0/86859125 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd400f3c0 con 0x7f1de4102500 2024-06-06T03:32:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.890+0000 7f1de9613700 1 -- 172.21.15.131:0/86859125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de41a0620 con 0x7f1de4102500 2024-06-06T03:32:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.890+0000 7f1de9613700 1 -- 172.21.15.131:0/86859125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de41a0b10 con 0x7f1de4102500 2024-06-06T03:32:23.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.891+0000 7f1de9613700 1 -- 172.21.15.131:0/86859125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de4069180 con 0x7f1de4102500 2024-06-06T03:32:23.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.893+0000 7f1de0ff9700 1 -- 172.21.15.131:0/86859125 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1dd4009930 con 0x7f1de4102500 2024-06-06T03:32:23.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.893+0000 7f1de0ff9700 1 --2- 172.21.15.131:0/86859125 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1dd00a2c20 0x7f1dd00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:32:23.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.893+0000 7f1de0ff9700 1 -- 172.21.15.131:0/86859125 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7f1dd40cdf70 con 0x7f1de4102500 2024-06-06T03:32:23.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.893+0000 7f1dda7fc700 1 --2- 172.21.15.131:0/86859125 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1dd00a2c20 0x7f1dd00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:32:23.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.894+0000 7f1dda7fc700 1 --2- 172.21.15.131:0/86859125 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1dd00a2c20 0x7f1dd00a50e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00b2e0 tx=0x7f1dcc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:32:23.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:23.897+0000 7f1de0ff9700 1 -- 172.21.15.131:0/86859125 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1dd409ba30 con 0x7f1de4102500 2024-06-06T03:32:24.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:24.101+0000 7f1de9613700 1 -- 172.21.15.131:0/86859125 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1de40628d0 con 0x7f1dd00a2c20 2024-06-06T03:32:24.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:24 smithi131 ceph-mon[36687]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T03:32:24.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:24 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:32:24.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:24 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:32:24.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:24 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:24.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:24 smithi040 ceph-mon[32220]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T03:32:24.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:24 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:32:24.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:24 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:32:24.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:24 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:24.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:24 smithi104 ceph-mon[36569]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T03:32:24.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:24 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:32:24.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:24 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:32:24.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:24 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:25.722 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:25 smithi131 ceph-mon[36687]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:25.722 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:25 smithi131 ceph-mon[36687]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:32:25.722 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:25 smithi131 ceph-mon[36687]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T03:32:25.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:25 smithi040 ceph-mon[32220]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:25.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:25 smithi040 ceph-mon[32220]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:32:25.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:25 smithi040 ceph-mon[32220]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T03:32:25.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:25 smithi104 ceph-mon[36569]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:25.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:25 smithi104 ceph-mon[36569]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:32:25.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:25 smithi104 ceph-mon[36569]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T03:32:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:27 smithi131 ceph-mon[36687]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:27 smithi131 ceph-mon[36687]: from='client.? 172.21.15.131:0/3040320112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe214f2-5ea3-4fc5-83bf-d9bccafe1529"}]: dispatch 2024-06-06T03:32:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:27 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe214f2-5ea3-4fc5-83bf-d9bccafe1529"}]: dispatch 2024-06-06T03:32:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:27 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fe214f2-5ea3-4fc5-83bf-d9bccafe1529"}]': finished 2024-06-06T03:32:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:27 smithi131 ceph-mon[36687]: osdmap e56: 9 total, 8 up, 9 in 2024-06-06T03:32:27.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:27 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:27.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:27 smithi131 ceph-mon[36687]: from='client.? 172.21.15.131:0/177807418' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:32:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:27 smithi040 ceph-mon[32220]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:27 smithi040 ceph-mon[32220]: from='client.? 172.21.15.131:0/3040320112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe214f2-5ea3-4fc5-83bf-d9bccafe1529"}]: dispatch 2024-06-06T03:32:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:27 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe214f2-5ea3-4fc5-83bf-d9bccafe1529"}]: dispatch 2024-06-06T03:32:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:27 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fe214f2-5ea3-4fc5-83bf-d9bccafe1529"}]': finished 2024-06-06T03:32:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:27 smithi040 ceph-mon[32220]: osdmap e56: 9 total, 8 up, 9 in 2024-06-06T03:32:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:27 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:27.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:27 smithi040 ceph-mon[32220]: from='client.? 172.21.15.131:0/177807418' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:32:27.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:27 smithi104 ceph-mon[36569]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:27.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:27 smithi104 ceph-mon[36569]: from='client.? 172.21.15.131:0/3040320112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe214f2-5ea3-4fc5-83bf-d9bccafe1529"}]: dispatch 2024-06-06T03:32:27.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:27 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe214f2-5ea3-4fc5-83bf-d9bccafe1529"}]: dispatch 2024-06-06T03:32:27.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:27 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fe214f2-5ea3-4fc5-83bf-d9bccafe1529"}]': finished 2024-06-06T03:32:27.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:27 smithi104 ceph-mon[36569]: osdmap e56: 9 total, 8 up, 9 in 2024-06-06T03:32:27.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:27 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:27.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:27 smithi104 ceph-mon[36569]: from='client.? 172.21.15.131:0/177807418' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:32:29.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:29 smithi131 ceph-mon[36687]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:29.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:29 smithi040 ceph-mon[32220]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:29.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:29 smithi104 ceph-mon[36569]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:31.746 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:31 smithi131 ceph-mon[36687]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:31.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:31 smithi040 ceph-mon[32220]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:31.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:31 smithi104 ceph-mon[36569]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:33.749 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:33 smithi131 ceph-mon[36687]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:33.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:33 smithi040 ceph-mon[32220]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:33.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:33 smithi104 ceph-mon[36569]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:34.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:34 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T03:32:34.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:34 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:34.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:34 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T03:32:34.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:34 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:34.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:34 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T03:32:34.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:34 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:35.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:35 smithi131 ceph-mon[36687]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:35.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:35 smithi131 ceph-mon[36687]: Deploying daemon osd.8 on smithi131 2024-06-06T03:32:35.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:35 smithi040 ceph-mon[32220]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:35.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:35 smithi040 ceph-mon[32220]: Deploying daemon osd.8 on smithi131 2024-06-06T03:32:35.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:35 smithi104 ceph-mon[36569]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:35.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:35 smithi104 ceph-mon[36569]: Deploying daemon osd.8 on smithi131 2024-06-06T03:32:37.779 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:37 smithi131 ceph-mon[36687]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:37.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:37 smithi040 ceph-mon[32220]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:37.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:37 smithi104 ceph-mon[36569]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:39.666 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:39 smithi131 ceph-mon[36687]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:39.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:39 smithi040 ceph-mon[32220]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:39.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:39 smithi104 ceph-mon[36569]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:40.999 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:40 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:40.999 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:40 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:40.999 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:40 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:32:40.999 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:40 smithi131 ceph-mon[36687]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:41.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:40 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:41.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:40 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:41.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:40 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:32:41.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:40 smithi040 ceph-mon[32220]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:41.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:40 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:41.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:40 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:41.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:40 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:32:41.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:40 smithi104 ceph-mon[36569]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:43.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:42 smithi131 ceph-mon[36687]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:43.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:43.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:43.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:43.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:32:43.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:43.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:42 smithi040 ceph-mon[32220]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:43.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:32:43.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:43.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:42 smithi104 ceph-mon[36569]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:43.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:43.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:43.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:43.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:32:43.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:43.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:43.575+0000 7f1de0ff9700 1 -- 172.21.15.131:0/86859125 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1de40628d0 con 0x7f1dd00a2c20 2024-06-06T03:32:43.576 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 8 on host 'smithi131' 2024-06-06T03:32:43.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:43.577+0000 7f1de9613700 1 -- 172.21.15.131:0/86859125 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1dd00a2c20 msgr2=0x7f1dd00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:32:43.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:43.577+0000 7f1de9613700 1 --2- 172.21.15.131:0/86859125 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1dd00a2c20 0x7f1dd00a50e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00b2e0 tx=0x7f1dcc005fd0 comp rx=0 tx=0).stop 2024-06-06T03:32:43.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:43.577+0000 7f1de9613700 1 -- 172.21.15.131:0/86859125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1de4102500 msgr2=0x7f1de41986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:32:43.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:43.577+0000 7f1de9613700 1 --2- 172.21.15.131:0/86859125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1de4102500 0x7f1de41986d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400d9b0 tx=0x7f1dd400dcc0 comp rx=0 tx=0).stop 2024-06-06T03:32:43.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:43.577+0000 7f1de9613700 1 -- 172.21.15.131:0/86859125 shutdown_connections 2024-06-06T03:32:43.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:43.578+0000 7f1de9613700 1 --2- 172.21.15.131:0/86859125 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1dd00a2c20 0x7f1dd00a50e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:43.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:43.578+0000 7f1de9613700 1 --2- 172.21.15.131:0/86859125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1de4102500 0x7f1de41986d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:43.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:43.578+0000 7f1de9613700 1 --2- 172.21.15.131:0/86859125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1de41040a0 0x7f1de419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:43.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:43.578+0000 7f1de9613700 1 --2- 172.21.15.131:0/86859125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1de4103700 0x7f1de4198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:43.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:43.578+0000 7f1de9613700 1 -- 172.21.15.131:0/86859125 >> 172.21.15.131:0/86859125 conn(0x7f1de40fdaa0 msgr2=0x7f1de4106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:32:43.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:43.578+0000 7f1de9613700 1 -- 172.21.15.131:0/86859125 shutdown_connections 2024-06-06T03:32:43.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:32:43.578+0000 7f1de9613700 1 -- 172.21.15.131:0/86859125 wait complete. 2024-06-06T03:32:43.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:43 smithi131 ceph-mon[36687]: from='osd.8 [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T03:32:43.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:43 smithi131 ceph-mon[36687]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T03:32:43.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:43 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:43.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:43 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:43.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:43 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:32:44.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:43 smithi040 ceph-mon[32220]: from='osd.8 [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T03:32:44.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:43 smithi040 ceph-mon[32220]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T03:32:44.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:43 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:44.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:43 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:44.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:43 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:32:44.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:43 smithi104 ceph-mon[36569]: from='osd.8 [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T03:32:44.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:43 smithi104 ceph-mon[36569]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T03:32:44.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:43 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:44.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:43 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:44.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:43 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:32:44.275 DEBUG:teuthology.orchestra.run.smithi131:osd.8> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.8.service 2024-06-06T03:32:44.277 INFO:tasks.cephadm:Deploying osd.9 on smithi040 with /dev/vg_nvme/lv_1... 2024-06-06T03:32:44.278 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T03:32:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:32:44.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:44 smithi131 ceph-mon[36687]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:44.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:44 smithi131 ceph-mon[36687]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T03:32:44.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:44 smithi131 ceph-mon[36687]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T03:32:44.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:44 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:44.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:44 smithi131 ceph-mon[36687]: from='osd.8 [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:32:44.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:44 smithi131 ceph-mon[36687]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:32:44.993 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:44 smithi040 ceph-mon[32220]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:44.993 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:44 smithi040 ceph-mon[32220]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T03:32:44.993 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:44 smithi040 ceph-mon[32220]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T03:32:44.993 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:44 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:44.993 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:44 smithi040 ceph-mon[32220]: from='osd.8 [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:32:44.993 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:44 smithi040 ceph-mon[32220]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:32:45.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:44 smithi104 ceph-mon[36569]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:45.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:44 smithi104 ceph-mon[36569]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T03:32:45.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:44 smithi104 ceph-mon[36569]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T03:32:45.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:44 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:45.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:44 smithi104 ceph-mon[36569]: from='osd.8 [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:32:45.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:44 smithi104 ceph-mon[36569]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:32:45.484 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:32:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:32:45.133+0000 7fed048e0700 -1 osd.8 0 waiting for initial osdmap 2024-06-06T03:32:45.484 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:32:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:32:45.143+0000 7fecfc6cb700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T03:32:45.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:45 smithi131 ceph-mon[36687]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-06-06T03:32:45.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:45 smithi131 ceph-mon[36687]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T03:32:45.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:45 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:45.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:45 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:45.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:45 smithi131 ceph-mon[36687]: from='osd.8 ' entity='osd.8' 2024-06-06T03:32:45.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:45 smithi131 ceph-mon[36687]: osd.8 [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794] boot 2024-06-06T03:32:45.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:45 smithi131 ceph-mon[36687]: osdmap e59: 9 total, 9 up, 9 in 2024-06-06T03:32:45.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:45 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:46.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:45 smithi040 ceph-mon[32220]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-06-06T03:32:46.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:45 smithi040 ceph-mon[32220]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T03:32:46.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:45 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:46.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:45 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:46.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:45 smithi040 ceph-mon[32220]: from='osd.8 ' entity='osd.8' 2024-06-06T03:32:46.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:45 smithi040 ceph-mon[32220]: osd.8 [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794] boot 2024-06-06T03:32:46.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:45 smithi040 ceph-mon[32220]: osdmap e59: 9 total, 9 up, 9 in 2024-06-06T03:32:46.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:45 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:46.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:45 smithi104 ceph-mon[36569]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-06-06T03:32:46.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:45 smithi104 ceph-mon[36569]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T03:32:46.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:45 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:46.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:45 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:46.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:45 smithi104 ceph-mon[36569]: from='osd.8 ' entity='osd.8' 2024-06-06T03:32:46.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:45 smithi104 ceph-mon[36569]: osd.8 [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794] boot 2024-06-06T03:32:46.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:45 smithi104 ceph-mon[36569]: osdmap e59: 9 total, 9 up, 9 in 2024-06-06T03:32:46.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:45 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:32:46.711 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T03:32:46.712 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T03:32:46.712 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-06-06T03:32:46.712 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-06-06T03:32:46.712 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111788 s, 938 MB/s 2024-06-06T03:32:46.712 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-06-06T03:32:47.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:47 smithi131 ceph-mon[36687]: purged_snaps scrub starts 2024-06-06T03:32:47.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:47 smithi131 ceph-mon[36687]: purged_snaps scrub ok 2024-06-06T03:32:47.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:47 smithi131 ceph-mon[36687]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:47.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:47 smithi131 ceph-mon[36687]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T03:32:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:47 smithi040 ceph-mon[32220]: purged_snaps scrub starts 2024-06-06T03:32:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:47 smithi040 ceph-mon[32220]: purged_snaps scrub ok 2024-06-06T03:32:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:47 smithi040 ceph-mon[32220]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:47 smithi040 ceph-mon[32220]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T03:32:47.401 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_1 2024-06-06T03:32:47.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:47 smithi104 ceph-mon[36569]: purged_snaps scrub starts 2024-06-06T03:32:47.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:47 smithi104 ceph-mon[36569]: purged_snaps scrub ok 2024-06-06T03:32:47.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:47 smithi104 ceph-mon[36569]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-06T03:32:47.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:47 smithi104 ceph-mon[36569]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T03:32:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:32:48.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:48 smithi040 ceph-mon[32220]: Detected new or changed devices on smithi131 2024-06-06T03:32:48.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:48.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:48.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:48.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:48.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:48.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:48 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi131 to 5696M 2024-06-06T03:32:48.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:48.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:48.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:32:48.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:48.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:48 smithi040 ceph-mon[32220]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T03:32:48.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:48 smithi104 ceph-mon[36569]: Detected new or changed devices on smithi131 2024-06-06T03:32:48.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:48.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:48.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:48.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:48.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:48.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:48 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi131 to 5696M 2024-06-06T03:32:48.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:48.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:48.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:32:48.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:48.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:48 smithi104 ceph-mon[36569]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T03:32:48.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:48 smithi131 ceph-mon[36687]: Detected new or changed devices on smithi131 2024-06-06T03:32:48.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:48.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:48.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:48.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:48.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:32:48.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:48 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi131 to 5696M 2024-06-06T03:32:48.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:48.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:48.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:32:48.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:32:48.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:48 smithi131 ceph-mon[36687]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T03:32:49.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.461+0000 7f64994da700 1 -- 172.21.15.40:0/409490365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6494102f50 msgr2=0x7f6494105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:32:49.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.461+0000 7f64994da700 1 --2- 172.21.15.40:0/409490365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6494102f50 0x7f6494105340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f648400b0a0 tx=0x7f648400b3b0 comp rx=0 tx=0).stop 2024-06-06T03:32:49.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.461+0000 7f64994da700 1 -- 172.21.15.40:0/409490365 shutdown_connections 2024-06-06T03:32:49.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.461+0000 7f64994da700 1 --2- 172.21.15.40:0/409490365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64941081b0 0x7f649410a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:49.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.461+0000 7f64994da700 1 --2- 172.21.15.40:0/409490365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6494102f50 0x7f6494105340 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:49.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.461+0000 7f64994da700 1 --2- 172.21.15.40:0/409490365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6494105880 0x7f6494107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:49.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.461+0000 7f64994da700 1 -- 172.21.15.40:0/409490365 >> 172.21.15.40:0/409490365 conn(0x7f64940fc980 msgr2=0x7f64940feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:32:49.462 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:49 smithi040 ceph-mon[32220]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:32:49.462 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:49 smithi040 ceph-mon[32220]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T03:32:49.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.461+0000 7f64994da700 1 -- 172.21.15.40:0/409490365 shutdown_connections 2024-06-06T03:32:49.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.461+0000 7f64994da700 1 -- 172.21.15.40:0/409490365 wait complete. 2024-06-06T03:32:49.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.462+0000 7f64994da700 1 Processor -- start 2024-06-06T03:32:49.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.462+0000 7f64994da700 1 -- start start 2024-06-06T03:32:49.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.462+0000 7f64994da700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6494102f50 0x7f649419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:32:49.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.462+0000 7f64994da700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6494105880 0x7f649419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:32:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.462+0000 7f64994da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64941081b0 0x7f64941a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:32:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.462+0000 7f64994da700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f649410e110 con 0x7f64941081b0 2024-06-06T03:32:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.462+0000 7f64994da700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f649410df90 con 0x7f6494102f50 2024-06-06T03:32:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.462+0000 7f64994da700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f649410e290 con 0x7f6494105880 2024-06-06T03:32:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.462+0000 7f64937fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64941081b0 0x7f64941a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:32:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.462+0000 7f64937fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64941081b0 0x7f64941a1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42890/0 (socket says 172.21.15.40:42890) 2024-06-06T03:32:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.462+0000 7f64937fe700 1 -- 172.21.15.40:0/1295173693 learned_addr learned my addr 172.21.15.40:0/1295173693 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:32:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.462+0000 7f6492ffd700 1 --2- 172.21.15.40:0/1295173693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6494102f50 0x7f649419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:32:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.463+0000 7f64927fc700 1 --2- 172.21.15.40:0/1295173693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6494105880 0x7f649419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:32:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.463+0000 7f64937fe700 1 -- 172.21.15.40:0/1295173693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6494105880 msgr2=0x7f649419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:32:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.463+0000 7f64937fe700 1 --2- 172.21.15.40:0/1295173693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6494105880 0x7f649419ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.463+0000 7f64937fe700 1 -- 172.21.15.40:0/1295173693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6494102f50 msgr2=0x7f649419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:32:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.463+0000 7f64937fe700 1 --2- 172.21.15.40:0/1295173693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6494102f50 0x7f649419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:32:49.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.463+0000 7f64937fe700 1 -- 172.21.15.40:0/1295173693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64941a2540 con 0x7f64941081b0 2024-06-06T03:32:49.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.463+0000 7f64937fe700 1 --2- 172.21.15.40:0/1295173693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64941081b0 0x7f64941a1ea0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f648c00c900 tx=0x7f648c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:32:49.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.463+0000 7f647bfff700 1 -- 172.21.15.40:0/1295173693 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c004460 con 0x7f64941081b0 2024-06-06T03:32:49.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.463+0000 7f64994da700 1 -- 172.21.15.40:0/1295173693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64941a2830 con 0x7f64941081b0 2024-06-06T03:32:49.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.463+0000 7f647bfff700 1 -- 172.21.15.40:0/1295173693 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f648c00ce50 con 0x7f64941081b0 2024-06-06T03:32:49.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.463+0000 7f647bfff700 1 -- 172.21.15.40:0/1295173693 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c003d60 con 0x7f64941081b0 2024-06-06T03:32:49.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.463+0000 7f64994da700 1 -- 172.21.15.40:0/1295173693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64941a2d70 con 0x7f64941081b0 2024-06-06T03:32:49.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.464+0000 7f64994da700 1 -- 172.21.15.40:0/1295173693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6494069180 con 0x7f64941081b0 2024-06-06T03:32:49.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.466+0000 7f647bfff700 1 -- 172.21.15.40:0/1295173693 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f648c0076b0 con 0x7f64941081b0 2024-06-06T03:32:49.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.467+0000 7f647bfff700 1 --2- 172.21.15.40:0/1295173693 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f647c0a2c20 0x7f647c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:32:49.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.467+0000 7f6492ffd700 1 --2- 172.21.15.40:0/1295173693 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f647c0a2c20 0x7f647c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:32:49.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.467+0000 7f647bfff700 1 -- 172.21.15.40:0/1295173693 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5829+0+0 (secure 0 0 0) 0x7f648c0ce990 con 0x7f64941081b0 2024-06-06T03:32:49.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.467+0000 7f6492ffd700 1 --2- 172.21.15.40:0/1295173693 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f647c0a2c20 0x7f647c0a50e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6484009fd0 tx=0x7f648400b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:32:49.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.470+0000 7f647bfff700 1 -- 172.21.15.40:0/1295173693 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f648c013030 con 0x7f64941081b0 2024-06-06T03:32:49.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:32:49.688+0000 7f64994da700 1 -- 172.21.15.40:0/1295173693 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f64940628d0 con 0x7f647c0a2c20 2024-06-06T03:32:49.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:49 smithi131 ceph-mon[36687]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:32:49.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:49 smithi131 ceph-mon[36687]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T03:32:49.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:49 smithi104 ceph-mon[36569]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:32:49.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:49 smithi104 ceph-mon[36569]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T03:32:51.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:50 smithi131 ceph-mon[36687]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T03:32:51.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:50 smithi131 ceph-mon[36687]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:32:51.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:50 smithi131 ceph-mon[36687]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:32:51.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:32:51.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:32:51.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:50 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:51.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:50 smithi040 ceph-mon[32220]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T03:32:51.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:50 smithi040 ceph-mon[32220]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:32:51.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:50 smithi040 ceph-mon[32220]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:32:51.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:32:51.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:32:51.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:50 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:51.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:50 smithi104 ceph-mon[36569]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T03:32:51.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:50 smithi104 ceph-mon[36569]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:32:51.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:50 smithi104 ceph-mon[36569]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:32:51.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:32:51.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:32:51.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:50 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:32:53.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:52 smithi131 ceph-mon[36687]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-06T03:32:53.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:52 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1479455465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b2b3555-6870-4ead-be50-b635799265a9"}]: dispatch 2024-06-06T03:32:53.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:52 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1479455465' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b2b3555-6870-4ead-be50-b635799265a9"}]': finished 2024-06-06T03:32:53.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:52 smithi131 ceph-mon[36687]: osdmap e64: 10 total, 9 up, 10 in 2024-06-06T03:32:53.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:52 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:32:53.051 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:52 smithi040 ceph-mon[32220]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-06T03:32:53.051 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:52 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1479455465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b2b3555-6870-4ead-be50-b635799265a9"}]: dispatch 2024-06-06T03:32:53.051 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:52 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1479455465' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b2b3555-6870-4ead-be50-b635799265a9"}]': finished 2024-06-06T03:32:53.051 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:52 smithi040 ceph-mon[32220]: osdmap e64: 10 total, 9 up, 10 in 2024-06-06T03:32:53.051 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:52 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:32:53.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:52 smithi104 ceph-mon[36569]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-06T03:32:53.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:52 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1479455465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b2b3555-6870-4ead-be50-b635799265a9"}]: dispatch 2024-06-06T03:32:53.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:52 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1479455465' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b2b3555-6870-4ead-be50-b635799265a9"}]': finished 2024-06-06T03:32:53.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:52 smithi104 ceph-mon[36569]: osdmap e64: 10 total, 9 up, 10 in 2024-06-06T03:32:53.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:52 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:32:54.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:53 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/4205286658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:32:54.062 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:53 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/4205286658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:32:54.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:53 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/4205286658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:32:55.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:54 smithi131 ceph-mon[36687]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-06T03:32:55.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:54 smithi040 ceph-mon[32220]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-06T03:32:55.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:54 smithi104 ceph-mon[36569]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-06T03:32:57.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:56 smithi131 ceph-mon[36687]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-06-06T03:32:57.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:56 smithi040 ceph-mon[32220]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-06-06T03:32:57.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:56 smithi104 ceph-mon[36569]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-06-06T03:32:59.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:32:58 smithi131 ceph-mon[36687]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T03:32:59.069 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:32:58 smithi040 ceph-mon[32220]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T03:32:59.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:32:58 smithi104 ceph-mon[36569]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T03:33:01.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:00 smithi131 ceph-mon[36687]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T03:33:01.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:00 smithi040 ceph-mon[32220]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T03:33:01.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:00 smithi104 ceph-mon[36569]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T03:33:01.991 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:01 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T03:33:01.991 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:01 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:01.991 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:01 smithi104 ceph-mon[36569]: Deploying daemon osd.9 on smithi040 2024-06-06T03:33:02.030 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:01 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T03:33:02.030 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:01 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:02.030 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:01 smithi131 ceph-mon[36687]: Deploying daemon osd.9 on smithi040 2024-06-06T03:33:02.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:01 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T03:33:02.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:01 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:02.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:01 smithi040 ceph-mon[32220]: Deploying daemon osd.9 on smithi040 2024-06-06T03:33:02.998 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:02 smithi040 ceph-mon[32220]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:03.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:02 smithi131 ceph-mon[36687]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:03.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:02 smithi104 ceph-mon[36569]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:05.026 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:04 smithi040 ceph-mon[32220]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:05.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:04 smithi131 ceph-mon[36687]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:05.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:04 smithi104 ceph-mon[36569]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:06.901 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:06 smithi040 ceph-mon[32220]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:07.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:06 smithi131 ceph-mon[36687]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:07.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:06 smithi104 ceph-mon[36569]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:08.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:08.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:08.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:33:08.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:08.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:08.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:33:08.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:08.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:08.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:33:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:09 smithi131 ceph-mon[36687]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:09.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:09 smithi040 ceph-mon[32220]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:09 smithi104 ceph-mon[36569]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:11.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:33:11.260+0000 7f647bfff700 1 -- 172.21.15.40:0/1295173693 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f64940628d0 con 0x7f647c0a2c20 2024-06-06T03:33:11.260 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 9 on host 'smithi040' 2024-06-06T03:33:11.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:33:11.263+0000 7f64994da700 1 -- 172.21.15.40:0/1295173693 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f647c0a2c20 msgr2=0x7f647c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:33:11.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:33:11.263+0000 7f64994da700 1 --2- 172.21.15.40:0/1295173693 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f647c0a2c20 0x7f647c0a50e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6484009fd0 tx=0x7f648400b7b0 comp rx=0 tx=0).stop 2024-06-06T03:33:11.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:33:11.263+0000 7f64994da700 1 -- 172.21.15.40:0/1295173693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64941081b0 msgr2=0x7f64941a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:33:11.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:33:11.263+0000 7f64994da700 1 --2- 172.21.15.40:0/1295173693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64941081b0 0x7f64941a1ea0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f648c00c900 tx=0x7f648c00cc10 comp rx=0 tx=0).stop 2024-06-06T03:33:11.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:33:11.263+0000 7f64994da700 1 -- 172.21.15.40:0/1295173693 shutdown_connections 2024-06-06T03:33:11.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:33:11.263+0000 7f64994da700 1 --2- 172.21.15.40:0/1295173693 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f647c0a2c20 0x7f647c0a50e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:11.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:33:11.263+0000 7f64994da700 1 --2- 172.21.15.40:0/1295173693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64941081b0 0x7f64941a1ea0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:11.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:33:11.263+0000 7f64994da700 1 --2- 172.21.15.40:0/1295173693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6494102f50 0x7f649419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:11.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:33:11.263+0000 7f64994da700 1 --2- 172.21.15.40:0/1295173693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6494105880 0x7f649419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:11.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:33:11.263+0000 7f64994da700 1 -- 172.21.15.40:0/1295173693 >> 172.21.15.40:0/1295173693 conn(0x7f64940fc980 msgr2=0x7f64941025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:33:11.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:33:11.263+0000 7f64994da700 1 -- 172.21.15.40:0/1295173693 shutdown_connections 2024-06-06T03:33:11.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:33:11.263+0000 7f64994da700 1 -- 172.21.15.40:0/1295173693 wait complete. 2024-06-06T03:33:11.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:11 smithi131 ceph-mon[36687]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:11.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:11 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:11.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:11 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:11.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:11 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:11.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:11 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:33:11.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:11 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:11.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:11 smithi131 ceph-mon[36687]: from='osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T03:33:11.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:11 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:11.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:11 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:33:11.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:11 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:11.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:11 smithi040 ceph-mon[32220]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:11.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:11 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:11.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:11 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:11.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:11 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:11.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:11 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:33:11.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:11 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:11.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:11 smithi040 ceph-mon[32220]: from='osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T03:33:11.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:11 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:11.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:11 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:33:11.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:11 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:11.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:11 smithi104 ceph-mon[36569]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:11.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:11 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:11.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:11 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:11.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:11 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:11.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:11 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:33:11.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:11 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:11.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:11 smithi104 ceph-mon[36569]: from='osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T03:33:11.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:11 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:11.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:11 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:33:11.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:11 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:12.202 DEBUG:teuthology.orchestra.run.smithi040:osd.9> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.9.service 2024-06-06T03:33:12.204 INFO:tasks.cephadm:Deploying osd.10 on smithi104 with /dev/vg_nvme/lv_1... 2024-06-06T03:33:12.204 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T03:33:12.449 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T03:33:12.475 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:12 smithi040 ceph-mon[32220]: from='osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T03:33:12.475 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:12 smithi040 ceph-mon[32220]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T03:33:12.475 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:12 smithi040 ceph-mon[32220]: from='osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-06-06T03:33:12.475 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:12 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:33:12.475 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:33:12 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:33:12.259+0000 7f1a8ad83700 -1 osd.9 0 waiting for initial osdmap 2024-06-06T03:33:12.475 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:33:12 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:33:12.271+0000 7f1a86b76700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T03:33:12.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:12 smithi131 ceph-mon[36687]: from='osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T03:33:12.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:12 smithi131 ceph-mon[36687]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T03:33:12.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:12 smithi131 ceph-mon[36687]: from='osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-06-06T03:33:12.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:12 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:33:12.571 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:12 smithi104 ceph-mon[36569]: from='osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T03:33:12.571 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:12 smithi104 ceph-mon[36569]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T03:33:12.571 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:12 smithi104 ceph-mon[36569]: from='osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-06-06T03:33:12.571 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:12 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:33:13.528 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:13 smithi040 ceph-mon[32220]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:13.528 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:13 smithi040 ceph-mon[32220]: from='osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-06-06T03:33:13.528 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:13 smithi040 ceph-mon[32220]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T03:33:13.528 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:13 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:33:13.528 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:13 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:33:13.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:13 smithi131 ceph-mon[36687]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:13.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:13 smithi131 ceph-mon[36687]: from='osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-06-06T03:33:13.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:13 smithi131 ceph-mon[36687]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T03:33:13.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:13 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:33:13.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:13 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:33:13.588 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:13 smithi104 ceph-mon[36569]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:13.589 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:13 smithi104 ceph-mon[36569]: from='osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-06-06T03:33:13.589 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:13 smithi104 ceph-mon[36569]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T03:33:13.589 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:13 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:33:13.589 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:13 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:33:14.267 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:14 smithi104 ceph-mon[36569]: purged_snaps scrub starts 2024-06-06T03:33:14.467 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T03:33:14.467 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T03:33:14.467 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-06-06T03:33:14.467 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-06-06T03:33:14.468 INFO:teuthology.orchestra.run.smithi104.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0154804 s, 677 MB/s 2024-06-06T03:33:14.468 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-06-06T03:33:14.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:14 smithi131 ceph-mon[36687]: purged_snaps scrub starts 2024-06-06T03:33:14.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:14 smithi131 ceph-mon[36687]: purged_snaps scrub ok 2024-06-06T03:33:14.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:14 smithi131 ceph-mon[36687]: osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190] boot 2024-06-06T03:33:14.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:14 smithi131 ceph-mon[36687]: osdmap e67: 10 total, 10 up, 10 in 2024-06-06T03:33:14.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:14 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:33:14.571 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:14 smithi040 ceph-mon[32220]: purged_snaps scrub starts 2024-06-06T03:33:14.571 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:14 smithi040 ceph-mon[32220]: purged_snaps scrub ok 2024-06-06T03:33:14.571 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:14 smithi040 ceph-mon[32220]: osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190] boot 2024-06-06T03:33:14.571 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:14 smithi040 ceph-mon[32220]: osdmap e67: 10 total, 10 up, 10 in 2024-06-06T03:33:14.571 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:14 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:33:14.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:14 smithi104 ceph-mon[36569]: purged_snaps scrub ok 2024-06-06T03:33:14.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:14 smithi104 ceph-mon[36569]: osd.9 [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190] boot 2024-06-06T03:33:14.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:14 smithi104 ceph-mon[36569]: osdmap e67: 10 total, 10 up, 10 in 2024-06-06T03:33:14.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:14 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:33:15.266 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_1 2024-06-06T03:33:15.504 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T03:33:15.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:15 smithi131 ceph-mon[36687]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:15.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:15 smithi131 ceph-mon[36687]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T03:33:15.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:15 smithi040 ceph-mon[32220]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:15.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:15 smithi040 ceph-mon[32220]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T03:33:15.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:15 smithi104 ceph-mon[36569]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-06T03:33:15.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:15 smithi104 ceph-mon[36569]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T03:33:16.713 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:16 smithi104 ceph-mon[36569]: Detected new or changed devices on smithi040 2024-06-06T03:33:16.713 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:16 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:16.713 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:16 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:16.713 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:16 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:16.713 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:16 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:16.713 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:16 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:16.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:16 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:16.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:16 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi040 to 4273M 2024-06-06T03:33:16.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:16 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:16.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:16 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:16.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:16 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:33:16.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:16 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:16.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:16 smithi104 ceph-mon[36569]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:16.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:16 smithi040 ceph-mon[32220]: Detected new or changed devices on smithi040 2024-06-06T03:33:16.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:16 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:16.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:16 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:16.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:16 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:16.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:16 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:16.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:16 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:16.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:16 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:16.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:16 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi040 to 4273M 2024-06-06T03:33:16.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:16 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:16.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:16 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:16.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:16 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:33:16.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:16 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:16.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:16 smithi040 ceph-mon[32220]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:17.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:16 smithi131 ceph-mon[36687]: Detected new or changed devices on smithi040 2024-06-06T03:33:17.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:16 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:17.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:16 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:17.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:16 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:17.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:16 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:17.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:16 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:17.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:16 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:17.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:16 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi040 to 4273M 2024-06-06T03:33:17.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:16 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:17.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:16 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:17.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:16 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:33:17.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:16 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:17.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:16 smithi131 ceph-mon[36687]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:17.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.268+0000 7f1e350d1700 1 -- 172.21.15.104:0/1956098813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e301040b0 msgr2=0x7f1e3010a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:33:17.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.268+0000 7f1e350d1700 1 --2- 172.21.15.104:0/1956098813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e301040b0 0x7f1e3010a960 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800b0d0 tx=0x7f1e2800b3e0 comp rx=0 tx=0).stop 2024-06-06T03:33:17.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.269+0000 7f1e350d1700 1 -- 172.21.15.104:0/1956098813 shutdown_connections 2024-06-06T03:33:17.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.269+0000 7f1e350d1700 1 --2- 172.21.15.104:0/1956098813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e30102510 0x7f1e30102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:17.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.269+0000 7f1e350d1700 1 --2- 172.21.15.104:0/1956098813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e301040b0 0x7f1e3010a960 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:17.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.269+0000 7f1e350d1700 1 --2- 172.21.15.104:0/1956098813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e30103710 0x7f1e30103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:17.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.269+0000 7f1e350d1700 1 -- 172.21.15.104:0/1956098813 >> 172.21.15.104:0/1956098813 conn(0x7f1e300fdab0 msgr2=0x7f1e300ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:33:17.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.269+0000 7f1e350d1700 1 -- 172.21.15.104:0/1956098813 shutdown_connections 2024-06-06T03:33:17.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.269+0000 7f1e350d1700 1 -- 172.21.15.104:0/1956098813 wait complete. 2024-06-06T03:33:17.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.269+0000 7f1e350d1700 1 Processor -- start 2024-06-06T03:33:17.301 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e350d1700 1 -- start start 2024-06-06T03:33:17.301 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e350d1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e30102510 0x7f1e30198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:33:17.301 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e350d1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e30103710 0x7f1e30198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:33:17.301 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e350d1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e301040b0 0x7f1e3019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:33:17.301 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e350d1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3010e180 con 0x7f1e301040b0 2024-06-06T03:33:17.301 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e350d1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3010e000 con 0x7f1e30102510 2024-06-06T03:33:17.301 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e350d1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3010e300 con 0x7f1e30103710 2024-06-06T03:33:17.301 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e2ed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e30102510 0x7f1e30198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:33:17.301 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e2ed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e30102510 0x7f1e30198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41408/0 (socket says 172.21.15.104:41408) 2024-06-06T03:33:17.301 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e2ed9d700 1 -- 172.21.15.104:0/3238635590 learned_addr learned my addr 172.21.15.104:0/3238635590 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-06-06T03:33:17.301 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e2f59e700 1 --2- 172.21.15.104:0/3238635590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e301040b0 0x7f1e3019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:33:17.301 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e2e59c700 1 --2- 172.21.15.104:0/3238635590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e30103710 0x7f1e30198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:33:17.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e2ed9d700 1 -- 172.21.15.104:0/3238635590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e30103710 msgr2=0x7f1e30198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:33:17.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e2ed9d700 1 --2- 172.21.15.104:0/3238635590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e30103710 0x7f1e30198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:17.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e2ed9d700 1 -- 172.21.15.104:0/3238635590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e301040b0 msgr2=0x7f1e3019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:33:17.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e2ed9d700 1 --2- 172.21.15.104:0/3238635590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e301040b0 0x7f1e3019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:17.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e2ed9d700 1 -- 172.21.15.104:0/3238635590 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e301a03d0 con 0x7f1e30102510 2024-06-06T03:33:17.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e2ed9d700 1 --2- 172.21.15.104:0/3238635590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e30102510 0x7f1e30198770 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000b500 tx=0x7f1e2000b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:33:17.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e17fff700 1 -- 172.21.15.104:0/3238635590 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e20015020 con 0x7f1e30102510 2024-06-06T03:33:17.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e350d1700 1 -- 172.21.15.104:0/3238635590 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e301a06c0 con 0x7f1e30102510 2024-06-06T03:33:17.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e17fff700 1 -- 172.21.15.104:0/3238635590 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e20011d10 con 0x7f1e30102510 2024-06-06T03:33:17.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.270+0000 7f1e17fff700 1 -- 172.21.15.104:0/3238635590 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2000f2e0 con 0x7f1e30102510 2024-06-06T03:33:17.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.271+0000 7f1e350d1700 1 -- 172.21.15.104:0/3238635590 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e301a0c80 con 0x7f1e30102510 2024-06-06T03:33:17.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.271+0000 7f1e350d1700 1 -- 172.21.15.104:0/3238635590 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e30069180 con 0x7f1e30102510 2024-06-06T03:33:17.302 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.273+0000 7f1e17fff700 1 -- 172.21.15.104:0/3238635590 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1e20018020 con 0x7f1e30102510 2024-06-06T03:33:17.303 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.274+0000 7f1e17fff700 1 --2- 172.21.15.104:0/3238635590 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1e180a2c20 0x7f1e180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:33:17.303 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.274+0000 7f1e17fff700 1 -- 172.21.15.104:0/3238635590 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f1e200ce560 con 0x7f1e30102510 2024-06-06T03:33:17.303 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.274+0000 7f1e2e59c700 1 --2- 172.21.15.104:0/3238635590 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1e180a2c20 0x7f1e180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:33:17.303 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.275+0000 7f1e2e59c700 1 --2- 172.21.15.104:0/3238635590 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1e180a2c20 0x7f1e180a50e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1e24005950 tx=0x7f1e240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:33:17.303 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.277+0000 7f1e17fff700 1 -- 172.21.15.104:0/3238635590 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1e2001f020 con 0x7f1e30102510 2024-06-06T03:33:17.482 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:17.483+0000 7f1e350d1700 1 -- 172.21.15.104:0/3238635590 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1e300628d0 con 0x7f1e180a2c20 2024-06-06T03:33:17.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:17 smithi040 ceph-mon[32220]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:33:17.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:33:17.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:33:17.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:17 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:17.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:17 smithi104 ceph-mon[36569]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:33:17.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:17 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:33:17.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:17 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:33:17.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:17 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:18.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:17 smithi131 ceph-mon[36687]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:33:18.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:17 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:33:18.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:17 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:33:18.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:17 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:18.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:18 smithi040 ceph-mon[32220]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:18.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:18 smithi104 ceph-mon[36569]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:19.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:18 smithi131 ceph-mon[36687]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:20 smithi131 ceph-mon[36687]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:20 smithi131 ceph-mon[36687]: from='client.? 172.21.15.104:0/1748989564' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70049e24-94ce-458e-97e3-dbcf9de05444"}]: dispatch 2024-06-06T03:33:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:20 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70049e24-94ce-458e-97e3-dbcf9de05444"}]: dispatch 2024-06-06T03:33:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:20 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70049e24-94ce-458e-97e3-dbcf9de05444"}]': finished 2024-06-06T03:33:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:20 smithi131 ceph-mon[36687]: osdmap e69: 11 total, 10 up, 11 in 2024-06-06T03:33:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:20 smithi040 ceph-mon[32220]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:20 smithi040 ceph-mon[32220]: from='client.? 172.21.15.104:0/1748989564' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70049e24-94ce-458e-97e3-dbcf9de05444"}]: dispatch 2024-06-06T03:33:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:20 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70049e24-94ce-458e-97e3-dbcf9de05444"}]: dispatch 2024-06-06T03:33:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:20 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70049e24-94ce-458e-97e3-dbcf9de05444"}]': finished 2024-06-06T03:33:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:20 smithi040 ceph-mon[32220]: osdmap e69: 11 total, 10 up, 11 in 2024-06-06T03:33:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:21.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:20 smithi104 ceph-mon[36569]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:21.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:20 smithi104 ceph-mon[36569]: from='client.? 172.21.15.104:0/1748989564' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70049e24-94ce-458e-97e3-dbcf9de05444"}]: dispatch 2024-06-06T03:33:21.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:20 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70049e24-94ce-458e-97e3-dbcf9de05444"}]: dispatch 2024-06-06T03:33:21.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:20 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70049e24-94ce-458e-97e3-dbcf9de05444"}]': finished 2024-06-06T03:33:21.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:20 smithi104 ceph-mon[36569]: osdmap e69: 11 total, 10 up, 11 in 2024-06-06T03:33:21.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:22.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:21 smithi131 ceph-mon[36687]: from='client.? 172.21.15.104:0/1642059348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:33:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:21 smithi040 ceph-mon[32220]: from='client.? 172.21.15.104:0/1642059348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:33:22.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:21 smithi104 ceph-mon[36569]: from='client.? 172.21.15.104:0/1642059348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:33:23.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:22 smithi131 ceph-mon[36687]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:23.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:22 smithi040 ceph-mon[32220]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:23.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:22 smithi104 ceph-mon[36569]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:25.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:24 smithi131 ceph-mon[36687]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:24 smithi040 ceph-mon[32220]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:25.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:24 smithi104 ceph-mon[36569]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:26.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:26 smithi104 ceph-mon[36569]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:27.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:26 smithi131 ceph-mon[36687]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:27.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:26 smithi040 ceph-mon[32220]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:28.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:27 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T03:33:28.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:27 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:28.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:27 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T03:33:28.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:27 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:28.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:27 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T03:33:28.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:27 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:28.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:28 smithi104 ceph-mon[36569]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:28.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:28 smithi104 ceph-mon[36569]: Deploying daemon osd.10 on smithi104 2024-06-06T03:33:29.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:28 smithi131 ceph-mon[36687]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:29.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:28 smithi131 ceph-mon[36687]: Deploying daemon osd.10 on smithi104 2024-06-06T03:33:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:28 smithi040 ceph-mon[32220]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:28 smithi040 ceph-mon[32220]: Deploying daemon osd.10 on smithi104 2024-06-06T03:33:31.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:30 smithi131 ceph-mon[36687]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:31.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:30 smithi040 ceph-mon[32220]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:31.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:30 smithi104 ceph-mon[36569]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:32.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:32 smithi104 ceph-mon[36569]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:33.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:32 smithi131 ceph-mon[36687]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:32 smithi040 ceph-mon[32220]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:34.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:34 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:34.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:34 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:34.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:34 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:33:34.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:34 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:34.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:34 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:34.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:34 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:33:34.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:34 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:34.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:34 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:34.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:34 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:33:35.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:35 smithi131 ceph-mon[36687]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:35.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:35 smithi040 ceph-mon[32220]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:35.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:35 smithi104 ceph-mon[36569]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:36.733 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:36 smithi104 ceph-mon[36569]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:36.781 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:36 smithi131 ceph-mon[36687]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:36.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:36 smithi040 ceph-mon[32220]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:37.101 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 10 on host 'smithi104' 2024-06-06T03:33:37.101 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:37.101+0000 7f1e17fff700 1 -- 172.21.15.104:0/3238635590 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1e300628d0 con 0x7f1e180a2c20 2024-06-06T03:33:37.103 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:37.104+0000 7f1e350d1700 1 -- 172.21.15.104:0/3238635590 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1e180a2c20 msgr2=0x7f1e180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:33:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:37.104+0000 7f1e350d1700 1 --2- 172.21.15.104:0/3238635590 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1e180a2c20 0x7f1e180a50e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1e24005950 tx=0x7f1e240058e0 comp rx=0 tx=0).stop 2024-06-06T03:33:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:37.104+0000 7f1e350d1700 1 -- 172.21.15.104:0/3238635590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e30102510 msgr2=0x7f1e30198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:33:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:37.104+0000 7f1e350d1700 1 --2- 172.21.15.104:0/3238635590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e30102510 0x7f1e30198770 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000b500 tx=0x7f1e2000b810 comp rx=0 tx=0).stop 2024-06-06T03:33:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:37.104+0000 7f1e350d1700 1 -- 172.21.15.104:0/3238635590 shutdown_connections 2024-06-06T03:33:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:37.104+0000 7f1e350d1700 1 --2- 172.21.15.104:0/3238635590 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1e180a2c20 0x7f1e180a50e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:37.104+0000 7f1e350d1700 1 --2- 172.21.15.104:0/3238635590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e301040b0 0x7f1e3019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:37.104+0000 7f1e350d1700 1 --2- 172.21.15.104:0/3238635590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e30102510 0x7f1e30198770 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:37.104+0000 7f1e350d1700 1 --2- 172.21.15.104:0/3238635590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e30103710 0x7f1e30198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:37.104+0000 7f1e350d1700 1 -- 172.21.15.104:0/3238635590 >> 172.21.15.104:0/3238635590 conn(0x7f1e300fdab0 msgr2=0x7f1e301069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:33:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:37.104+0000 7f1e350d1700 1 -- 172.21.15.104:0/3238635590 shutdown_connections 2024-06-06T03:33:37.104 INFO:teuthology.orchestra.run.smithi104.stderr:2024-06-06T03:33:37.104+0000 7f1e350d1700 1 -- 172.21.15.104:0/3238635590 wait complete. 2024-06-06T03:33:37.972 DEBUG:teuthology.orchestra.run.smithi104:osd.10> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.10.service 2024-06-06T03:33:37.974 INFO:tasks.cephadm:Deploying osd.11 on smithi131 with /dev/vg_nvme/lv_1... 2024-06-06T03:33:37.975 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T03:33:38.224 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:33:38.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:38 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:38.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:38 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:38.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:38 smithi131 ceph-mon[36687]: from='osd.10 [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T03:33:38.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:38 smithi131 ceph-mon[36687]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T03:33:38.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:38 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:38.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:38 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:38.291 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:38 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:38.291 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:38 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:38.291 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:38 smithi104 ceph-mon[36569]: from='osd.10 [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T03:33:38.291 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:38 smithi104 ceph-mon[36569]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T03:33:38.291 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:38 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:38.292 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:38 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:38.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:38 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:38.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:38 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:38.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:38 smithi040 ceph-mon[32220]: from='osd.10 [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T03:33:38.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:38 smithi040 ceph-mon[32220]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T03:33:38.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:38 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:38.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:38 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:39.276 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:39 smithi104 ceph-mon[36569]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:39.276 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:39 smithi104 ceph-mon[36569]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T03:33:39.276 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:39 smithi104 ceph-mon[36569]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T03:33:39.276 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:39 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:39.276 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:39 smithi104 ceph-mon[36569]: from='osd.10 [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:33:39.276 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:39 smithi104 ceph-mon[36569]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:33:39.277 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:33:39 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:33:39.088+0000 7fd40eb77700 -1 osd.10 0 waiting for initial osdmap 2024-06-06T03:33:39.277 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:33:39 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:33:39.098+0000 7fd40a96a700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T03:33:39.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:39 smithi131 ceph-mon[36687]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:39.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:39 smithi131 ceph-mon[36687]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T03:33:39.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:39 smithi131 ceph-mon[36687]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T03:33:39.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:39 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:39.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:39 smithi131 ceph-mon[36687]: from='osd.10 [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:33:39.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:39 smithi131 ceph-mon[36687]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:33:39.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:39 smithi040 ceph-mon[32220]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:39.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:39 smithi040 ceph-mon[32220]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T03:33:39.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:39 smithi040 ceph-mon[32220]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T03:33:39.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:39 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:39.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:39 smithi040 ceph-mon[32220]: from='osd.10 [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:33:39.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:39 smithi040 ceph-mon[32220]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-06-06T03:33:40.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:40 smithi131 ceph-mon[36687]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-06-06T03:33:40.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:40 smithi131 ceph-mon[36687]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T03:33:40.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:40 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:40.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:40 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:40.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:40 smithi104 ceph-mon[36569]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-06-06T03:33:40.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:40 smithi104 ceph-mon[36569]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T03:33:40.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:40 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:40.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:40 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:40.440 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T03:33:40.440 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T03:33:40.440 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-06-06T03:33:40.440 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-06-06T03:33:40.440 INFO:teuthology.orchestra.run.smithi131.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113716 s, 922 MB/s 2024-06-06T03:33:40.440 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 2024-06-06T03:33:40.440 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-06-06T03:33:40.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:40 smithi040 ceph-mon[32220]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-06-06T03:33:40.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:40 smithi040 ceph-mon[32220]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T03:33:40.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:40 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:40.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:40 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:41.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_1 2024-06-06T03:33:41.386 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:33:41.508 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:41 smithi131 ceph-mon[36687]: purged_snaps scrub starts 2024-06-06T03:33:41.508 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:41 smithi131 ceph-mon[36687]: purged_snaps scrub ok 2024-06-06T03:33:41.509 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:41 smithi131 ceph-mon[36687]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:41.509 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:41 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:41.509 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:41 smithi131 ceph-mon[36687]: osd.10 [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832] boot 2024-06-06T03:33:41.509 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:41 smithi131 ceph-mon[36687]: osdmap e72: 11 total, 11 up, 11 in 2024-06-06T03:33:41.509 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:41 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:41.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:41 smithi040 ceph-mon[32220]: purged_snaps scrub starts 2024-06-06T03:33:41.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:41 smithi040 ceph-mon[32220]: purged_snaps scrub ok 2024-06-06T03:33:41.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:41 smithi040 ceph-mon[32220]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:41.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:41 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:41.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:41 smithi040 ceph-mon[32220]: osd.10 [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832] boot 2024-06-06T03:33:41.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:41 smithi040 ceph-mon[32220]: osdmap e72: 11 total, 11 up, 11 in 2024-06-06T03:33:41.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:41 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:41.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:41 smithi104 ceph-mon[36569]: purged_snaps scrub starts 2024-06-06T03:33:41.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:41 smithi104 ceph-mon[36569]: purged_snaps scrub ok 2024-06-06T03:33:41.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:41 smithi104 ceph-mon[36569]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-06-06T03:33:41.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:41 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:41.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:41 smithi104 ceph-mon[36569]: osd.10 [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832] boot 2024-06-06T03:33:41.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:41 smithi104 ceph-mon[36569]: osdmap e72: 11 total, 11 up, 11 in 2024-06-06T03:33:41.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:41 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:33:42.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T03:33:42.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: Detected new or changed devices on smithi104 2024-06-06T03:33:42.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:42.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:42.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:42.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:42.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:42.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:42.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi104 to 4272M 2024-06-06T03:33:42.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:42.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:42.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:33:42.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:42.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:33:42.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:42.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:33:42.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:42.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T03:33:42.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: Detected new or changed devices on smithi104 2024-06-06T03:33:42.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:42.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:42.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:42.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:42.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:42.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:42.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi104 to 4272M 2024-06-06T03:33:42.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:42.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:42.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:33:42.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:42.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:33:42.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:42.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:33:42.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:42.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T03:33:42.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: Detected new or changed devices on smithi104 2024-06-06T03:33:42.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:42.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:42.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:42.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:42.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:42.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:33:42.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi104 to 4272M 2024-06-06T03:33:42.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:42.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:42.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:33:42.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:42.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:33:42.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:42.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:33:42.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:33:43.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.162+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/1168301612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb0108220 msgr2=0x7fdcb010a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:33:43.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.162+0000 7fdcb5ae1700 1 --2- 172.21.15.131:0/1168301612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb0108220 0x7fdcb010a690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdca800b0a0 tx=0x7fdca800b3b0 comp rx=0 tx=0).stop 2024-06-06T03:33:43.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.162+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/1168301612 shutdown_connections 2024-06-06T03:33:43.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.162+0000 7fdcb5ae1700 1 --2- 172.21.15.131:0/1168301612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb0108220 0x7fdcb010a690 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:43.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.162+0000 7fdcb5ae1700 1 --2- 172.21.15.131:0/1168301612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb0102fc0 0x7fdcb01053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:43.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.162+0000 7fdcb5ae1700 1 --2- 172.21.15.131:0/1168301612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcb01058f0 0x7fdcb0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:43.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.162+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/1168301612 >> 172.21.15.131:0/1168301612 conn(0x7fdcb00fc9f0 msgr2=0x7fdcb00fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:33:43.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.162+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/1168301612 shutdown_connections 2024-06-06T03:33:43.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.162+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/1168301612 wait complete. 2024-06-06T03:33:43.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.163+0000 7fdcb5ae1700 1 Processor -- start 2024-06-06T03:33:43.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.163+0000 7fdcb5ae1700 1 -- start start 2024-06-06T03:33:43.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcb5ae1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb0102fc0 0x7fdcb0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:33:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcb5ae1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb01058f0 0x7fdcb0198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:33:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcb5ae1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcb0108220 0x7fdcb019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:33:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcb5ae1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb010e1e0 con 0x7fdcb01058f0 2024-06-06T03:33:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcb5ae1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb010e060 con 0x7fdcb0108220 2024-06-06T03:33:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcb5ae1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb010e360 con 0x7fdcb0102fc0 2024-06-06T03:33:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcaf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb0102fc0 0x7fdcb0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:33:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcaf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb0102fc0 0x7fdcb0198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37808/0 (socket says 172.21.15.131:37808) 2024-06-06T03:33:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcaf7fe700 1 -- 172.21.15.131:0/811238128 learned_addr learned my addr 172.21.15.131:0/811238128 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-06-06T03:33:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcaeffd700 1 --2- 172.21.15.131:0/811238128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb01058f0 0x7fdcb0198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:33:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcaf7fe700 1 -- 172.21.15.131:0/811238128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcb0108220 msgr2=0x7fdcb019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:33:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcaf7fe700 1 --2- 172.21.15.131:0/811238128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcb0108220 0x7fdcb019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcaf7fe700 1 -- 172.21.15.131:0/811238128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb01058f0 msgr2=0x7fdcb0198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:33:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcaf7fe700 1 --2- 172.21.15.131:0/811238128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb01058f0 0x7fdcb0198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:33:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcaf7fe700 1 -- 172.21.15.131:0/811238128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb01a03d0 con 0x7fdcb0102fc0 2024-06-06T03:33:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcaeffd700 1 --2- 172.21.15.131:0/811238128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb01058f0 0x7fdcb0198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:33:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcaf7fe700 1 --2- 172.21.15.131:0/811238128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb0102fc0 0x7fdcb0198770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdca000b560 tx=0x7fdca000b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:33:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcacff9700 1 -- 172.21.15.131:0/811238128 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca0015020 con 0x7fdcb0102fc0 2024-06-06T03:33:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcacff9700 1 -- 172.21.15.131:0/811238128 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca0011d10 con 0x7fdcb0102fc0 2024-06-06T03:33:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/811238128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb01a06c0 con 0x7fdcb0102fc0 2024-06-06T03:33:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcacff9700 1 -- 172.21.15.131:0/811238128 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca000f2e0 con 0x7fdcb0102fc0 2024-06-06T03:33:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.164+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/811238128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb01a0c80 con 0x7fdcb0102fc0 2024-06-06T03:33:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.165+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/811238128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb0069180 con 0x7fdcb0102fc0 2024-06-06T03:33:43.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.167+0000 7fdcacff9700 1 -- 172.21.15.131:0/811238128 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdca0018020 con 0x7fdcb0102fc0 2024-06-06T03:33:43.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.167+0000 7fdcacff9700 1 --2- 172.21.15.131:0/811238128 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fdc980a2c20 0x7fdc980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:33:43.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.167+0000 7fdcacff9700 1 -- 172.21.15.131:0/811238128 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7fdca00ce7b0 con 0x7fdcb0102fc0 2024-06-06T03:33:43.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.167+0000 7fdcaeffd700 1 --2- 172.21.15.131:0/811238128 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fdc980a2c20 0x7fdc980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:33:43.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.168+0000 7fdcaeffd700 1 --2- 172.21.15.131:0/811238128 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fdc980a2c20 0x7fdc980a50e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdca4007900 tx=0x7fdca4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:33:43.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.171+0000 7fdcacff9700 1 -- 172.21.15.131:0/811238128 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdca001f020 con 0x7fdcb0102fc0 2024-06-06T03:33:43.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:43 smithi131 ceph-mon[36687]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:33:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:43 smithi131 ceph-mon[36687]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T03:33:43.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:33:43.373+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/811238128 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdcb00628d0 con 0x7fdc980a2c20 2024-06-06T03:33:43.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:43 smithi040 ceph-mon[32220]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:33:43.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:43 smithi040 ceph-mon[32220]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T03:33:43.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:43 smithi104 ceph-mon[36569]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:33:43.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:43 smithi104 ceph-mon[36569]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T03:33:44.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:44 smithi131 ceph-mon[36687]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T03:33:44.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:44 smithi131 ceph-mon[36687]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:33:44.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:44 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:33:44.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:44 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:33:44.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:44 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:44.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:44 smithi104 ceph-mon[36569]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T03:33:44.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:44 smithi104 ceph-mon[36569]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:33:44.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:44 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:33:44.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:44 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:33:44.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:44 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:44.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:44 smithi040 ceph-mon[32220]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T03:33:44.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:44 smithi040 ceph-mon[32220]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:33:44.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:44 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T03:33:44.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:44 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T03:33:44.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:44 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:45.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:45 smithi131 ceph-mon[36687]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:33:45.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:45 smithi131 ceph-mon[36687]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T03:33:45.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:45 smithi040 ceph-mon[32220]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:33:45.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:45 smithi040 ceph-mon[32220]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T03:33:45.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:45 smithi104 ceph-mon[36569]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:33:45.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:45 smithi104 ceph-mon[36569]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T03:33:47.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:47 smithi131 ceph-mon[36687]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-06-06T03:33:47.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:47 smithi131 ceph-mon[36687]: from='client.? 172.21.15.131:0/3926769405' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80447548-1fbe-4892-ac13-eb4bb6acf10d"}]: dispatch 2024-06-06T03:33:47.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:47 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80447548-1fbe-4892-ac13-eb4bb6acf10d"}]: dispatch 2024-06-06T03:33:47.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:47 smithi131 ceph-mon[36687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80447548-1fbe-4892-ac13-eb4bb6acf10d"}]': finished 2024-06-06T03:33:47.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:47 smithi131 ceph-mon[36687]: osdmap e77: 12 total, 11 up, 12 in 2024-06-06T03:33:47.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:47 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:33:47.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:47 smithi131 ceph-mon[36687]: from='client.? 172.21.15.131:0/588956441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:33:47.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:47 smithi040 ceph-mon[32220]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-06-06T03:33:47.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:47 smithi040 ceph-mon[32220]: from='client.? 172.21.15.131:0/3926769405' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80447548-1fbe-4892-ac13-eb4bb6acf10d"}]: dispatch 2024-06-06T03:33:47.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:47 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80447548-1fbe-4892-ac13-eb4bb6acf10d"}]: dispatch 2024-06-06T03:33:47.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:47 smithi040 ceph-mon[32220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80447548-1fbe-4892-ac13-eb4bb6acf10d"}]': finished 2024-06-06T03:33:47.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:47 smithi040 ceph-mon[32220]: osdmap e77: 12 total, 11 up, 12 in 2024-06-06T03:33:47.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:33:47.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:47 smithi040 ceph-mon[32220]: from='client.? 172.21.15.131:0/588956441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:33:47.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:47 smithi104 ceph-mon[36569]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-06-06T03:33:47.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:47 smithi104 ceph-mon[36569]: from='client.? 172.21.15.131:0/3926769405' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80447548-1fbe-4892-ac13-eb4bb6acf10d"}]: dispatch 2024-06-06T03:33:47.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:47 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80447548-1fbe-4892-ac13-eb4bb6acf10d"}]: dispatch 2024-06-06T03:33:47.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:47 smithi104 ceph-mon[36569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80447548-1fbe-4892-ac13-eb4bb6acf10d"}]': finished 2024-06-06T03:33:47.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:47 smithi104 ceph-mon[36569]: osdmap e77: 12 total, 11 up, 12 in 2024-06-06T03:33:47.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:47 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:33:47.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:47 smithi104 ceph-mon[36569]: from='client.? 172.21.15.131:0/588956441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T03:33:49.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:49 smithi131 ceph-mon[36687]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-06-06T03:33:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:49 smithi040 ceph-mon[32220]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-06-06T03:33:49.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:49 smithi104 ceph-mon[36569]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-06-06T03:33:51.414 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:51 smithi131 ceph-mon[36687]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-06-06T03:33:51.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:51 smithi040 ceph-mon[32220]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-06-06T03:33:51.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:51 smithi104 ceph-mon[36569]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-06-06T03:33:53.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:53 smithi131 ceph-mon[36687]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T03:33:53.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:53 smithi040 ceph-mon[32220]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T03:33:53.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:53 smithi104 ceph-mon[36569]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T03:33:54.389 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:54 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T03:33:54.389 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:54 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:54.481 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:54 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T03:33:54.481 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:54 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:54.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:54 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T03:33:54.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:54 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:33:55.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:55 smithi131 ceph-mon[36687]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-06-06T03:33:55.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:55 smithi131 ceph-mon[36687]: Deploying daemon osd.11 on smithi131 2024-06-06T03:33:55.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:55 smithi040 ceph-mon[32220]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-06-06T03:33:55.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:55 smithi040 ceph-mon[32220]: Deploying daemon osd.11 on smithi131 2024-06-06T03:33:55.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:55 smithi104 ceph-mon[36569]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-06-06T03:33:55.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:55 smithi104 ceph-mon[36569]: Deploying daemon osd.11 on smithi131 2024-06-06T03:33:57.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:57 smithi131 ceph-mon[36687]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:33:57.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:57 smithi040 ceph-mon[32220]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:33:57.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:57 smithi104 ceph-mon[36569]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:33:59.503 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:33:59 smithi131 ceph-mon[36687]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:33:59.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:33:59 smithi040 ceph-mon[32220]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:33:59.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:33:59 smithi104 ceph-mon[36569]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:34:01.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:00 smithi040 ceph-mon[32220]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:34:01.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:01.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:01.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:34:01.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:00 smithi104 ceph-mon[36569]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:34:01.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:00 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:01.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:00 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:01.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:00 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:34:01.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:00 smithi131 ceph-mon[36687]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:34:01.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:00 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:01.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:00 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:01.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:00 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:34:03.009 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:02 smithi131 ceph-mon[36687]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:34:03.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:02 smithi040 ceph-mon[32220]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:34:03.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:02 smithi104 ceph-mon[36569]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:34:03.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:03.644+0000 7fdcacff9700 1 -- 172.21.15.131:0/811238128 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdcb00628d0 con 0x7fdc980a2c20 2024-06-06T03:34:03.644 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 11 on host 'smithi131' 2024-06-06T03:34:03.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:03.647+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/811238128 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fdc980a2c20 msgr2=0x7fdc980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:03.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:03.647+0000 7fdcb5ae1700 1 --2- 172.21.15.131:0/811238128 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fdc980a2c20 0x7fdc980a50e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdca4007900 tx=0x7fdca4008040 comp rx=0 tx=0).stop 2024-06-06T03:34:03.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:03.647+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/811238128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb0102fc0 msgr2=0x7fdcb0198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:03.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:03.647+0000 7fdcb5ae1700 1 --2- 172.21.15.131:0/811238128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb0102fc0 0x7fdcb0198770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdca000b560 tx=0x7fdca000b920 comp rx=0 tx=0).stop 2024-06-06T03:34:03.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:03.647+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/811238128 shutdown_connections 2024-06-06T03:34:03.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:03.647+0000 7fdcb5ae1700 1 --2- 172.21.15.131:0/811238128 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fdc980a2c20 0x7fdc980a50e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:03.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:03.647+0000 7fdcb5ae1700 1 --2- 172.21.15.131:0/811238128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb0102fc0 0x7fdcb0198770 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:03.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:03.647+0000 7fdcb5ae1700 1 --2- 172.21.15.131:0/811238128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcb0108220 0x7fdcb019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:03.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:03.647+0000 7fdcb5ae1700 1 --2- 172.21.15.131:0/811238128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb01058f0 0x7fdcb0198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:03.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:03.647+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/811238128 >> 172.21.15.131:0/811238128 conn(0x7fdcb00fc9f0 msgr2=0x7fdcb0103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:03.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:03.647+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/811238128 shutdown_connections 2024-06-06T03:34:03.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:03.647+0000 7fdcb5ae1700 1 -- 172.21.15.131:0/811238128 wait complete. 2024-06-06T03:34:04.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:04.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:34:04.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:34:04.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:03 smithi131 ceph-mon[36687]: from='osd.11 [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T03:34:04.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:03 smithi131 ceph-mon[36687]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T03:34:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:34:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:34:04.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:03 smithi040 ceph-mon[32220]: from='osd.11 [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T03:34:04.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:03 smithi040 ceph-mon[32220]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T03:34:04.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:04.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:34:04.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:04.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:34:04.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:03 smithi104 ceph-mon[36569]: from='osd.11 [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T03:34:04.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:03 smithi104 ceph-mon[36569]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T03:34:04.370 DEBUG:teuthology.orchestra.run.smithi131:osd.11> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.11.service 2024-06-06T03:34:04.372 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-06T03:34:04.372 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-06T03:34:04.635 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:34:05.042 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:04 smithi040 ceph-mon[32220]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:34:05.042 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:04 smithi040 ceph-mon[32220]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T03:34:05.042 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:04 smithi040 ceph-mon[32220]: from='osd.11 [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:34:05.042 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:04 smithi040 ceph-mon[32220]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T03:34:05.042 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:04 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:05.043 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:04 smithi040 ceph-mon[32220]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:34:05.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:04 smithi131 ceph-mon[36687]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:34:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:04 smithi131 ceph-mon[36687]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T03:34:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:04 smithi131 ceph-mon[36687]: from='osd.11 [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:34:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:04 smithi131 ceph-mon[36687]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T03:34:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:04 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:04 smithi131 ceph-mon[36687]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:34:05.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:04 smithi104 ceph-mon[36569]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:34:05.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:04 smithi104 ceph-mon[36569]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T03:34:05.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:04 smithi104 ceph-mon[36569]: from='osd.11 [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:34:05.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:04 smithi104 ceph-mon[36569]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T03:34:05.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:04 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:05.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:04 smithi104 ceph-mon[36569]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-06-06T03:34:06.126 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:34:06 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:34:06.111+0000 7f888d0de700 -1 osd.11 0 waiting for initial osdmap 2024-06-06T03:34:06.126 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:34:06 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:34:06.126+0000 7f8884ec9700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T03:34:06.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.602+0000 7fd712c7d700 1 -- 172.21.15.40:0/3814714958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd70c1025f0 msgr2=0x7fd70c102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.602+0000 7fd712c7d700 1 --2- 172.21.15.40:0/3814714958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd70c1025f0 0x7fd70c102a50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0099c0 tx=0x7fd6fc009cd0 comp rx=0 tx=0).stop 2024-06-06T03:34:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.602+0000 7fd712c7d700 1 -- 172.21.15.40:0/3814714958 shutdown_connections 2024-06-06T03:34:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.602+0000 7fd712c7d700 1 --2- 172.21.15.40:0/3814714958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd70c1013f0 0x7fd70c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.602+0000 7fd712c7d700 1 --2- 172.21.15.40:0/3814714958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd70c102f90 0x7fd70c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.602+0000 7fd712c7d700 1 --2- 172.21.15.40:0/3814714958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd70c1025f0 0x7fd70c102a50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.602+0000 7fd712c7d700 1 -- 172.21.15.40:0/3814714958 >> 172.21.15.40:0/3814714958 conn(0x7fd70c0fc990 msgr2=0x7fd70c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.602+0000 7fd712c7d700 1 -- 172.21.15.40:0/3814714958 shutdown_connections 2024-06-06T03:34:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.602+0000 7fd712c7d700 1 -- 172.21.15.40:0/3814714958 wait complete. 2024-06-06T03:34:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd712c7d700 1 Processor -- start 2024-06-06T03:34:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd712c7d700 1 -- start start 2024-06-06T03:34:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd712c7d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd70c1013f0 0x7fd70c1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:06.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd712c7d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd70c1025f0 0x7fd70c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:06.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd712c7d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd70c102f90 0x7fd70c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:06.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd712c7d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c10cfe0 con 0x7fd70c102f90 2024-06-06T03:34:06.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd712c7d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c10ce60 con 0x7fd70c1025f0 2024-06-06T03:34:06.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd712c7d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c10d160 con 0x7fd70c1013f0 2024-06-06T03:34:06.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd710a19700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd70c1013f0 0x7fd70c1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:06.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd71121a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd70c102f90 0x7fd70c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:06.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd710a19700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd70c1013f0 0x7fd70c1975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.40:44234/0 (socket says 172.21.15.40:44234) 2024-06-06T03:34:06.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd710a19700 1 -- 172.21.15.40:0/2059798618 learned_addr learned my addr 172.21.15.40:0/2059798618 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:34:06.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd70bfff700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd70c1025f0 0x7fd70c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:06.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd710a19700 1 -- 172.21.15.40:0/2059798618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd70c1025f0 msgr2=0x7fd70c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:06.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd710a19700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd70c1025f0 0x7fd70c197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:06.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd710a19700 1 -- 172.21.15.40:0/2059798618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd70c102f90 msgr2=0x7fd70c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:06.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd710a19700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd70c102f90 0x7fd70c19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:06.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd710a19700 1 -- 172.21.15.40:0/2059798618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70c19f250 con 0x7fd70c1013f0 2024-06-06T03:34:06.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd70bfff700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd70c1025f0 0x7fd70c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:34:06.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.603+0000 7fd71121a700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd70c102f90 0x7fd70c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:34:06.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.604+0000 7fd710a19700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd70c1013f0 0x7fd70c1975f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400e980 tx=0x7fd6f400ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:06.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.604+0000 7fd709ffb700 1 -- 172.21.15.40:0/2059798618 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400cbd0 con 0x7fd70c1013f0 2024-06-06T03:34:06.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.604+0000 7fd712c7d700 1 -- 172.21.15.40:0/2059798618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70c19f540 con 0x7fd70c1013f0 2024-06-06T03:34:06.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.604+0000 7fd709ffb700 1 -- 172.21.15.40:0/2059798618 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6f4004510 con 0x7fd70c1013f0 2024-06-06T03:34:06.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.604+0000 7fd709ffb700 1 -- 172.21.15.40:0/2059798618 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4010430 con 0x7fd70c1013f0 2024-06-06T03:34:06.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.604+0000 7fd712c7d700 1 -- 172.21.15.40:0/2059798618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70c19fa80 con 0x7fd70c1013f0 2024-06-06T03:34:06.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.605+0000 7fd712c7d700 1 -- 172.21.15.40:0/2059798618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd70c069180 con 0x7fd70c1013f0 2024-06-06T03:34:06.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.606+0000 7fd709ffb700 1 -- 172.21.15.40:0/2059798618 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd6f4044b30 con 0x7fd70c1013f0 2024-06-06T03:34:06.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.607+0000 7fd709ffb700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd6f80a2c20 0x7fd6f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:06.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.607+0000 7fd70bfff700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd6f80a2c20 0x7fd6f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:06.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.607+0000 7fd709ffb700 1 -- 172.21.15.40:0/2059798618 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7fd6f40cedf0 con 0x7fd70c1013f0 2024-06-06T03:34:06.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.607+0000 7fd70bfff700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd6f80a2c20 0x7fd6f80a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc005cd0 tx=0x7fd6fc005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:06.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.610+0000 7fd709ffb700 1 -- 172.21.15.40:0/2059798618 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd6f4044e20 con 0x7fd70c1013f0 2024-06-06T03:34:06.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.817+0000 7fd712c7d700 1 -- 172.21.15.40:0/2059798618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fd70c02dca0 con 0x7fd70c1013f0 2024-06-06T03:34:06.818 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:06 smithi040 ceph-mon[32220]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-06-06T03:34:06.818 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:06 smithi040 ceph-mon[32220]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T03:34:06.818 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:06 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:06.818 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:06 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:06.818 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:06 smithi040 ceph-mon[32220]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:34:06.818 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:06 smithi040 ceph-mon[32220]: from='osd.11 ' entity='osd.11' 2024-06-06T03:34:06.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.818+0000 7fd709ffb700 1 -- 172.21.15.40:0/2059798618 <== mon.1 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7fd6f409c270 con 0x7fd70c1013f0 2024-06-06T03:34:06.819 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:34:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.820+0000 7fd712c7d700 1 -- 172.21.15.40:0/2059798618 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd6f80a2c20 msgr2=0x7fd6f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.820+0000 7fd712c7d700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd6f80a2c20 0x7fd6f80a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc005cd0 tx=0x7fd6fc005c40 comp rx=0 tx=0).stop 2024-06-06T03:34:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.820+0000 7fd712c7d700 1 -- 172.21.15.40:0/2059798618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd70c1013f0 msgr2=0x7fd70c1975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.820+0000 7fd712c7d700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd70c1013f0 0x7fd70c1975f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400e980 tx=0x7fd6f400ec90 comp rx=0 tx=0).stop 2024-06-06T03:34:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.820+0000 7fd712c7d700 1 -- 172.21.15.40:0/2059798618 shutdown_connections 2024-06-06T03:34:06.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.820+0000 7fd712c7d700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd6f80a2c20 0x7fd6f80a50e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:06.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.820+0000 7fd712c7d700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd70c1013f0 0x7fd70c1975f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:06.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.820+0000 7fd712c7d700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd70c102f90 0x7fd70c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:06.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.820+0000 7fd712c7d700 1 --2- 172.21.15.40:0/2059798618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd70c1025f0 0x7fd70c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:06.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.820+0000 7fd712c7d700 1 -- 172.21.15.40:0/2059798618 >> 172.21.15.40:0/2059798618 conn(0x7fd70c0fc990 msgr2=0x7fd70c1058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:06.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.820+0000 7fd712c7d700 1 -- 172.21.15.40:0/2059798618 shutdown_connections 2024-06-06T03:34:06.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:06.820+0000 7fd712c7d700 1 -- 172.21.15.40:0/2059798618 wait complete. 2024-06-06T03:34:07.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:06 smithi131 ceph-mon[36687]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-06-06T03:34:07.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:06 smithi131 ceph-mon[36687]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T03:34:07.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:06 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:07.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:06 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:07.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:06 smithi131 ceph-mon[36687]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:34:07.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:06 smithi131 ceph-mon[36687]: from='osd.11 ' entity='osd.11' 2024-06-06T03:34:07.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:06 smithi104 ceph-mon[36569]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-06-06T03:34:07.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:06 smithi104 ceph-mon[36569]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T03:34:07.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:06 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:07.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:06 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:07.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:06 smithi104 ceph-mon[36569]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-06T03:34:07.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:06 smithi104 ceph-mon[36569]: from='osd.11 ' entity='osd.11' 2024-06-06T03:34:07.445 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1717644820,"num_in_osds":12,"osd_in_since":1717644826,"num_remapped_pgs":0} 2024-06-06T03:34:07.701 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:07 smithi131 ceph-mon[36687]: purged_snaps scrub starts 2024-06-06T03:34:07.702 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:07 smithi131 ceph-mon[36687]: purged_snaps scrub ok 2024-06-06T03:34:07.702 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:07 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:07.702 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:07 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2059798618' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T03:34:07.702 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:07 smithi131 ceph-mon[36687]: osd.11 [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670] boot 2024-06-06T03:34:07.702 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:07 smithi131 ceph-mon[36687]: osdmap e80: 12 total, 12 up, 12 in 2024-06-06T03:34:07.702 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:07 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:07.722 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:07 smithi040 ceph-mon[32220]: purged_snaps scrub starts 2024-06-06T03:34:07.723 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:07 smithi040 ceph-mon[32220]: purged_snaps scrub ok 2024-06-06T03:34:07.723 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:07 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:07.723 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:07 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2059798618' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T03:34:07.723 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:07 smithi040 ceph-mon[32220]: osd.11 [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670] boot 2024-06-06T03:34:07.723 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:07 smithi040 ceph-mon[32220]: osdmap e80: 12 total, 12 up, 12 in 2024-06-06T03:34:07.723 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:07 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:08.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:07 smithi104 ceph-mon[36569]: purged_snaps scrub starts 2024-06-06T03:34:08.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:07 smithi104 ceph-mon[36569]: purged_snaps scrub ok 2024-06-06T03:34:08.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:07 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:08.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:07 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2059798618' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T03:34:08.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:07 smithi104 ceph-mon[36569]: osd.11 [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670] boot 2024-06-06T03:34:08.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:07 smithi104 ceph-mon[36569]: osdmap e80: 12 total, 12 up, 12 in 2024-06-06T03:34:08.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:07 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:34:08.446 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-06T03:34:08.696 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:34:09.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:08 smithi131 ceph-mon[36687]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:09.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:08 smithi131 ceph-mon[36687]: Detected new or changed devices on smithi131 2024-06-06T03:34:09.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:09.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:09.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:34:09.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:34:09.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:34:09.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:34:09.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:08 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi131 to 4272M 2024-06-06T03:34:09.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:09.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:09.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:34:09.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:09.059 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:08 smithi040 ceph-mon[32220]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:09.060 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:08 smithi040 ceph-mon[32220]: Detected new or changed devices on smithi131 2024-06-06T03:34:09.060 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:09.060 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:09.060 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:34:09.060 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:34:09.060 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:34:09.060 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:34:09.060 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:08 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi131 to 4272M 2024-06-06T03:34:09.060 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:09.060 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:09.060 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:34:09.060 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:09.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:08 smithi104 ceph-mon[36569]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:09.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:08 smithi104 ceph-mon[36569]: Detected new or changed devices on smithi131 2024-06-06T03:34:09.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:09.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:09.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:34:09.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:34:09.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:34:09.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:34:09.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:08 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi131 to 4272M 2024-06-06T03:34:09.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:09.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:09.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:34:09.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:10.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:09 smithi131 ceph-mon[36687]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T03:34:10.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:09 smithi040 ceph-mon[32220]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T03:34:10.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:09 smithi104 ceph-mon[36569]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T03:34:10.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.544+0000 7f69949f7700 1 -- 172.21.15.40:0/4181373554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69901013f0 msgr2=0x7f6990101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:10.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.544+0000 7f69949f7700 1 --2- 172.21.15.40:0/4181373554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69901013f0 0x7f6990101810 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f69840099c0 tx=0x7f6984009cd0 comp rx=0 tx=0).stop 2024-06-06T03:34:10.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.544+0000 7f69949f7700 1 -- 172.21.15.40:0/4181373554 shutdown_connections 2024-06-06T03:34:10.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.544+0000 7f69949f7700 1 --2- 172.21.15.40:0/4181373554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6990102f90 0x7f6990109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:10.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.544+0000 7f69949f7700 1 --2- 172.21.15.40:0/4181373554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69901013f0 0x7f6990101810 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:10.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.544+0000 7f69949f7700 1 --2- 172.21.15.40:0/4181373554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69901025f0 0x7f6990102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:10.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.544+0000 7f69949f7700 1 -- 172.21.15.40:0/4181373554 >> 172.21.15.40:0/4181373554 conn(0x7f69900fc990 msgr2=0x7f69900fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:10.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.544+0000 7f69949f7700 1 -- 172.21.15.40:0/4181373554 shutdown_connections 2024-06-06T03:34:10.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.544+0000 7f69949f7700 1 -- 172.21.15.40:0/4181373554 wait complete. 2024-06-06T03:34:10.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.544+0000 7f69949f7700 1 Processor -- start 2024-06-06T03:34:10.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f69949f7700 1 -- start start 2024-06-06T03:34:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f69949f7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69901013f0 0x7f6990105380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f69949f7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69901025f0 0x7f69901039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f69949f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6990102f90 0x7f6990103f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f69949f7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f699010d050 con 0x7f6990102f90 2024-06-06T03:34:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f69949f7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f699010ced0 con 0x7f69901013f0 2024-06-06T03:34:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f69949f7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f699010d1d0 con 0x7f69901025f0 2024-06-06T03:34:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f698ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6990102f90 0x7f6990103f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f698ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6990102f90 0x7f6990103f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33024/0 (socket says 172.21.15.40:33024) 2024-06-06T03:34:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f698ed9d700 1 -- 172.21.15.40:0/951589929 learned_addr learned my addr 172.21.15.40:0/951589929 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:34:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f698dd9b700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69901025f0 0x7f69901039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f698e59c700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69901013f0 0x7f6990105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f698ed9d700 1 -- 172.21.15.40:0/951589929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69901025f0 msgr2=0x7f69901039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f698ed9d700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69901025f0 0x7f69901039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:10.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f698ed9d700 1 -- 172.21.15.40:0/951589929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69901013f0 msgr2=0x7f6990105380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:10.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f698ed9d700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69901013f0 0x7f6990105380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:10.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f698ed9d700 1 -- 172.21.15.40:0/951589929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69901047c0 con 0x7f6990102f90 2024-06-06T03:34:10.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.545+0000 7f698e59c700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69901013f0 0x7f6990105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:34:10.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.546+0000 7f698ed9d700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6990102f90 0x7f6990103f10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6988007ee0 tx=0x7f698800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:10.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.546+0000 7f697f7fe700 1 -- 172.21.15.40:0/951589929 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698800dbd0 con 0x7f6990102f90 2024-06-06T03:34:10.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.546+0000 7f69949f7700 1 -- 172.21.15.40:0/951589929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69901a28f0 con 0x7f6990102f90 2024-06-06T03:34:10.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.546+0000 7f697f7fe700 1 -- 172.21.15.40:0/951589929 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f698800dd70 con 0x7f6990102f90 2024-06-06T03:34:10.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.546+0000 7f697f7fe700 1 -- 172.21.15.40:0/951589929 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988005a60 con 0x7f6990102f90 2024-06-06T03:34:10.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.546+0000 7f69949f7700 1 -- 172.21.15.40:0/951589929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69901a2eb0 con 0x7f6990102f90 2024-06-06T03:34:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.547+0000 7f69949f7700 1 -- 172.21.15.40:0/951589929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6990069180 con 0x7f6990102f90 2024-06-06T03:34:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.548+0000 7f697f7fe700 1 -- 172.21.15.40:0/951589929 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f698801a020 con 0x7f6990102f90 2024-06-06T03:34:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.548+0000 7f697f7fe700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f69780a2c20 0x7f69780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:10.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.549+0000 7f698e59c700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f69780a2c20 0x7f69780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:10.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.549+0000 7f697f7fe700 1 -- 172.21.15.40:0/951589929 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f69880ce2a0 con 0x7f6990102f90 2024-06-06T03:34:10.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.549+0000 7f698e59c700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f69780a2c20 0x7f69780a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f698400b580 tx=0x7f6984005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:10.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.553+0000 7f697f7fe700 1 -- 172.21.15.40:0/951589929 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6988010070 con 0x7f6990102f90 2024-06-06T03:34:10.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.760+0000 7f69949f7700 1 -- 172.21.15.40:0/951589929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f69901a3290 con 0x7f6990102f90 2024-06-06T03:34:10.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.761+0000 7f697f7fe700 1 -- 172.21.15.40:0/951589929 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f69901a3290 con 0x7f6990102f90 2024-06-06T03:34:10.761 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:34:10.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.763+0000 7f69949f7700 1 -- 172.21.15.40:0/951589929 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f69780a2c20 msgr2=0x7f69780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:10.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.763+0000 7f69949f7700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f69780a2c20 0x7f69780a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f698400b580 tx=0x7f6984005fb0 comp rx=0 tx=0).stop 2024-06-06T03:34:10.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.763+0000 7f69949f7700 1 -- 172.21.15.40:0/951589929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6990102f90 msgr2=0x7f6990103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:10.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.763+0000 7f69949f7700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6990102f90 0x7f6990103f10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6988007ee0 tx=0x7f698800d3b0 comp rx=0 tx=0).stop 2024-06-06T03:34:10.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.763+0000 7f69949f7700 1 -- 172.21.15.40:0/951589929 shutdown_connections 2024-06-06T03:34:10.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.763+0000 7f69949f7700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f69780a2c20 0x7f69780a50e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:10.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.763+0000 7f69949f7700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6990102f90 0x7f6990103f10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:10.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.763+0000 7f69949f7700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69901013f0 0x7f6990105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:10.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.763+0000 7f69949f7700 1 --2- 172.21.15.40:0/951589929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69901025f0 0x7f69901039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:10.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.763+0000 7f69949f7700 1 -- 172.21.15.40:0/951589929 >> 172.21.15.40:0/951589929 conn(0x7f69900fc990 msgr2=0x7f69900fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:10.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.763+0000 7f69949f7700 1 -- 172.21.15.40:0/951589929 shutdown_connections 2024-06-06T03:34:10.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:10.763+0000 7f69949f7700 1 -- 172.21.15.40:0/951589929 wait complete. 2024-06-06T03:34:11.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:10 smithi040 ceph-mon[32220]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:11.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:10 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/951589929' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T03:34:11.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:10 smithi104 ceph-mon[36569]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:11.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:10 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/951589929' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T03:34:11.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:10 smithi131 ceph-mon[36687]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:11.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:10 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/951589929' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T03:34:11.597 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1717644847,"num_in_osds":12,"osd_in_since":1717644826,"num_remapped_pgs":0} 2024-06-06T03:34:11.598 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T03:34:11.850 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:34:13.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:12 smithi040 ceph-mon[32220]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:13.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:12 smithi104 ceph-mon[36569]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:13.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:12 smithi131 ceph-mon[36687]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:13.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.639+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2581210467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4acc107840 msgr2=0x7f4acc10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:13.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.639+0000 7f4ad26f1700 1 --2- 172.21.15.40:0/2581210467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4acc107840 0x7f4acc10f3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400b0d0 tx=0x7f4ac400b3e0 comp rx=0 tx=0).stop 2024-06-06T03:34:13.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.640+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2581210467 shutdown_connections 2024-06-06T03:34:13.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.640+0000 7f4ad26f1700 1 --2- 172.21.15.40:0/2581210467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4acc107840 0x7f4acc10f3c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:13.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.640+0000 7f4ad26f1700 1 --2- 172.21.15.40:0/2581210467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4acc102740 0x7f4acc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:13.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.640+0000 7f4ad26f1700 1 --2- 172.21.15.40:0/2581210467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4acc1030f0 0x7f4acc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:13.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.640+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2581210467 >> 172.21.15.40:0/2581210467 conn(0x7f4acc0795f0 msgr2=0x7f4acc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:13.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.640+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2581210467 shutdown_connections 2024-06-06T03:34:13.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.640+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2581210467 wait complete. 2024-06-06T03:34:13.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.640+0000 7f4ad26f1700 1 Processor -- start 2024-06-06T03:34:13.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.640+0000 7f4ad26f1700 1 -- start start 2024-06-06T03:34:13.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.641+0000 7f4ad26f1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4acc102740 0x7f4acc19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:13.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.641+0000 7f4ad26f1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4acc1030f0 0x7f4acc19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:13.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.641+0000 7f4ad26f1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4acc107840 0x7f4acc198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:13.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.641+0000 7f4ad26f1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc112760 con 0x7f4acc1030f0 2024-06-06T03:34:13.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.641+0000 7f4ad26f1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1125e0 con 0x7f4acc107840 2024-06-06T03:34:13.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.641+0000 7f4ad26f1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1128e0 con 0x7f4acc102740 2024-06-06T03:34:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.641+0000 7f4acb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4acc1030f0 0x7f4acc19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.641+0000 7f4acb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4acc1030f0 0x7f4acc19ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33050/0 (socket says 172.21.15.40:33050) 2024-06-06T03:34:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.641+0000 7f4acb7fe700 1 -- 172.21.15.40:0/2751978671 learned_addr learned my addr 172.21.15.40:0/2751978671 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:34:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.641+0000 7f4acbfff700 1 --2- 172.21.15.40:0/2751978671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4acc102740 0x7f4acc19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.641+0000 7f4ad0c8e700 1 --2- 172.21.15.40:0/2751978671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4acc107840 0x7f4acc198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.642+0000 7f4acb7fe700 1 -- 172.21.15.40:0/2751978671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4acc102740 msgr2=0x7f4acc19e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.642+0000 7f4acb7fe700 1 --2- 172.21.15.40:0/2751978671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4acc102740 0x7f4acc19e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.642+0000 7f4acb7fe700 1 -- 172.21.15.40:0/2751978671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4acc107840 msgr2=0x7f4acc198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.642+0000 7f4acb7fe700 1 --2- 172.21.15.40:0/2751978671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4acc107840 0x7f4acc198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:13.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.642+0000 7f4acb7fe700 1 -- 172.21.15.40:0/2751978671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc1990e0 con 0x7f4acc1030f0 2024-06-06T03:34:13.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.642+0000 7f4acb7fe700 1 --2- 172.21.15.40:0/2751978671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4acc1030f0 0x7f4acc19ecb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000d950 tx=0x7f4ac000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:13.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.642+0000 7f4ac97fa700 1 -- 172.21.15.40:0/2751978671 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0016070 con 0x7f4acc1030f0 2024-06-06T03:34:13.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.642+0000 7f4ac97fa700 1 -- 172.21.15.40:0/2751978671 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac0009e20 con 0x7f4acc1030f0 2024-06-06T03:34:13.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.642+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2751978671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4acc1993d0 con 0x7f4acc1030f0 2024-06-06T03:34:13.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.642+0000 7f4ac97fa700 1 -- 172.21.15.40:0/2751978671 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac000f620 con 0x7f4acc1030f0 2024-06-06T03:34:13.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.642+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2751978671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4acc1a5640 con 0x7f4acc1030f0 2024-06-06T03:34:13.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.643+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2751978671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4acc069180 con 0x7f4acc1030f0 2024-06-06T03:34:13.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.644+0000 7f4ac97fa700 1 -- 172.21.15.40:0/2751978671 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4ac0019070 con 0x7f4acc1030f0 2024-06-06T03:34:13.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.645+0000 7f4ac97fa700 1 --2- 172.21.15.40:0/2751978671 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4ab40a2ca0 0x7f4ab40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:13.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.645+0000 7f4acbfff700 1 --2- 172.21.15.40:0/2751978671 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4ab40a2ca0 0x7f4ab40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:13.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.645+0000 7f4ac97fa700 1 -- 172.21.15.40:0/2751978671 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4ac00ce6d0 con 0x7f4acc1030f0 2024-06-06T03:34:13.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.645+0000 7f4acbfff700 1 --2- 172.21.15.40:0/2751978671 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4ab40a2ca0 0x7f4ab40a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005950 tx=0x7f4abc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:13.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.649+0000 7f4ac97fa700 1 -- 172.21.15.40:0/2751978671 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4ac009bad0 con 0x7f4acc1030f0 2024-06-06T03:34:13.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.859+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2751978671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4acc02dca0 con 0x7f4acc1030f0 2024-06-06T03:34:13.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.860+0000 7f4ac97fa700 1 -- 172.21.15.40:0/2751978671 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18479 (secure 0 0 0) 0x7f4ac009b3d0 con 0x7f4acc1030f0 2024-06-06T03:34:13.860 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:34:13.861 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":81,"fsid":"77c25d68-23b4-11ef-bc9b-c7b262605968","created":"2024-06-06T03:26:53.165489+0000","modified":"2024-06-06T03:34:08.777863+0000","last_up_change":"2024-06-06T03:34:07.114448+0000","last_in_change":"2024-06-06T03:33:46.152948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T03:30:14.102785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63c0a320-a2f2-4b84-87e7-68501d335285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6803","nonce":3453526793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6805","nonce":3453526793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6809","nonce":3453526793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6807","nonce":3453526793}]},"public_addr":"172.21.15.40:6803/3453526793","cluster_addr":"172.21.15.40:6805/3453526793","heartbeat_back_addr":"172.21.15.40:6809/3453526793","heartbeat_front_addr":"172.21.15.40:6807/3453526793","state":["exists","up"]},{"osd":1,"uuid":"70a31534-06a1-4ffa-a54d-577d45bccf98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6801","nonce":405854252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6803","nonce":405854252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6807","nonce":405854252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6805","nonce":405854252}]},"public_addr":"172.21.15.104:6801/405854252","cluster_addr":"172.21.15.104:6803/405854252","heartbeat_back_addr":"172.21.15.104:6807/405854252","heartbeat_front_addr":"172.21.15.104:6805/405854252","state":["exists","up"]},{"osd":2,"uuid":"f5a89f43-e533-45f0-a777-bd3e9bfa8072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6801","nonce":3919221141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6803","nonce":3919221141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6807","nonce":3919221141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6805","nonce":3919221141}]},"public_addr":"172.21.15.131:6801/3919221141","cluster_addr":"172.21.15.131:6803/3919221141","heartbeat_back_addr":"172.21.15.131:6807/3919221141","heartbeat_front_addr":"172.21.15.131:6805/3919221141","state":["exists","up"]},{"osd":3,"uuid":"4df19390-6378-4346-a2be-29bce1dd2380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6811","nonce":2008541522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6813","nonce":2008541522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6817","nonce":2008541522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6815","nonce":2008541522}]},"public_addr":"172.21.15.40:6811/2008541522","cluster_addr":"172.21.15.40:6813/2008541522","heartbeat_back_addr":"172.21.15.40:6817/2008541522","heartbeat_front_addr":"172.21.15.40:6815/2008541522","state":["exists","up"]},{"osd":4,"uuid":"508de408-1cb1-4423-8199-db6bde9034f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6809","nonce":1153394847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6811","nonce":1153394847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6815","nonce":1153394847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6813","nonce":1153394847}]},"public_addr":"172.21.15.104:6809/1153394847","cluster_addr":"172.21.15.104:6811/1153394847","heartbeat_back_addr":"172.21.15.104:6815/1153394847","heartbeat_front_addr":"172.21.15.104:6813/1153394847","state":["exists","up"]},{"osd":5,"uuid":"a8c0cec9-a8ad-4a5d-800b-8b891ca6d836","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6809","nonce":1051933303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6811","nonce":1051933303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6815","nonce":1051933303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6813","nonce":1051933303}]},"public_addr":"172.21.15.131:6809/1051933303","cluster_addr":"172.21.15.131:6811/1051933303","heartbeat_back_addr":"172.21.15.131:6815/1051933303","heartbeat_front_addr":"172.21.15.131:6813/1051933303","state":["exists","up"]},{"osd":6,"uuid":"5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6819","nonce":2670922011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6821","nonce":2670922011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6825","nonce":2670922011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6823","nonce":2670922011}]},"public_addr":"172.21.15.40:6819/2670922011","cluster_addr":"172.21.15.40:6821/2670922011","heartbeat_back_addr":"172.21.15.40:6825/2670922011","heartbeat_front_addr":"172.21.15.40:6823/2670922011","state":["exists","up"]},{"osd":7,"uuid":"bfc536bb-6ae4-459e-afd9-8197c7420bd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6817","nonce":2353571051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6819","nonce":2353571051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6823","nonce":2353571051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6821","nonce":2353571051}]},"public_addr":"172.21.15.104:6817/2353571051","cluster_addr":"172.21.15.104:6819/2353571051","heartbeat_back_addr":"172.21.15.104:6823/2353571051","heartbeat_front_addr":"172.21.15.104:6821/2353571051","state":["exists","up"]},{"osd":8,"uuid":"1fe214f2-5ea3-4fc5-83bf-d9bccafe1529","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6817","nonce":1364533794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6819","nonce":1364533794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6823","nonce":1364533794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6821","nonce":1364533794}]},"public_addr":"172.21.15.131:6817/1364533794","cluster_addr":"172.21.15.131:6819/1364533794","heartbeat_back_addr":"172.21.15.131:6823/1364533794","heartbeat_front_addr":"172.21.15.131:6821/1364533794","state":["exists","up"]},{"osd":9,"uuid":"4b2b3555-6870-4ead-be50-b635799265a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6827","nonce":2604133190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6829","nonce":2604133190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6833","nonce":2604133190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6831","nonce":2604133190}]},"public_addr":"172.21.15.40:6827/2604133190","cluster_addr":"172.21.15.40:6829/2604133190","heartbeat_back_addr":"172.21.15.40:6833/2604133190","heartbeat_front_addr":"172.21.15.40:6831/2604133190","state":["exists","up"]},{"osd":10,"uuid":"70049e24-94ce-458e-97e3-dbcf9de05444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6825","nonce":2231164832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6827","nonce":2231164832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6831","nonce":2231164832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6829","nonce":2231164832}]},"public_addr":"172.21.15.104:6825/2231164832","cluster_addr":"172.21.15.104:6827/2231164832","heartbeat_back_addr":"172.21.15.104:6831/2231164832","heartbeat_front_addr":"172.21.15.104:6829/2231164832","state":["exists","up"]},{"osd":11,"uuid":"80447548-1fbe-4892-ac13-eb4bb6acf10d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6825","nonce":4128185670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6827","nonce":4128185670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6831","nonce":4128185670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6829","nonce":4128185670}]},"public_addr":"172.21.15.131:6825/4128185670","cluster_addr":"172.21.15.131:6827/4128185670","heartbeat_back_addr":"172.21.15.131:6831/4128185670","heartbeat_front_addr":"172.21.15.131:6829/4128185670","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:23.289784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:47.271677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:10.866792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:36.781924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:01.995848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:26.904606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:52.997438+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:19.221567+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:44.543977+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:11.948545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:38.402943+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:34:04.712200+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/23942578":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2234980855":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6801/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6800/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2301695357":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/1676880727":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6801/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6800/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/1597665640":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/513375601":"2024-06-07T03:27:19.799499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T03:34:13.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.862+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2751978671 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4ab40a2ca0 msgr2=0x7f4ab40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:13.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.862+0000 7f4ad26f1700 1 --2- 172.21.15.40:0/2751978671 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4ab40a2ca0 0x7f4ab40a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005950 tx=0x7f4abc00a300 comp rx=0 tx=0).stop 2024-06-06T03:34:13.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.862+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2751978671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4acc1030f0 msgr2=0x7f4acc19ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:13.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.862+0000 7f4ad26f1700 1 --2- 172.21.15.40:0/2751978671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4acc1030f0 0x7f4acc19ecb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000d950 tx=0x7f4ac000dc60 comp rx=0 tx=0).stop 2024-06-06T03:34:13.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.862+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2751978671 shutdown_connections 2024-06-06T03:34:13.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.862+0000 7f4ad26f1700 1 --2- 172.21.15.40:0/2751978671 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4ab40a2ca0 0x7f4ab40a5160 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:13.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.862+0000 7f4ad26f1700 1 --2- 172.21.15.40:0/2751978671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4acc102740 0x7f4acc19e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:13.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.862+0000 7f4ad26f1700 1 --2- 172.21.15.40:0/2751978671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4acc107840 0x7f4acc198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:13.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.862+0000 7f4ad26f1700 1 --2- 172.21.15.40:0/2751978671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4acc1030f0 0x7f4acc19ecb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:13.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.863+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2751978671 >> 172.21.15.40:0/2751978671 conn(0x7f4acc0795f0 msgr2=0x7f4acc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:13.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.863+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2751978671 shutdown_connections 2024-06-06T03:34:13.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:13.863+0000 7f4ad26f1700 1 -- 172.21.15.40:0/2751978671 wait complete. 2024-06-06T03:34:14.637 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-06T03:30:14.102785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-06T03:34:14.637 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-06T03:34:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:34:15.011 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:14 smithi040 ceph-mon[32220]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:15.011 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:14 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2751978671' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:34:15.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:14 smithi104 ceph-mon[36569]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:15.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:14 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2751978671' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:34:15.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:14 smithi131 ceph-mon[36687]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:15.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:14 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2751978671' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:34:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.614+0000 7f4c6a294700 1 -- 172.21.15.40:0/237843602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c64101620 msgr2=0x7f4c64101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.614+0000 7f4c6a294700 1 --2- 172.21.15.40:0/237843602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c64101620 0x7f4c64101a00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4c540099c0 tx=0x7f4c54009cd0 comp rx=0 tx=0).stop 2024-06-06T03:34:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.614+0000 7f4c6a294700 1 -- 172.21.15.40:0/237843602 shutdown_connections 2024-06-06T03:34:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.614+0000 7f4c6a294700 1 --2- 172.21.15.40:0/237843602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c641066b0 0x7f4c6410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.614+0000 7f4c6a294700 1 --2- 172.21.15.40:0/237843602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c64101620 0x7f4c64101a00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.614+0000 7f4c6a294700 1 --2- 172.21.15.40:0/237843602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c64101fd0 0x7f4c64105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.614+0000 7f4c6a294700 1 -- 172.21.15.40:0/237843602 >> 172.21.15.40:0/237843602 conn(0x7f4c64078340 msgr2=0x7f4c64078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.614+0000 7f4c6a294700 1 -- 172.21.15.40:0/237843602 shutdown_connections 2024-06-06T03:34:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.614+0000 7f4c6a294700 1 -- 172.21.15.40:0/237843602 wait complete. 2024-06-06T03:34:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.615+0000 7f4c6a294700 1 Processor -- start 2024-06-06T03:34:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.615+0000 7f4c6a294700 1 -- start start 2024-06-06T03:34:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.615+0000 7f4c6a294700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c64101620 0x7f4c64103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.615+0000 7f4c6a294700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c64101fd0 0x7f4c641035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.615+0000 7f4c6a294700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c641066b0 0x7f4c64103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.615+0000 7f4c6a294700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641116c0 con 0x7f4c64101fd0 2024-06-06T03:34:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.615+0000 7f4c6a294700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64111540 con 0x7f4c64101620 2024-06-06T03:34:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.615+0000 7f4c6a294700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64111840 con 0x7f4c641066b0 2024-06-06T03:34:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.615+0000 7f4c5bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c64101fd0 0x7f4c641035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.615+0000 7f4c5bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c64101fd0 0x7f4c641035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33066/0 (socket says 172.21.15.40:33066) 2024-06-06T03:34:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.615+0000 7f4c5bfff700 1 -- 172.21.15.40:0/3743516354 learned_addr learned my addr 172.21.15.40:0/3743516354 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:34:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.615+0000 7f4c63fff700 1 --2- 172.21.15.40:0/3743516354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c64101620 0x7f4c64103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.616+0000 7f4c68831700 1 --2- 172.21.15.40:0/3743516354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c641066b0 0x7f4c64103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.616+0000 7f4c5bfff700 1 -- 172.21.15.40:0/3743516354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c641066b0 msgr2=0x7f4c64103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.616+0000 7f4c5bfff700 1 --2- 172.21.15.40:0/3743516354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c641066b0 0x7f4c64103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.616+0000 7f4c5bfff700 1 -- 172.21.15.40:0/3743516354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c64101620 msgr2=0x7f4c64103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.616+0000 7f4c5bfff700 1 --2- 172.21.15.40:0/3743516354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c64101620 0x7f4c64103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.616+0000 7f4c5bfff700 1 -- 172.21.15.40:0/3743516354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c641a71e0 con 0x7f4c64101fd0 2024-06-06T03:34:16.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.616+0000 7f4c5bfff700 1 --2- 172.21.15.40:0/3743516354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c64101fd0 0x7f4c641035b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00e970 tx=0x7f4c4c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:16.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.616+0000 7f4c61ffb700 1 -- 172.21.15.40:0/3743516354 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00cbd0 con 0x7f4c64101fd0 2024-06-06T03:34:16.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.616+0000 7f4c61ffb700 1 -- 172.21.15.40:0/3743516354 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c4c00cd70 con 0x7f4c64101fd0 2024-06-06T03:34:16.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.616+0000 7f4c6a294700 1 -- 172.21.15.40:0/3743516354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c641a74d0 con 0x7f4c64101fd0 2024-06-06T03:34:16.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.616+0000 7f4c61ffb700 1 -- 172.21.15.40:0/3743516354 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c0056b0 con 0x7f4c64101fd0 2024-06-06T03:34:16.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.616+0000 7f4c6a294700 1 -- 172.21.15.40:0/3743516354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c641a7a10 con 0x7f4c64101fd0 2024-06-06T03:34:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.617+0000 7f4c6a294700 1 -- 172.21.15.40:0/3743516354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c64069180 con 0x7f4c64101fd0 2024-06-06T03:34:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.618+0000 7f4c61ffb700 1 -- 172.21.15.40:0/3743516354 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4c4c004750 con 0x7f4c64101fd0 2024-06-06T03:34:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.619+0000 7f4c61ffb700 1 --2- 172.21.15.40:0/3743516354 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4c500a2c20 0x7f4c500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.619+0000 7f4c63fff700 1 --2- 172.21.15.40:0/3743516354 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4c500a2c20 0x7f4c500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.619+0000 7f4c61ffb700 1 -- 172.21.15.40:0/3743516354 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4c4c0cf0c0 con 0x7f4c64101fd0 2024-06-06T03:34:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.620+0000 7f4c63fff700 1 --2- 172.21.15.40:0/3743516354 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4c500a2c20 0x7f4c500a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4c54000c00 tx=0x7f4c54005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.623+0000 7f4c61ffb700 1 -- 172.21.15.40:0/3743516354 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4c4c09c420 con 0x7f4c64101fd0 2024-06-06T03:34:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.831+0000 7f4c6a294700 1 -- 172.21.15.40:0/3743516354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c6402dca0 con 0x7f4c64101fd0 2024-06-06T03:34:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.832+0000 7f4c61ffb700 1 -- 172.21.15.40:0/3743516354 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f4c4c09bd20 con 0x7f4c64101fd0 2024-06-06T03:34:16.832 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-06-06T03:34:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.834+0000 7f4c6a294700 1 -- 172.21.15.40:0/3743516354 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4c500a2c20 msgr2=0x7f4c500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.834+0000 7f4c6a294700 1 --2- 172.21.15.40:0/3743516354 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4c500a2c20 0x7f4c500a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4c54000c00 tx=0x7f4c54005fb0 comp rx=0 tx=0).stop 2024-06-06T03:34:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.834+0000 7f4c6a294700 1 -- 172.21.15.40:0/3743516354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c64101fd0 msgr2=0x7f4c641035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.834+0000 7f4c6a294700 1 --2- 172.21.15.40:0/3743516354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c64101fd0 0x7f4c641035b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00e970 tx=0x7f4c4c00ec80 comp rx=0 tx=0).stop 2024-06-06T03:34:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.834+0000 7f4c6a294700 1 -- 172.21.15.40:0/3743516354 shutdown_connections 2024-06-06T03:34:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.834+0000 7f4c6a294700 1 --2- 172.21.15.40:0/3743516354 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4c500a2c20 0x7f4c500a50e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.834+0000 7f4c6a294700 1 --2- 172.21.15.40:0/3743516354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c641066b0 0x7f4c64103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.834+0000 7f4c6a294700 1 --2- 172.21.15.40:0/3743516354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c64101620 0x7f4c64103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.834+0000 7f4c6a294700 1 --2- 172.21.15.40:0/3743516354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c64101fd0 0x7f4c641035b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.834+0000 7f4c6a294700 1 -- 172.21.15.40:0/3743516354 >> 172.21.15.40:0/3743516354 conn(0x7f4c64078340 msgr2=0x7f4c6406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.834+0000 7f4c6a294700 1 -- 172.21.15.40:0/3743516354 shutdown_connections 2024-06-06T03:34:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:16.834+0000 7f4c6a294700 1 -- 172.21.15.40:0/3743516354 wait complete. 2024-06-06T03:34:17.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:16 smithi040 ceph-mon[32220]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:17.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:16 smithi104 ceph-mon[36569]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:17.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:16 smithi131 ceph-mon[36687]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:17.610 INFO:tasks.cephadm:Adding mds.a on smithi040 2024-06-06T03:34:17.610 INFO:tasks.cephadm:Adding mds.d on smithi040 2024-06-06T03:34:17.610 INFO:tasks.cephadm:Adding mds.g on smithi040 2024-06-06T03:34:17.610 INFO:tasks.cephadm:Adding mds.j on smithi040 2024-06-06T03:34:17.610 INFO:tasks.cephadm:Adding mds.b on smithi104 2024-06-06T03:34:17.610 INFO:tasks.cephadm:Adding mds.e on smithi104 2024-06-06T03:34:17.610 INFO:tasks.cephadm:Adding mds.h on smithi104 2024-06-06T03:34:17.611 INFO:tasks.cephadm:Adding mds.k on smithi104 2024-06-06T03:34:17.611 INFO:tasks.cephadm:Adding mds.c on smithi131 2024-06-06T03:34:17.611 INFO:tasks.cephadm:Adding mds.f on smithi131 2024-06-06T03:34:17.611 INFO:tasks.cephadm:Adding mds.i on smithi131 2024-06-06T03:34:17.611 INFO:tasks.cephadm:Adding mds.l on smithi131 2024-06-06T03:34:17.611 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l' 2024-06-06T03:34:17.859 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:34:17.980 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:17 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3743516354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T03:34:18.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:17 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3743516354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T03:34:18.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:17 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3743516354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T03:34:19.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:18 smithi040 ceph-mon[32220]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:19.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:18 smithi104 ceph-mon[36569]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:19.133 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:18 smithi131 ceph-mon[36687]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:19.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.474+0000 7f1d9898e700 1 -- 172.21.15.131:0/442333723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d941025c0 msgr2=0x7f1d94102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:19.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.474+0000 7f1d9898e700 1 --2- 172.21.15.131:0/442333723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d941025c0 0x7f1d94102a20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009990 tx=0x7f1d88009ca0 comp rx=0 tx=0).stop 2024-06-06T03:34:19.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.474+0000 7f1d9898e700 1 -- 172.21.15.131:0/442333723 shutdown_connections 2024-06-06T03:34:19.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.474+0000 7f1d9898e700 1 --2- 172.21.15.131:0/442333723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d94102f60 0x7f1d94109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:19.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.474+0000 7f1d9898e700 1 --2- 172.21.15.131:0/442333723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d941013c0 0x7f1d941017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:19.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.474+0000 7f1d9898e700 1 --2- 172.21.15.131:0/442333723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d941025c0 0x7f1d94102a20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:19.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.474+0000 7f1d9898e700 1 -- 172.21.15.131:0/442333723 >> 172.21.15.131:0/442333723 conn(0x7f1d940fc960 msgr2=0x7f1d940feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:19.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.474+0000 7f1d9898e700 1 -- 172.21.15.131:0/442333723 shutdown_connections 2024-06-06T03:34:19.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.474+0000 7f1d9898e700 1 -- 172.21.15.131:0/442333723 wait complete. 2024-06-06T03:34:19.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.475+0000 7f1d9898e700 1 Processor -- start 2024-06-06T03:34:19.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.475+0000 7f1d9898e700 1 -- start start 2024-06-06T03:34:19.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.475+0000 7f1d9898e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d941013c0 0x7f1d94198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:19.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.475+0000 7f1d9898e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d941025c0 0x7f1d94198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:19.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.475+0000 7f1d9898e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d94102f60 0x7f1d9419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.475+0000 7f1d9898e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9410cf80 con 0x7f1d941025c0 2024-06-06T03:34:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.475+0000 7f1d9898e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9410ce00 con 0x7f1d941013c0 2024-06-06T03:34:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.475+0000 7f1d9898e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9410d100 con 0x7f1d94102f60 2024-06-06T03:34:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.475+0000 7f1d92d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d94102f60 0x7f1d9419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.475+0000 7f1d92d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d94102f60 0x7f1d9419fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43416/0 (socket says 172.21.15.131:43416) 2024-06-06T03:34:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.475+0000 7f1d92d9d700 1 -- 172.21.15.131:0/3713193270 learned_addr learned my addr 172.21.15.131:0/3713193270 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-06-06T03:34:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.475+0000 7f1d91d9b700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d941025c0 0x7f1d94198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.476+0000 7f1d9259c700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d941013c0 0x7f1d94198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.476+0000 7f1d92d9d700 1 -- 172.21.15.131:0/3713193270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d941013c0 msgr2=0x7f1d94198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.476+0000 7f1d92d9d700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d941013c0 0x7f1d94198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.476+0000 7f1d92d9d700 1 -- 172.21.15.131:0/3713193270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d941025c0 msgr2=0x7f1d94198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.476+0000 7f1d92d9d700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d941025c0 0x7f1d94198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.476+0000 7f1d92d9d700 1 -- 172.21.15.131:0/3713193270 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d941a0390 con 0x7f1d94102f60 2024-06-06T03:34:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.476+0000 7f1d9259c700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d941013c0 0x7f1d94198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:34:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.476+0000 7f1d92d9d700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d94102f60 0x7f1d9419fcf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c007f10 tx=0x7f1d8c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.476+0000 7f1d837fe700 1 -- 172.21.15.131:0/3713193270 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c004bb0 con 0x7f1d94102f60 2024-06-06T03:34:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.476+0000 7f1d9898e700 1 -- 172.21.15.131:0/3713193270 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d941a0680 con 0x7f1d94102f60 2024-06-06T03:34:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.476+0000 7f1d837fe700 1 -- 172.21.15.131:0/3713193270 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d8c004d50 con 0x7f1d94102f60 2024-06-06T03:34:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.476+0000 7f1d837fe700 1 -- 172.21.15.131:0/3713193270 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c005a60 con 0x7f1d94102f60 2024-06-06T03:34:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.476+0000 7f1d9898e700 1 -- 172.21.15.131:0/3713193270 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d941a0b70 con 0x7f1d94102f60 2024-06-06T03:34:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.477+0000 7f1d9898e700 1 -- 172.21.15.131:0/3713193270 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d94069180 con 0x7f1d94102f60 2024-06-06T03:34:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.478+0000 7f1d837fe700 1 -- 172.21.15.131:0/3713193270 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1d8c01a020 con 0x7f1d94102f60 2024-06-06T03:34:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.479+0000 7f1d837fe700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1d7c0a2c20 0x7f1d7c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.479+0000 7f1d837fe700 1 -- 172.21.15.131:0/3713193270 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1d8c0ce3b0 con 0x7f1d94102f60 2024-06-06T03:34:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.479+0000 7f1d9259c700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1d7c0a2c20 0x7f1d7c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.480+0000 7f1d9259c700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1d7c0a2c20 0x7f1d7c0a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1d94102420 tx=0x7f1d84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:19.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.483+0000 7f1d837fe700 1 -- 172.21.15.131:0/3713193270 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d8c010070 con 0x7f1d94102f60 2024-06-06T03:34:19.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.690+0000 7f1d9898e700 1 -- 172.21.15.131:0/3713193270 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l", "target": ["mon-mgr", ""]}) -- 0x7f1d940feb90 con 0x7f1d7c0a2c20 2024-06-06T03:34:19.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.698+0000 7f1d837fe700 1 -- 172.21.15.131:0/3713193270 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f1d940feb90 con 0x7f1d7c0a2c20 2024-06-06T03:34:19.698 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled mds.all update... 2024-06-06T03:34:19.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.701+0000 7f1d9898e700 1 -- 172.21.15.131:0/3713193270 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1d7c0a2c20 msgr2=0x7f1d7c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:19.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.701+0000 7f1d9898e700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1d7c0a2c20 0x7f1d7c0a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1d94102420 tx=0x7f1d84005f90 comp rx=0 tx=0).stop 2024-06-06T03:34:19.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.701+0000 7f1d9898e700 1 -- 172.21.15.131:0/3713193270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d94102f60 msgr2=0x7f1d9419fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:19.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.701+0000 7f1d9898e700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d94102f60 0x7f1d9419fcf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c007f10 tx=0x7f1d8c00d3b0 comp rx=0 tx=0).stop 2024-06-06T03:34:19.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.701+0000 7f1d9898e700 1 -- 172.21.15.131:0/3713193270 shutdown_connections 2024-06-06T03:34:19.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.701+0000 7f1d9898e700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1d7c0a2c20 0x7f1d7c0a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:19.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.701+0000 7f1d9898e700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d94102f60 0x7f1d9419fcf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:19.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.701+0000 7f1d9898e700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d941013c0 0x7f1d94198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:19.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.701+0000 7f1d9898e700 1 --2- 172.21.15.131:0/3713193270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d941025c0 0x7f1d94198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:19.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.701+0000 7f1d9898e700 1 -- 172.21.15.131:0/3713193270 >> 172.21.15.131:0/3713193270 conn(0x7f1d940fc960 msgr2=0x7f1d940fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:19.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.701+0000 7f1d9898e700 1 -- 172.21.15.131:0/3713193270 shutdown_connections 2024-06-06T03:34:19.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-06-06T03:34:19.701+0000 7f1d9898e700 1 -- 172.21.15.131:0/3713193270 wait complete. 2024-06-06T03:34:20.404 DEBUG:teuthology.orchestra.run.smithi040:mds.a> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.a.service 2024-06-06T03:34:20.407 DEBUG:teuthology.orchestra.run.smithi040:mds.d> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.d.service 2024-06-06T03:34:20.451 DEBUG:teuthology.orchestra.run.smithi040:mds.g> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.g.service 2024-06-06T03:34:20.493 DEBUG:teuthology.orchestra.run.smithi040:mds.j> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.j.service 2024-06-06T03:34:20.536 DEBUG:teuthology.orchestra.run.smithi104:mds.b> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.b.service 2024-06-06T03:34:20.539 DEBUG:teuthology.orchestra.run.smithi104:mds.e> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.e.service 2024-06-06T03:34:20.582 DEBUG:teuthology.orchestra.run.smithi104:mds.h> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.h.service 2024-06-06T03:34:20.626 DEBUG:teuthology.orchestra.run.smithi104:mds.k> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.k.service 2024-06-06T03:34:20.669 DEBUG:teuthology.orchestra.run.smithi131:mds.c> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.c.service 2024-06-06T03:34:20.672 DEBUG:teuthology.orchestra.run.smithi131:mds.f> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.f.service 2024-06-06T03:34:20.715 DEBUG:teuthology.orchestra.run.smithi131:mds.i> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.i.service 2024-06-06T03:34:20.758 DEBUG:teuthology.orchestra.run.smithi131:mds.l> sudo journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.l.service 2024-06-06T03:34:20.802 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-06T03:34:20.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:34:20.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:20 smithi104 ceph-mon[36569]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:20.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:20 smithi104 ceph-mon[36569]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:34:20.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:20 smithi104 ceph-mon[36569]: Saving service mds.all spec with placement smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l;count:12 2024-06-06T03:34:20.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:20.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:34:20.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:20.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:34:20.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:20.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:20.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:20.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:20 smithi131 ceph-mon[36687]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:20 smithi131 ceph-mon[36687]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:34:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:20 smithi131 ceph-mon[36687]: Saving service mds.all spec with placement smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l;count:12 2024-06-06T03:34:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:34:21.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:21.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:34:21.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:21.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:21.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:21.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:21.052 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:34:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:20 smithi040 ceph-mon[32220]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:20 smithi040 ceph-mon[32220]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:34:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:20 smithi040 ceph-mon[32220]: Saving service mds.all spec with placement smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l;count:12 2024-06-06T03:34:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:34:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:34:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:21.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:21.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:21 smithi131 ceph-mon[36687]: Deploying daemon mds.l on smithi131 2024-06-06T03:34:21.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:21 smithi040 ceph-mon[32220]: Deploying daemon mds.l on smithi131 2024-06-06T03:34:22.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:21 smithi104 ceph-mon[36569]: Deploying daemon mds.l on smithi131 2024-06-06T03:34:22.703 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:34:22 smithi131 systemd[1]: Starting Ceph mds.l for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:34:22.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.818+0000 7f53cdd70700 1 -- 172.21.15.40:0/2034364006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8101d50 msgr2=0x7f53c810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:22.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.818+0000 7f53cdd70700 1 --2- 172.21.15.40:0/2034364006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8101d50 0x7f53c810e240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f53c000b0a0 tx=0x7f53c000b3b0 comp rx=0 tx=0).stop 2024-06-06T03:34:22.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.818+0000 7f53cdd70700 1 -- 172.21.15.40:0/2034364006 shutdown_connections 2024-06-06T03:34:22.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.818+0000 7f53cdd70700 1 --2- 172.21.15.40:0/2034364006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8101d50 0x7f53c810e240 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:22.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.818+0000 7f53cdd70700 1 --2- 172.21.15.40:0/2034364006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53c8107400 0x7f53c81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:22.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.818+0000 7f53cdd70700 1 --2- 172.21.15.40:0/2034364006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53c8101390 0x7f53c8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:22.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.818+0000 7f53cdd70700 1 -- 172.21.15.40:0/2034364006 >> 172.21.15.40:0/2034364006 conn(0x7f53c8078340 msgr2=0x7f53c8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:22.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.818+0000 7f53cdd70700 1 -- 172.21.15.40:0/2034364006 shutdown_connections 2024-06-06T03:34:22.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.818+0000 7f53cdd70700 1 -- 172.21.15.40:0/2034364006 wait complete. 2024-06-06T03:34:22.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.819+0000 7f53cdd70700 1 Processor -- start 2024-06-06T03:34:22.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.819+0000 7f53cdd70700 1 -- start start 2024-06-06T03:34:22.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.819+0000 7f53cdd70700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53c8101390 0x7f53c819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:22.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.819+0000 7f53cdd70700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53c8101d50 0x7f53c819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:22.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.819+0000 7f53cdd70700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8107400 0x7f53c8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:22.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.819+0000 7f53cdd70700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f53c8111670 con 0x7f53c8107400 2024-06-06T03:34:22.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.819+0000 7f53cdd70700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81114f0 con 0x7f53c8101390 2024-06-06T03:34:22.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.819+0000 7f53cdd70700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81117f0 con 0x7f53c8101d50 2024-06-06T03:34:22.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.819+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8107400 0x7f53c8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:22.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.819+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8107400 0x7f53c8198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38336/0 (socket says 172.21.15.40:38336) 2024-06-06T03:34:22.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.819+0000 7f53c7fff700 1 -- 172.21.15.40:0/1464533556 learned_addr learned my addr 172.21.15.40:0/1464533556 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:34:22.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.820+0000 7f53c77fe700 1 --2- 172.21.15.40:0/1464533556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53c8101390 0x7f53c819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:22.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.820+0000 7f53c7fff700 1 -- 172.21.15.40:0/1464533556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53c8101d50 msgr2=0x7f53c819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:22.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.820+0000 7f53c7fff700 1 --2- 172.21.15.40:0/1464533556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53c8101d50 0x7f53c819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:22.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.820+0000 7f53c7fff700 1 -- 172.21.15.40:0/1464533556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53c8101390 msgr2=0x7f53c819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:22.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.820+0000 7f53c7fff700 1 --2- 172.21.15.40:0/1464533556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53c8101390 0x7f53c819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:22.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.820+0000 7f53c7fff700 1 -- 172.21.15.40:0/1464533556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c81990a0 con 0x7f53c8107400 2024-06-06T03:34:22.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.820+0000 7f53c77fe700 1 --2- 172.21.15.40:0/1464533556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53c8101390 0x7f53c819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:34:22.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.820+0000 7f53c7fff700 1 --2- 172.21.15.40:0/1464533556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8107400 0x7f53c8198940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f53c00062a0 tx=0x7f53c000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:22.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.820+0000 7f53c4ff9700 1 -- 172.21.15.40:0/1464533556 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0021070 con 0x7f53c8107400 2024-06-06T03:34:22.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.820+0000 7f53cdd70700 1 -- 172.21.15.40:0/1464533556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c8199330 con 0x7f53c8107400 2024-06-06T03:34:22.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.820+0000 7f53c4ff9700 1 -- 172.21.15.40:0/1464533556 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53c0007db0 con 0x7f53c8107400 2024-06-06T03:34:22.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.820+0000 7f53c4ff9700 1 -- 172.21.15.40:0/1464533556 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0005460 con 0x7f53c8107400 2024-06-06T03:34:22.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.820+0000 7f53cdd70700 1 -- 172.21.15.40:0/1464533556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c81a5650 con 0x7f53c8107400 2024-06-06T03:34:22.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.821+0000 7f53cdd70700 1 -- 172.21.15.40:0/1464533556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c8069180 con 0x7f53c8107400 2024-06-06T03:34:22.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.822+0000 7f53c4ff9700 1 -- 172.21.15.40:0/1464533556 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f53c0024070 con 0x7f53c8107400 2024-06-06T03:34:22.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.823+0000 7f53c4ff9700 1 --2- 172.21.15.40:0/1464533556 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f53b00a2c20 0x7f53b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:22.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.823+0000 7f53c77fe700 1 --2- 172.21.15.40:0/1464533556 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f53b00a2c20 0x7f53b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:22.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.824+0000 7f53c4ff9700 1 -- 172.21.15.40:0/1464533556 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f53c00cfea0 con 0x7f53c8107400 2024-06-06T03:34:22.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.824+0000 7f53c77fe700 1 --2- 172.21.15.40:0/1464533556 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f53b00a2c20 0x7f53b00a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005fd0 tx=0x7f53b8005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:22.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:22.827+0000 7f53c4ff9700 1 -- 172.21.15.40:0/1464533556 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f53c009d2d0 con 0x7f53c8107400 2024-06-06T03:34:22.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:22 smithi040 ceph-mon[32220]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:23.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:22 smithi131 ceph-mon[36687]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:23.032 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:34:22 smithi131 podman[67190]: 2024-06-06 03:34:22.728687721 +0000 UTC m=+0.016751665 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:34:23.032 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:34:22 smithi131 podman[67190]: 2024-06-06 03:34:22.83932592 +0000 UTC m=+0.127389858 container create 220c34dc6a944942aa87d736ae4a04ca12980ed047019063b32b203a56849078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-l, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:34:23.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:23.081+0000 7f53cdd70700 1 -- 172.21.15.40:0/1464533556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53c81a5820 con 0x7f53c8107400 2024-06-06T03:34:23.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:23.082+0000 7f53c4ff9700 1 -- 172.21.15.40:0/1464533556 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f53c009cbd0 con 0x7f53c8107400 2024-06-06T03:34:23.082 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:34:23.082 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-06-06T03:34:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:23.084+0000 7f53cdd70700 1 -- 172.21.15.40:0/1464533556 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f53b00a2c20 msgr2=0x7f53b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:23.084+0000 7f53cdd70700 1 --2- 172.21.15.40:0/1464533556 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f53b00a2c20 0x7f53b00a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005fd0 tx=0x7f53b8005d60 comp rx=0 tx=0).stop 2024-06-06T03:34:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:23.084+0000 7f53cdd70700 1 -- 172.21.15.40:0/1464533556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8107400 msgr2=0x7f53c8198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:23.084+0000 7f53cdd70700 1 --2- 172.21.15.40:0/1464533556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8107400 0x7f53c8198940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f53c00062a0 tx=0x7f53c000bf10 comp rx=0 tx=0).stop 2024-06-06T03:34:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:23.084+0000 7f53cdd70700 1 -- 172.21.15.40:0/1464533556 shutdown_connections 2024-06-06T03:34:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:23.084+0000 7f53cdd70700 1 --2- 172.21.15.40:0/1464533556 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f53b00a2c20 0x7f53b00a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:23.084+0000 7f53cdd70700 1 --2- 172.21.15.40:0/1464533556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8107400 0x7f53c8198940 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:23.084+0000 7f53cdd70700 1 --2- 172.21.15.40:0/1464533556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53c8101390 0x7f53c819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:23.084+0000 7f53cdd70700 1 --2- 172.21.15.40:0/1464533556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53c8101d50 0x7f53c819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:23.084+0000 7f53cdd70700 1 -- 172.21.15.40:0/1464533556 >> 172.21.15.40:0/1464533556 conn(0x7f53c8078340 msgr2=0x7f53c8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:23.084+0000 7f53cdd70700 1 -- 172.21.15.40:0/1464533556 shutdown_connections 2024-06-06T03:34:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:23.084+0000 7f53cdd70700 1 -- 172.21.15.40:0/1464533556 wait complete. 2024-06-06T03:34:23.085 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 1 2024-06-06T03:34:23.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:22 smithi104 ceph-mon[36569]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:23.532 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:34:23 smithi131 podman[67190]: 2024-06-06 03:34:23.209255529 +0000 UTC m=+0.497319456 container init 220c34dc6a944942aa87d736ae4a04ca12980ed047019063b32b203a56849078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T03:34:23.532 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:34:23 smithi131 podman[67190]: 2024-06-06 03:34:23.212105708 +0000 UTC m=+0.500169658 container start 220c34dc6a944942aa87d736ae4a04ca12980ed047019063b32b203a56849078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-l, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-06-06T03:34:23.532 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:34:23 smithi131 ceph-mds[67209]: -- 172.21.15.131:0/801267438 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f065ef5a40 con 0x55f065ec8c00 2024-06-06T03:34:23.532 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:34:23 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-l[67205]: starting mds.l at 2024-06-06T03:34:23.532 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:34:23 smithi131 bash[67190]: 220c34dc6a944942aa87d736ae4a04ca12980ed047019063b32b203a56849078 2024-06-06T03:34:23.532 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:34:23 smithi131 systemd[1]: Started Ceph mds.l for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:34:23.992 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:23 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1464533556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:23.992 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:23 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.131:6832/1833802668,v1:172.21.15.131:6833/1833802668] up:boot 2024-06-06T03:34:23.992 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:23 smithi104 ceph-mon[36569]: fsmap 1 up:standby 2024-06-06T03:34:23.992 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T03:34:23.992 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:23.992 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:23.992 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:23.992 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:23.992 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:23.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:23 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:23.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:23 smithi104 ceph-mon[36569]: Deploying daemon mds.k on smithi104 2024-06-06T03:34:24.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:23 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1464533556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:24.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:23 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.131:6832/1833802668,v1:172.21.15.131:6833/1833802668] up:boot 2024-06-06T03:34:24.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:23 smithi131 ceph-mon[36687]: fsmap 1 up:standby 2024-06-06T03:34:24.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T03:34:24.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:24.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:24.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:24.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:24.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:24.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:23 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:24.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:23 smithi131 ceph-mon[36687]: Deploying daemon mds.k on smithi104 2024-06-06T03:34:24.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:23 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1464533556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:24.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:23 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.131:6832/1833802668,v1:172.21.15.131:6833/1833802668] up:boot 2024-06-06T03:34:24.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:23 smithi040 ceph-mon[32220]: fsmap 1 up:standby 2024-06-06T03:34:24.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T03:34:24.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:24.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:24.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:24.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:24.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:24.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:23 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:24.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:23 smithi040 ceph-mon[32220]: Deploying daemon mds.k on smithi104 2024-06-06T03:34:25.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:25 smithi104 ceph-mon[36569]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:25.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:25 smithi131 ceph-mon[36687]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:25.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:34:25.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:25 smithi040 ceph-mon[32220]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:26.077 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:34:26.340 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jun 06 03:34:26 smithi104 systemd[1]: Started Ceph mds.k for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:34:27.494 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:27 smithi040 ceph-mon[32220]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:27.494 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:27 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:27.494 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:27 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:27.495 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:27 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:27.495 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:27 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:27.495 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:27 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:27.495 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:27 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:27.495 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:27 smithi040 ceph-mon[32220]: Deploying daemon mds.j on smithi040 2024-06-06T03:34:27.495 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:27 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] up:boot 2024-06-06T03:34:27.496 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:27 smithi040 ceph-mon[32220]: fsmap 2 up:standby 2024-06-06T03:34:27.496 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:27 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T03:34:27.579 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:27 smithi104 ceph-mon[36569]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:27.579 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:27 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:27.579 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:27 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:27.579 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:27 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:27.579 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:27 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:27.579 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:27 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:27.579 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:27 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:27.580 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:27 smithi104 ceph-mon[36569]: Deploying daemon mds.j on smithi040 2024-06-06T03:34:27.580 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:27 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] up:boot 2024-06-06T03:34:27.580 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:27 smithi104 ceph-mon[36569]: fsmap 2 up:standby 2024-06-06T03:34:27.580 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:27 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T03:34:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:27 smithi131 ceph-mon[36687]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:27 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:27 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:27 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:27 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:27 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:27 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:27 smithi131 ceph-mon[36687]: Deploying daemon mds.j on smithi040 2024-06-06T03:34:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:27 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] up:boot 2024-06-06T03:34:27.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:27 smithi131 ceph-mon[36687]: fsmap 2 up:standby 2024-06-06T03:34:27.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:27 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T03:34:27.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.929+0000 7f307f91d700 1 -- 172.21.15.40:0/14346069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3078107400 msgr2=0x7f30781077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:27.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.929+0000 7f307f91d700 1 --2- 172.21.15.40:0/14346069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3078107400 0x7f30781077e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3074009990 tx=0x7f3074009ca0 comp rx=0 tx=0).stop 2024-06-06T03:34:27.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.929+0000 7f307f91d700 1 -- 172.21.15.40:0/14346069 shutdown_connections 2024-06-06T03:34:27.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.929+0000 7f307f91d700 1 --2- 172.21.15.40:0/14346069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3078101d50 0x7f307810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:27.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.929+0000 7f307f91d700 1 --2- 172.21.15.40:0/14346069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3078107400 0x7f30781077e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:27.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.929+0000 7f307f91d700 1 --2- 172.21.15.40:0/14346069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3078101390 0x7f3078101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:27.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.929+0000 7f307f91d700 1 -- 172.21.15.40:0/14346069 >> 172.21.15.40:0/14346069 conn(0x7f3078078340 msgr2=0x7f3078078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:27.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.930+0000 7f307f91d700 1 -- 172.21.15.40:0/14346069 shutdown_connections 2024-06-06T03:34:27.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.930+0000 7f307f91d700 1 -- 172.21.15.40:0/14346069 wait complete. 2024-06-06T03:34:27.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.930+0000 7f307f91d700 1 Processor -- start 2024-06-06T03:34:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.930+0000 7f307f91d700 1 -- start start 2024-06-06T03:34:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307f91d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3078101390 0x7f3078199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307f91d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3078101d50 0x7f307819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307f91d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3078107400 0x7f307819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307f91d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f30781116f0 con 0x7f3078107400 2024-06-06T03:34:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307f91d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3078111570 con 0x7f3078101d50 2024-06-06T03:34:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307f91d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3078111870 con 0x7f3078101390 2024-06-06T03:34:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307deba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3078107400 0x7f307819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307deba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3078107400 0x7f307819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38352/0 (socket says 172.21.15.40:38352) 2024-06-06T03:34:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307deba700 1 -- 172.21.15.40:0/3291929143 learned_addr learned my addr 172.21.15.40:0/3291929143 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:34:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307ceb8700 1 --2- 172.21.15.40:0/3291929143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3078101d50 0x7f307819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307d6b9700 1 --2- 172.21.15.40:0/3291929143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3078101390 0x7f3078199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307deba700 1 -- 172.21.15.40:0/3291929143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3078101390 msgr2=0x7f3078199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:27.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307deba700 1 --2- 172.21.15.40:0/3291929143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3078101390 0x7f3078199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:27.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307deba700 1 -- 172.21.15.40:0/3291929143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3078101d50 msgr2=0x7f307819a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:27.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307deba700 1 --2- 172.21.15.40:0/3291929143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3078101d50 0x7f307819a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:27.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307deba700 1 -- 172.21.15.40:0/3291929143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30781a4ff0 con 0x7f3078107400 2024-06-06T03:34:27.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.931+0000 7f307deba700 1 --2- 172.21.15.40:0/3291929143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3078107400 0x7f307819a670 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f307000b970 tx=0x7f307000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:27.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.932+0000 7f306e7fc700 1 -- 172.21.15.40:0/3291929143 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307000d860 con 0x7f3078107400 2024-06-06T03:34:27.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.932+0000 7f306e7fc700 1 -- 172.21.15.40:0/3291929143 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f307000e430 con 0x7f3078107400 2024-06-06T03:34:27.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.932+0000 7f307f91d700 1 -- 172.21.15.40:0/3291929143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30781a52e0 con 0x7f3078107400 2024-06-06T03:34:27.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.932+0000 7f306e7fc700 1 -- 172.21.15.40:0/3291929143 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070017660 con 0x7f3078107400 2024-06-06T03:34:27.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.932+0000 7f307f91d700 1 -- 172.21.15.40:0/3291929143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30781a5820 con 0x7f3078107400 2024-06-06T03:34:27.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.933+0000 7f307f91d700 1 -- 172.21.15.40:0/3291929143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3078069180 con 0x7f3078107400 2024-06-06T03:34:27.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.934+0000 7f306e7fc700 1 -- 172.21.15.40:0/3291929143 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f307000e5e0 con 0x7f3078107400 2024-06-06T03:34:27.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.935+0000 7f306e7fc700 1 --2- 172.21.15.40:0/3291929143 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f30640a2ca0 0x7f30640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:27.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.935+0000 7f307d6b9700 1 --2- 172.21.15.40:0/3291929143 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f30640a2ca0 0x7f30640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:27.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.935+0000 7f306e7fc700 1 -- 172.21.15.40:0/3291929143 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f30700ce6d0 con 0x7f3078107400 2024-06-06T03:34:27.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.935+0000 7f307d6b9700 1 --2- 172.21.15.40:0/3291929143 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f30640a2ca0 0x7f30640a5160 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f307400b5c0 tx=0x7f3074005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:27.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:27.939+0000 7f306e7fc700 1 -- 172.21.15.40:0/3291929143 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3070013030 con 0x7f3078107400 2024-06-06T03:34:28.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:28.192+0000 7f307f91d700 1 -- 172.21.15.40:0/3291929143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f307806a730 con 0x7f3078107400 2024-06-06T03:34:28.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:28.192+0000 7f306e7fc700 1 -- 172.21.15.40:0/3291929143 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f307009bad0 con 0x7f3078107400 2024-06-06T03:34:28.193 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:34:28.193 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24358,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-06T03:34:28.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:28.195+0000 7f307f91d700 1 -- 172.21.15.40:0/3291929143 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f30640a2ca0 msgr2=0x7f30640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:28.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:28.195+0000 7f307f91d700 1 --2- 172.21.15.40:0/3291929143 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f30640a2ca0 0x7f30640a5160 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f307400b5c0 tx=0x7f3074005960 comp rx=0 tx=0).stop 2024-06-06T03:34:28.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:28.195+0000 7f307f91d700 1 -- 172.21.15.40:0/3291929143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3078107400 msgr2=0x7f307819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:28.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:28.195+0000 7f307f91d700 1 --2- 172.21.15.40:0/3291929143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3078107400 0x7f307819a670 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f307000b970 tx=0x7f307000bc80 comp rx=0 tx=0).stop 2024-06-06T03:34:28.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:28.195+0000 7f307f91d700 1 -- 172.21.15.40:0/3291929143 shutdown_connections 2024-06-06T03:34:28.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:28.195+0000 7f307f91d700 1 --2- 172.21.15.40:0/3291929143 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f30640a2ca0 0x7f30640a5160 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:28.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:28.195+0000 7f307f91d700 1 --2- 172.21.15.40:0/3291929143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3078101390 0x7f3078199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:28.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:28.195+0000 7f307f91d700 1 --2- 172.21.15.40:0/3291929143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3078107400 0x7f307819a670 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:28.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:28.195+0000 7f307f91d700 1 --2- 172.21.15.40:0/3291929143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3078101d50 0x7f307819a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:28.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:28.195+0000 7f307f91d700 1 -- 172.21.15.40:0/3291929143 >> 172.21.15.40:0/3291929143 conn(0x7f3078078340 msgr2=0x7f30780fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:28.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:28.195+0000 7f307f91d700 1 -- 172.21.15.40:0/3291929143 shutdown_connections 2024-06-06T03:34:28.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:28.195+0000 7f307f91d700 1 -- 172.21.15.40:0/3291929143 wait complete. 2024-06-06T03:34:28.196 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 3 2024-06-06T03:34:28.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:28 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3291929143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:28.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:28 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3291929143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:28.781 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:28 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3291929143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:29.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:29 smithi040 ceph-mon[32220]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:29.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:29 smithi104 ceph-mon[36569]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:29.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:29 smithi131 ceph-mon[36687]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:30.330 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:34:30 smithi040 systemd[1]: Starting Ceph mds.j for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:34:30.682 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:34:30 smithi040 podman[73042]: 2024-06-06 03:34:30.308688325 +0000 UTC m=+0.017791867 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:34:30.682 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:34:30 smithi040 podman[73042]: 2024-06-06 03:34:30.454669342 +0000 UTC m=+0.163772876 container create d0b64280c765686e90c59401ab1cc5271ef9bc0e48bab1a7086f17afa48d8923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-j, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-06T03:34:30.683 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:34:30 smithi040 podman[73042]: 2024-06-06 03:34:30.683908957 +0000 UTC m=+0.393012484 container init d0b64280c765686e90c59401ab1cc5271ef9bc0e48bab1a7086f17afa48d8923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T03:34:31.080 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:34:30 smithi040 podman[73042]: 2024-06-06 03:34:30.686850313 +0000 UTC m=+0.395953848 container start d0b64280c765686e90c59401ab1cc5271ef9bc0e48bab1a7086f17afa48d8923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-j, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-06T03:34:31.080 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:34:30 smithi040 ceph-mds[73061]: -- 172.21.15.40:0/3483619008 <== mon.0 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c546701a40 con 0x55c5466d5000 2024-06-06T03:34:31.080 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:34:30 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-j[73057]: starting mds.j at 2024-06-06T03:34:31.080 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:34:30 smithi040 bash[73042]: d0b64280c765686e90c59401ab1cc5271ef9bc0e48bab1a7086f17afa48d8923 2024-06-06T03:34:31.080 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:34:30 smithi040 systemd[1]: Started Ceph mds.j for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:34:31.293 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:34:31.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:31 smithi131 ceph-mon[36687]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:31.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:31 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:31.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:31 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:31.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:31 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:31.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:31 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:31.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:31 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:31.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:31 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:31.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:31 smithi040 ceph-mon[32220]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:31.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:31 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:31.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:31 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:31.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:31 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:31.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:31 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:31.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:31 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:31.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:31 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:31.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:31 smithi104 ceph-mon[36569]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:31.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:31 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:31.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:31 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:31.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:31 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:31.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:31 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:31.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:31 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:31.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:31 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:32.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:32 smithi104 ceph-mon[36569]: Deploying daemon mds.d on smithi040 2024-06-06T03:34:32.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:32 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.40:6834/1980325130,v1:172.21.15.40:6835/1980325130] up:boot 2024-06-06T03:34:32.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:32 smithi104 ceph-mon[36569]: fsmap 3 up:standby 2024-06-06T03:34:32.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:32 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T03:34:32.691 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:32 smithi040 ceph-mon[32220]: Deploying daemon mds.d on smithi040 2024-06-06T03:34:32.691 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:32 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.40:6834/1980325130,v1:172.21.15.40:6835/1980325130] up:boot 2024-06-06T03:34:32.692 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:32 smithi040 ceph-mon[32220]: fsmap 3 up:standby 2024-06-06T03:34:32.692 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:32 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T03:34:32.716 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:34:32.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:32 smithi131 ceph-mon[36687]: Deploying daemon mds.d on smithi040 2024-06-06T03:34:32.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:32 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.40:6834/1980325130,v1:172.21.15.40:6835/1980325130] up:boot 2024-06-06T03:34:32.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:32 smithi131 ceph-mon[36687]: fsmap 3 up:standby 2024-06-06T03:34:32.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:32 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T03:34:33.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:33 smithi040 ceph-mon[32220]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:33.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:33 smithi104 ceph-mon[36569]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:33.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:33 smithi131 ceph-mon[36687]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:34.330 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jun 06 03:34:34 smithi040 systemd[1]: Started Ceph mds.d for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:34:34.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.983+0000 7f0c58378700 1 -- 172.21.15.40:0/556025565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c5006b620 msgr2=0x7f0c5010f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:34.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.983+0000 7f0c58378700 1 --2- 172.21.15.40:0/556025565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c5006b620 0x7f0c5010f350 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800b0d0 tx=0x7f0c4800b3e0 comp rx=0 tx=0).stop 2024-06-06T03:34:34.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.983+0000 7f0c58378700 1 -- 172.21.15.40:0/556025565 shutdown_connections 2024-06-06T03:34:34.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.983+0000 7f0c58378700 1 --2- 172.21.15.40:0/556025565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c5006b620 0x7f0c5010f350 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:34.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.983+0000 7f0c58378700 1 --2- 172.21.15.40:0/556025565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c50108580 0x7f0c50108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:34.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.983+0000 7f0c58378700 1 --2- 172.21.15.40:0/556025565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c5006ac60 0x7f0c5006b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:34.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.983+0000 7f0c58378700 1 -- 172.21.15.40:0/556025565 >> 172.21.15.40:0/556025565 conn(0x7f0c50078430 msgr2=0x7f0c50078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:34.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.983+0000 7f0c58378700 1 -- 172.21.15.40:0/556025565 shutdown_connections 2024-06-06T03:34:34.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.983+0000 7f0c58378700 1 -- 172.21.15.40:0/556025565 wait complete. 2024-06-06T03:34:34.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c58378700 1 Processor -- start 2024-06-06T03:34:34.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c58378700 1 -- start start 2024-06-06T03:34:34.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c58378700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c5006ac60 0x7f0c5019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c58378700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c5006b620 0x7f0c5019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c58378700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c50108580 0x7f0c501989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c58378700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501127c0 con 0x7f0c5006b620 2024-06-06T03:34:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c58378700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0c50112640 con 0x7f0c5006ac60 2024-06-06T03:34:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c58378700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0c50112940 con 0x7f0c50108580 2024-06-06T03:34:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c55913700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c5006b620 0x7f0c5019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c55913700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c5006b620 0x7f0c5019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46660/0 (socket says 172.21.15.40:46660) 2024-06-06T03:34:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c55913700 1 -- 172.21.15.40:0/3986632792 learned_addr learned my addr 172.21.15.40:0/3986632792 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:34:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c56114700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c5006ac60 0x7f0c5019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c56915700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c50108580 0x7f0c501989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c55913700 1 -- 172.21.15.40:0/3986632792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c50108580 msgr2=0x7f0c501989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c55913700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c50108580 0x7f0c501989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c55913700 1 -- 172.21.15.40:0/3986632792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c5006ac60 msgr2=0x7f0c5019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:34.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c55913700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c5006ac60 0x7f0c5019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:34.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c55913700 1 -- 172.21.15.40:0/3986632792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c50199100 con 0x7f0c5006b620 2024-06-06T03:34:34.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.984+0000 7f0c56114700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c5006ac60 0x7f0c5019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:34:34.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.985+0000 7f0c55913700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c5006b620 0x7f0c5019ed20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000c9b0 tx=0x7f0c4000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:34.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.985+0000 7f0c477fe700 1 -- 172.21.15.40:0/3986632792 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c40007b00 con 0x7f0c5006b620 2024-06-06T03:34:34.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.985+0000 7f0c477fe700 1 -- 172.21.15.40:0/3986632792 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c4000ce50 con 0x7f0c5006b620 2024-06-06T03:34:34.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.985+0000 7f0c58378700 1 -- 172.21.15.40:0/3986632792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c501993f0 con 0x7f0c5006b620 2024-06-06T03:34:34.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.985+0000 7f0c477fe700 1 -- 172.21.15.40:0/3986632792 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c40018660 con 0x7f0c5006b620 2024-06-06T03:34:34.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.985+0000 7f0c58378700 1 -- 172.21.15.40:0/3986632792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c501a56b0 con 0x7f0c5006b620 2024-06-06T03:34:34.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.986+0000 7f0c58378700 1 -- 172.21.15.40:0/3986632792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c50069180 con 0x7f0c5006b620 2024-06-06T03:34:34.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.987+0000 7f0c477fe700 1 -- 172.21.15.40:0/3986632792 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0c4000f450 con 0x7f0c5006b620 2024-06-06T03:34:34.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.988+0000 7f0c477fe700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0c3c0a2c20 0x7f0c3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:34.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.988+0000 7f0c56114700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0c3c0a2c20 0x7f0c3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:34.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.988+0000 7f0c477fe700 1 -- 172.21.15.40:0/3986632792 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0c400ced50 con 0x7f0c5006b620 2024-06-06T03:34:34.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.988+0000 7f0c56114700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0c3c0a2c20 0x7f0c3c0a50e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0c50199ae0 tx=0x7f0c4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:34.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:34.991+0000 7f0c477fe700 1 -- 172.21.15.40:0/3986632792 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0c40014030 con 0x7f0c5006b620 2024-06-06T03:34:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:35.248+0000 7f0c58378700 1 -- 172.21.15.40:0/3986632792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c5002dca0 con 0x7f0c5006b620 2024-06-06T03:34:35.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:35.249+0000 7f0c477fe700 1 -- 172.21.15.40:0/3986632792 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7f0c4009c1e0 con 0x7f0c5006b620 2024-06-06T03:34:35.249 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:34:35.249 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24358,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-06T03:34:35.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:35.251+0000 7f0c58378700 1 -- 172.21.15.40:0/3986632792 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0c3c0a2c20 msgr2=0x7f0c3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:35.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:35.251+0000 7f0c58378700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0c3c0a2c20 0x7f0c3c0a50e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0c50199ae0 tx=0x7f0c4c009260 comp rx=0 tx=0).stop 2024-06-06T03:34:35.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:35.251+0000 7f0c58378700 1 -- 172.21.15.40:0/3986632792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c5006b620 msgr2=0x7f0c5019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:35.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:35.251+0000 7f0c58378700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c5006b620 0x7f0c5019ed20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000c9b0 tx=0x7f0c4000ccc0 comp rx=0 tx=0).stop 2024-06-06T03:34:35.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:35.251+0000 7f0c58378700 1 -- 172.21.15.40:0/3986632792 shutdown_connections 2024-06-06T03:34:35.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:35.251+0000 7f0c58378700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0c3c0a2c20 0x7f0c3c0a50e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:35.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:35.251+0000 7f0c58378700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c50108580 0x7f0c501989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:35.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:35.251+0000 7f0c58378700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c5006ac60 0x7f0c5019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:35.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:35.251+0000 7f0c58378700 1 --2- 172.21.15.40:0/3986632792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c5006b620 0x7f0c5019ed20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:35.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:35.251+0000 7f0c58378700 1 -- 172.21.15.40:0/3986632792 >> 172.21.15.40:0/3986632792 conn(0x7f0c50078430 msgr2=0x7f0c500ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:35.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:35.251+0000 7f0c58378700 1 -- 172.21.15.40:0/3986632792 shutdown_connections 2024-06-06T03:34:35.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:35.251+0000 7f0c58378700 1 -- 172.21.15.40:0/3986632792 wait complete. 2024-06-06T03:34:35.252 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 5 2024-06-06T03:34:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:35 smithi131 ceph-mon[36687]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:35 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:35 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:35 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:35 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:35 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:35 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:35 smithi131 ceph-mon[36687]: Deploying daemon mds.e on smithi104 2024-06-06T03:34:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:35 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.40:6836/1373355997,v1:172.21.15.40:6837/1373355997] up:boot 2024-06-06T03:34:35.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:35 smithi131 ceph-mon[36687]: fsmap 4 up:standby 2024-06-06T03:34:35.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:35 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T03:34:35.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:35 smithi040 ceph-mon[32220]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:35.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:35 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:35.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:35 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:35.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:35 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:35.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:35 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:35.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:35 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:35.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:35 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:35.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:35 smithi040 ceph-mon[32220]: Deploying daemon mds.e on smithi104 2024-06-06T03:34:35.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:35 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.40:6836/1373355997,v1:172.21.15.40:6837/1373355997] up:boot 2024-06-06T03:34:35.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:35 smithi040 ceph-mon[32220]: fsmap 4 up:standby 2024-06-06T03:34:35.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:35 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T03:34:35.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:35 smithi104 ceph-mon[36569]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:35.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:35 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:35.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:35 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:35.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:35 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:35.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:35 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:35.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:35 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:35.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:35 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:35.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:35 smithi104 ceph-mon[36569]: Deploying daemon mds.e on smithi104 2024-06-06T03:34:35.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:35 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.40:6836/1373355997,v1:172.21.15.40:6837/1373355997] up:boot 2024-06-06T03:34:35.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:35 smithi104 ceph-mon[36569]: fsmap 4 up:standby 2024-06-06T03:34:35.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:35 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T03:34:36.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:36 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3986632792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:36.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:36 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3986632792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:36.457 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:34:36 smithi104 systemd[1]: Starting Ceph mds.e for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:34:36.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:36 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3986632792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:36.721 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:34:36 smithi104 podman[66307]: 2024-06-06 03:34:36.574819936 +0000 UTC m=+0.017085221 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:34:37.032 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:34:36 smithi104 podman[66307]: 2024-06-06 03:34:36.722101699 +0000 UTC m=+0.164366974 container create 4643e9f7b44930925dc7e40db3b23699e12a8571eaa19f7a471fd497795cada2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-06T03:34:37.032 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:34:36 smithi104 podman[66307]: 2024-06-06 03:34:36.941501476 +0000 UTC m=+0.383766752 container init 4643e9f7b44930925dc7e40db3b23699e12a8571eaa19f7a471fd497795cada2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-e, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-06T03:34:37.032 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:34:36 smithi104 podman[66307]: 2024-06-06 03:34:36.94414552 +0000 UTC m=+0.386410804 container start 4643e9f7b44930925dc7e40db3b23699e12a8571eaa19f7a471fd497795cada2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-e, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc) 2024-06-06T03:34:37.033 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:34:36 smithi104 ceph-mds[66326]: -- 172.21.15.104:0/2121603576 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55575e7bba40 con 0x55575e78f000 2024-06-06T03:34:37.033 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:34:36 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-e[66322]: starting mds.e at 2024-06-06T03:34:37.340 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:34:37 smithi104 bash[66307]: 4643e9f7b44930925dc7e40db3b23699e12a8571eaa19f7a471fd497795cada2 2024-06-06T03:34:37.340 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:34:37 smithi104 systemd[1]: Started Ceph mds.e for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:34:37.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:37 smithi104 ceph-mon[36569]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:37.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:37 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:37.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:37 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:37.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:37 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:37.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:37 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:37.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:37 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:37.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:37 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:37.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:37 smithi131 ceph-mon[36687]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:37.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:37.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:37.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:37.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:37.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:37.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:37 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:37.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:37 smithi040 ceph-mon[32220]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:37.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:37.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:37.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:37.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:37.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:37.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:37 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:37.979 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:34:38.247 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:34:38.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:38 smithi040 ceph-mon[32220]: Deploying daemon mds.f on smithi131 2024-06-06T03:34:38.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:38 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] up:boot 2024-06-06T03:34:38.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:38 smithi040 ceph-mon[32220]: fsmap 5 up:standby 2024-06-06T03:34:38.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:38 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T03:34:38.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:38 smithi131 ceph-mon[36687]: Deploying daemon mds.f on smithi131 2024-06-06T03:34:38.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:38 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] up:boot 2024-06-06T03:34:38.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:38 smithi131 ceph-mon[36687]: fsmap 5 up:standby 2024-06-06T03:34:38.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:38 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T03:34:38.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:38 smithi104 ceph-mon[36569]: Deploying daemon mds.f on smithi131 2024-06-06T03:34:38.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:38 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] up:boot 2024-06-06T03:34:38.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:38 smithi104 ceph-mon[36569]: fsmap 5 up:standby 2024-06-06T03:34:38.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:38 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T03:34:39.457 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:39 smithi040 ceph-mon[32220]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:39.497 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:39 smithi131 ceph-mon[36687]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:39.498 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:34:39 smithi131 systemd[1]: Starting Ceph mds.f for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:34:39.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:39 smithi104 ceph-mon[36569]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:39.763 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:34:39 smithi131 podman[67500]: 2024-06-06 03:34:39.401237644 +0000 UTC m=+0.017373930 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:34:39.763 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:34:39 smithi131 podman[67500]: 2024-06-06 03:34:39.511784769 +0000 UTC m=+0.127921024 container create b3bcd351fd22303fe9458b6c2deb6a7b070e7302888b4de81bfe96b6e779ff18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-06T03:34:39.763 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:34:39 smithi131 podman[67500]: 2024-06-06 03:34:39.764658781 +0000 UTC m=+0.380795045 container init b3bcd351fd22303fe9458b6c2deb6a7b070e7302888b4de81bfe96b6e779ff18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-06-06T03:34:40.032 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:34:39 smithi131 podman[67500]: 2024-06-06 03:34:39.767461088 +0000 UTC m=+0.383597358 container start b3bcd351fd22303fe9458b6c2deb6a7b070e7302888b4de81bfe96b6e779ff18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-f, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-06T03:34:40.032 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:34:39 smithi131 ceph-mds[67520]: -- 172.21.15.131:0/304030356 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559788025a40 con 0x559787ff8c00 2024-06-06T03:34:40.032 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:34:39 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-f[67516]: starting mds.f at 2024-06-06T03:34:40.032 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:34:39 smithi131 bash[67500]: b3bcd351fd22303fe9458b6c2deb6a7b070e7302888b4de81bfe96b6e779ff18 2024-06-06T03:34:40.032 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:34:39 smithi131 systemd[1]: Started Ceph mds.f for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:34:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.050+0000 7f19750d6700 1 -- 172.21.15.40:0/1200961993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1970107660 msgr2=0x7f1970107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.050+0000 7f19750d6700 1 --2- 172.21.15.40:0/1200961993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1970107660 0x7f1970107a40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f19600099c0 tx=0x7f1960009cd0 comp rx=0 tx=0).stop 2024-06-06T03:34:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.050+0000 7f19750d6700 1 -- 172.21.15.40:0/1200961993 shutdown_connections 2024-06-06T03:34:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.050+0000 7f19750d6700 1 --2- 172.21.15.40:0/1200961993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1970102900 0x7f197010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.050+0000 7f19750d6700 1 --2- 172.21.15.40:0/1200961993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1970107660 0x7f1970107a40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.050+0000 7f19750d6700 1 --2- 172.21.15.40:0/1200961993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1970101f40 0x7f19701023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.051+0000 7f19750d6700 1 -- 172.21.15.40:0/1200961993 >> 172.21.15.40:0/1200961993 conn(0x7f19700766b0 msgr2=0x7f1970076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.051+0000 7f19750d6700 1 -- 172.21.15.40:0/1200961993 shutdown_connections 2024-06-06T03:34:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.051+0000 7f19750d6700 1 -- 172.21.15.40:0/1200961993 wait complete. 2024-06-06T03:34:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.051+0000 7f19750d6700 1 Processor -- start 2024-06-06T03:34:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f19750d6700 1 -- start start 2024-06-06T03:34:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f19750d6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1970101f40 0x7f1970199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f19750d6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1970102900 0x7f197019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f19750d6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1970107660 0x7f19701a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f19750d6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f19701117b0 con 0x7f1970102900 2024-06-06T03:34:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f19750d6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1970111630 con 0x7f1970101f40 2024-06-06T03:34:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f19750d6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1970111930 con 0x7f1970107660 2024-06-06T03:34:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f196e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1970102900 0x7f197019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f196e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1970102900 0x7f197019a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33956/0 (socket says 172.21.15.40:33956) 2024-06-06T03:34:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f196e59c700 1 -- 172.21.15.40:0/109563520 learned_addr learned my addr 172.21.15.40:0/109563520 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:34:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f196f59e700 1 --2- 172.21.15.40:0/109563520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1970107660 0x7f19701a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f196ed9d700 1 --2- 172.21.15.40:0/109563520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1970101f40 0x7f1970199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f196e59c700 1 -- 172.21.15.40:0/109563520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1970107660 msgr2=0x7f19701a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f196e59c700 1 --2- 172.21.15.40:0/109563520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1970107660 0x7f19701a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f196e59c700 1 -- 172.21.15.40:0/109563520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1970101f40 msgr2=0x7f1970199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f196e59c700 1 --2- 172.21.15.40:0/109563520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1970101f40 0x7f1970199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f196e59c700 1 -- 172.21.15.40:0/109563520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19701a5110 con 0x7f1970102900 2024-06-06T03:34:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.052+0000 7f196e59c700 1 --2- 172.21.15.40:0/109563520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1970102900 0x7f197019a090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f196400c920 tx=0x7f196400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.053+0000 7f1957fff700 1 -- 172.21.15.40:0/109563520 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964007b00 con 0x7f1970102900 2024-06-06T03:34:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.053+0000 7f19750d6700 1 -- 172.21.15.40:0/109563520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19701a5340 con 0x7f1970102900 2024-06-06T03:34:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.053+0000 7f1957fff700 1 -- 172.21.15.40:0/109563520 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1964004d10 con 0x7f1970102900 2024-06-06T03:34:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.053+0000 7f1957fff700 1 -- 172.21.15.40:0/109563520 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964005630 con 0x7f1970102900 2024-06-06T03:34:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.053+0000 7f19750d6700 1 -- 172.21.15.40:0/109563520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19701a58a0 con 0x7f1970102900 2024-06-06T03:34:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.054+0000 7f19750d6700 1 -- 172.21.15.40:0/109563520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1970069180 con 0x7f1970102900 2024-06-06T03:34:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.055+0000 7f1957fff700 1 -- 172.21.15.40:0/109563520 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1964004750 con 0x7f1970102900 2024-06-06T03:34:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.055+0000 7f1957fff700 1 --2- 172.21.15.40:0/109563520 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f19580a2c20 0x7f19580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.056+0000 7f196ed9d700 1 --2- 172.21.15.40:0/109563520 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f19580a2c20 0x7f19580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.056+0000 7f1957fff700 1 -- 172.21.15.40:0/109563520 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f19640ceb00 con 0x7f1970102900 2024-06-06T03:34:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.056+0000 7f196ed9d700 1 --2- 172.21.15.40:0/109563520 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f19580a2c20 0x7f19580a50e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1960000c00 tx=0x7f1960005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:40.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.060+0000 7f1957fff700 1 -- 172.21.15.40:0/109563520 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1964014030 con 0x7f1970102900 2024-06-06T03:34:40.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.312+0000 7f19750d6700 1 -- 172.21.15.40:0/109563520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1970101420 con 0x7f1970102900 2024-06-06T03:34:40.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.312+0000 7f1957fff700 1 -- 172.21.15.40:0/109563520 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7f196409bf00 con 0x7f1970102900 2024-06-06T03:34:40.312 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:34:40.312 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24358,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24376,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-06-06T03:34:40.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.315+0000 7f19750d6700 1 -- 172.21.15.40:0/109563520 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f19580a2c20 msgr2=0x7f19580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:40.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.315+0000 7f19750d6700 1 --2- 172.21.15.40:0/109563520 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f19580a2c20 0x7f19580a50e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1960000c00 tx=0x7f1960005fd0 comp rx=0 tx=0).stop 2024-06-06T03:34:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.315+0000 7f19750d6700 1 -- 172.21.15.40:0/109563520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1970102900 msgr2=0x7f197019a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.315+0000 7f19750d6700 1 --2- 172.21.15.40:0/109563520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1970102900 0x7f197019a090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f196400c920 tx=0x7f196400cc30 comp rx=0 tx=0).stop 2024-06-06T03:34:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.315+0000 7f19750d6700 1 -- 172.21.15.40:0/109563520 shutdown_connections 2024-06-06T03:34:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.315+0000 7f19750d6700 1 --2- 172.21.15.40:0/109563520 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f19580a2c20 0x7f19580a50e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.315+0000 7f19750d6700 1 --2- 172.21.15.40:0/109563520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1970107660 0x7f19701a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.315+0000 7f19750d6700 1 --2- 172.21.15.40:0/109563520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1970101f40 0x7f1970199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.315+0000 7f19750d6700 1 --2- 172.21.15.40:0/109563520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1970102900 0x7f197019a090 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.315+0000 7f19750d6700 1 -- 172.21.15.40:0/109563520 >> 172.21.15.40:0/109563520 conn(0x7f19700766b0 msgr2=0x7f19700fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.315+0000 7f19750d6700 1 -- 172.21.15.40:0/109563520 shutdown_connections 2024-06-06T03:34:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:40.315+0000 7f19750d6700 1 -- 172.21.15.40:0/109563520 wait complete. 2024-06-06T03:34:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2024-06-06T03:34:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:40 smithi131 ceph-mon[36687]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:40 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:40 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:40 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:40 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:40 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:40 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:40 smithi131 ceph-mon[36687]: Deploying daemon mds.c on smithi131 2024-06-06T03:34:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:40 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.131:6834/1221528906,v1:172.21.15.131:6835/1221528906] up:boot 2024-06-06T03:34:41.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:40 smithi131 ceph-mon[36687]: fsmap 6 up:standby 2024-06-06T03:34:41.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:40 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T03:34:41.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:40 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/109563520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:41.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:40 smithi040 ceph-mon[32220]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:41.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:40 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:41.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:40 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:41.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:40 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:41.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:40 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:41.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:40 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:41.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:40 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:41.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:40 smithi040 ceph-mon[32220]: Deploying daemon mds.c on smithi131 2024-06-06T03:34:41.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:40 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.131:6834/1221528906,v1:172.21.15.131:6835/1221528906] up:boot 2024-06-06T03:34:41.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:40 smithi040 ceph-mon[32220]: fsmap 6 up:standby 2024-06-06T03:34:41.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:40 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T03:34:41.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:40 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/109563520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:41.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:40 smithi104 ceph-mon[36569]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:41.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:40 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:41.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:40 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:41.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:40 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:41.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:40 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:41.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:40 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:41.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:40 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:41.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:40 smithi104 ceph-mon[36569]: Deploying daemon mds.c on smithi131 2024-06-06T03:34:41.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:40 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.131:6834/1221528906,v1:172.21.15.131:6835/1221528906] up:boot 2024-06-06T03:34:41.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:40 smithi104 ceph-mon[36569]: fsmap 6 up:standby 2024-06-06T03:34:41.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:40 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T03:34:41.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:40 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/109563520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:42.231 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:34:42 smithi131 systemd[1]: Starting Ceph mds.c for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:34:42.514 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:34:42 smithi131 podman[67807]: 2024-06-06 03:34:42.13488427 +0000 UTC m=+0.016998418 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:34:42.514 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:34:42 smithi131 podman[67807]: 2024-06-06 03:34:42.251326677 +0000 UTC m=+0.133440846 container create 0e5f414d00ff75f688ee3aff5e501b9747e9ba28c860f9a0979d5a5eec9c8a74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-06T03:34:42.514 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:34:42 smithi131 podman[67807]: 2024-06-06 03:34:42.514405219 +0000 UTC m=+0.396519370 container init 0e5f414d00ff75f688ee3aff5e501b9747e9ba28c860f9a0979d5a5eec9c8a74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-06T03:34:42.782 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:34:42 smithi131 podman[67807]: 2024-06-06 03:34:42.51724899 +0000 UTC m=+0.399363137 container start 0e5f414d00ff75f688ee3aff5e501b9747e9ba28c860f9a0979d5a5eec9c8a74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-06T03:34:42.782 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:34:42 smithi131 ceph-mds[67827]: -- 172.21.15.131:0/2664145255 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fca6e51a40 con 0x55fca6e25000 2024-06-06T03:34:42.782 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:34:42 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-c[67822]: starting mds.c at 2024-06-06T03:34:42.782 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:34:42 smithi131 bash[67807]: 0e5f414d00ff75f688ee3aff5e501b9747e9ba28c860f9a0979d5a5eec9c8a74 2024-06-06T03:34:42.782 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:34:42 smithi131 systemd[1]: Started Ceph mds.c for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:34:42.953 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:34:43.216 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:34:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:42 smithi131 ceph-mon[36687]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:42 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:43.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:42 smithi040 ceph-mon[32220]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:43.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:43.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:43.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:43.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:43.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:43.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:42 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:43.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:42 smithi104 ceph-mon[36569]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:43.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:43.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:43.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:43.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:43.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:43.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:42 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:44.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:43 smithi104 ceph-mon[36569]: Deploying daemon mds.h on smithi104 2024-06-06T03:34:44.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:43 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.131:6836/526159320,v1:172.21.15.131:6837/526159320] up:boot 2024-06-06T03:34:44.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:43 smithi104 ceph-mon[36569]: fsmap 7 up:standby 2024-06-06T03:34:44.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:43 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T03:34:44.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:43 smithi131 ceph-mon[36687]: Deploying daemon mds.h on smithi104 2024-06-06T03:34:44.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:43 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.131:6836/526159320,v1:172.21.15.131:6837/526159320] up:boot 2024-06-06T03:34:44.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:43 smithi131 ceph-mon[36687]: fsmap 7 up:standby 2024-06-06T03:34:44.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:43 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T03:34:44.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:43 smithi040 ceph-mon[32220]: Deploying daemon mds.h on smithi104 2024-06-06T03:34:44.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:43 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.131:6836/526159320,v1:172.21.15.131:6837/526159320] up:boot 2024-06-06T03:34:44.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:43 smithi040 ceph-mon[32220]: fsmap 7 up:standby 2024-06-06T03:34:44.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:43 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T03:34:44.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.964+0000 7fc36de1f700 1 -- 172.21.15.40:0/352064722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc368102740 msgr2=0x7fc368102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:44.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.964+0000 7fc36de1f700 1 --2- 172.21.15.40:0/352064722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc368102740 0x7fc368102b20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc3580099c0 tx=0x7fc358009cd0 comp rx=0 tx=0).stop 2024-06-06T03:34:44.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.964+0000 7fc36de1f700 1 -- 172.21.15.40:0/352064722 shutdown_connections 2024-06-06T03:34:44.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.964+0000 7fc36de1f700 1 --2- 172.21.15.40:0/352064722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368107840 0x7fc36810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:44.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.964+0000 7fc36de1f700 1 --2- 172.21.15.40:0/352064722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc368102740 0x7fc368102b20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:44.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.964+0000 7fc36de1f700 1 --2- 172.21.15.40:0/352064722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3681030f0 0x7fc368107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:44.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.964+0000 7fc36de1f700 1 -- 172.21.15.40:0/352064722 >> 172.21.15.40:0/352064722 conn(0x7fc3680795f0 msgr2=0x7fc368079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:44.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.964+0000 7fc36de1f700 1 -- 172.21.15.40:0/352064722 shutdown_connections 2024-06-06T03:34:44.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.964+0000 7fc36de1f700 1 -- 172.21.15.40:0/352064722 wait complete. 2024-06-06T03:34:44.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.964+0000 7fc36de1f700 1 Processor -- start 2024-06-06T03:34:44.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc36de1f700 1 -- start start 2024-06-06T03:34:44.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc36de1f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368102740 0x7fc36819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:44.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc36de1f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3681030f0 0x7fc36819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:44.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc36de1f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc368107840 0x7fc368198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:44.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc36de1f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc368112780 con 0x7fc368107840 2024-06-06T03:34:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc36de1f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc368112600 con 0x7fc3681030f0 2024-06-06T03:34:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc36de1f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc368112900 con 0x7fc368102740 2024-06-06T03:34:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc367fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc368107840 0x7fc368198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc367fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc368107840 0x7fc368198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33988/0 (socket says 172.21.15.40:33988) 2024-06-06T03:34:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc367fff700 1 -- 172.21.15.40:0/4079933120 learned_addr learned my addr 172.21.15.40:0/4079933120 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:34:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc366ffd700 1 --2- 172.21.15.40:0/4079933120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3681030f0 0x7fc36819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc3677fe700 1 --2- 172.21.15.40:0/4079933120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368102740 0x7fc36819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc367fff700 1 -- 172.21.15.40:0/4079933120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368102740 msgr2=0x7fc36819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc367fff700 1 --2- 172.21.15.40:0/4079933120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368102740 0x7fc36819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc367fff700 1 -- 172.21.15.40:0/4079933120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3681030f0 msgr2=0x7fc36819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc367fff700 1 --2- 172.21.15.40:0/4079933120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3681030f0 0x7fc36819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.965+0000 7fc367fff700 1 -- 172.21.15.40:0/4079933120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc368199160 con 0x7fc368107840 2024-06-06T03:34:44.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.966+0000 7fc367fff700 1 --2- 172.21.15.40:0/4079933120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc368107840 0x7fc368198a00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc360007ee0 tx=0x7fc36000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:44.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.966+0000 7fc364ff9700 1 -- 172.21.15.40:0/4079933120 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360004bb0 con 0x7fc368107840 2024-06-06T03:34:44.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.966+0000 7fc364ff9700 1 -- 172.21.15.40:0/4079933120 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc360004d50 con 0x7fc368107840 2024-06-06T03:34:44.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.966+0000 7fc36de1f700 1 -- 172.21.15.40:0/4079933120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc368199450 con 0x7fc368107840 2024-06-06T03:34:44.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.966+0000 7fc364ff9700 1 -- 172.21.15.40:0/4079933120 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360005960 con 0x7fc368107840 2024-06-06T03:34:44.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.966+0000 7fc36de1f700 1 -- 172.21.15.40:0/4079933120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3681a5710 con 0x7fc368107840 2024-06-06T03:34:44.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.967+0000 7fc36de1f700 1 -- 172.21.15.40:0/4079933120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc36810aa80 con 0x7fc368107840 2024-06-06T03:34:44.968 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:44 smithi040 ceph-mon[32220]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:44.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.968+0000 7fc364ff9700 1 -- 172.21.15.40:0/4079933120 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc36001a020 con 0x7fc368107840 2024-06-06T03:34:44.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.969+0000 7fc364ff9700 1 --2- 172.21.15.40:0/4079933120 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc3500a2c20 0x7fc3500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:44.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.969+0000 7fc3677fe700 1 --2- 172.21.15.40:0/4079933120 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc3500a2c20 0x7fc3500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:44.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.969+0000 7fc364ff9700 1 -- 172.21.15.40:0/4079933120 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc3600ce380 con 0x7fc368107840 2024-06-06T03:34:44.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.970+0000 7fc3677fe700 1 --2- 172.21.15.40:0/4079933120 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc3500a2c20 0x7fc3500a50e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc358000c00 tx=0x7fc358005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:44.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:44.974+0000 7fc364ff9700 1 -- 172.21.15.40:0/4079933120 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc360010070 con 0x7fc368107840 2024-06-06T03:34:45.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:44 smithi104 ceph-mon[36569]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:45.228+0000 7fc36de1f700 1 -- 172.21.15.40:0/4079933120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc36810a460 con 0x7fc368107840 2024-06-06T03:34:45.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:45.229+0000 7fc364ff9700 1 -- 172.21.15.40:0/4079933120 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7fc36009b6e0 con 0x7fc368107840 2024-06-06T03:34:45.228 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:34:45.228 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24358,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24376,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-06-06T03:34:45.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:45.231+0000 7fc36de1f700 1 -- 172.21.15.40:0/4079933120 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc3500a2c20 msgr2=0x7fc3500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:45.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:45.231+0000 7fc36de1f700 1 --2- 172.21.15.40:0/4079933120 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc3500a2c20 0x7fc3500a50e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc358000c00 tx=0x7fc358005fb0 comp rx=0 tx=0).stop 2024-06-06T03:34:45.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:45.231+0000 7fc36de1f700 1 -- 172.21.15.40:0/4079933120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc368107840 msgr2=0x7fc368198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:45.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:45.231+0000 7fc36de1f700 1 --2- 172.21.15.40:0/4079933120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc368107840 0x7fc368198a00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc360007ee0 tx=0x7fc36000d3b0 comp rx=0 tx=0).stop 2024-06-06T03:34:45.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:45.231+0000 7fc36de1f700 1 -- 172.21.15.40:0/4079933120 shutdown_connections 2024-06-06T03:34:45.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:45.231+0000 7fc36de1f700 1 --2- 172.21.15.40:0/4079933120 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc3500a2c20 0x7fc3500a50e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:45.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:45.231+0000 7fc36de1f700 1 --2- 172.21.15.40:0/4079933120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368102740 0x7fc36819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:45.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:45.231+0000 7fc36de1f700 1 --2- 172.21.15.40:0/4079933120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc368107840 0x7fc368198a00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:45.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:45.231+0000 7fc36de1f700 1 --2- 172.21.15.40:0/4079933120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3681030f0 0x7fc36819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:45.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:45.231+0000 7fc36de1f700 1 -- 172.21.15.40:0/4079933120 >> 172.21.15.40:0/4079933120 conn(0x7fc3680795f0 msgr2=0x7fc368100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:45.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:45.231+0000 7fc36de1f700 1 -- 172.21.15.40:0/4079933120 shutdown_connections 2024-06-06T03:34:45.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:45.232+0000 7fc36de1f700 1 -- 172.21.15.40:0/4079933120 wait complete. 2024-06-06T03:34:45.232 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-06-06T03:34:45.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:44 smithi131 ceph-mon[36687]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:45.840 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jun 06 03:34:45 smithi104 systemd[1]: Started Ceph mds.h for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:34:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:45 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/4079933120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:45 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:45 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:45 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:45 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:45 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:45 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:46.261 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:45 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/4079933120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:46.262 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:45 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:46.262 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:45 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:46.262 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:45 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:46.262 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:45 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:46.262 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:45 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:46.262 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:45 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:46.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:45 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/4079933120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:46.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:45 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:46.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:45 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:46.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:45 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:46.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:45 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:46.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:45 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:46.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:45 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:46 smithi131 ceph-mon[36687]: Deploying daemon mds.i on smithi131 2024-06-06T03:34:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:46 smithi131 ceph-mon[36687]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:46 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] up:boot 2024-06-06T03:34:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:46 smithi131 ceph-mon[36687]: fsmap 8 up:standby 2024-06-06T03:34:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:46 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T03:34:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:46 smithi040 ceph-mon[32220]: Deploying daemon mds.i on smithi131 2024-06-06T03:34:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:46 smithi040 ceph-mon[32220]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:46 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] up:boot 2024-06-06T03:34:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:46 smithi040 ceph-mon[32220]: fsmap 8 up:standby 2024-06-06T03:34:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:46 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T03:34:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:46 smithi104 ceph-mon[36569]: Deploying daemon mds.i on smithi131 2024-06-06T03:34:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:46 smithi104 ceph-mon[36569]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:46 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] up:boot 2024-06-06T03:34:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:46 smithi104 ceph-mon[36569]: fsmap 8 up:standby 2024-06-06T03:34:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:46 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T03:34:47.856 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:34:48.032 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:34:47 smithi131 systemd[1]: Starting Ceph mds.i for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:34:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:34:48.370 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:34:48 smithi131 podman[68108]: 2024-06-06 03:34:48.017928101 +0000 UTC m=+0.017147391 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:34:48.370 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:34:48 smithi131 podman[68108]: 2024-06-06 03:34:48.126786914 +0000 UTC m=+0.126006172 container create 4d9bd292e896d9b3b0cc27a88add1de880213fa2d077824c46cef89365209259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-i, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-06T03:34:48.370 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:34:48 smithi131 podman[68108]: 2024-06-06 03:34:48.37133503 +0000 UTC m=+0.370554297 container init 4d9bd292e896d9b3b0cc27a88add1de880213fa2d077824c46cef89365209259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:34:48.782 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:34:48 smithi131 podman[68108]: 2024-06-06 03:34:48.37386669 +0000 UTC m=+0.373085959 container start 4d9bd292e896d9b3b0cc27a88add1de880213fa2d077824c46cef89365209259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-06T03:34:48.782 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:34:48 smithi131 ceph-mds[68127]: -- 172.21.15.131:0/553344975 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624cb45ba40 con 0x5624cb42ec00 2024-06-06T03:34:48.782 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:34:48 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-i[68123]: starting mds.i at 2024-06-06T03:34:48.782 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:34:48 smithi131 bash[68108]: 4d9bd292e896d9b3b0cc27a88add1de880213fa2d077824c46cef89365209259 2024-06-06T03:34:48.782 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:34:48 smithi131 systemd[1]: Started Ceph mds.i for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:34:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:49 smithi040 ceph-mon[32220]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:49 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] up:boot 2024-06-06T03:34:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:49 smithi040 ceph-mon[32220]: fsmap 9 up:standby 2024-06-06T03:34:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:49 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T03:34:49.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:49 smithi131 ceph-mon[36687]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:49.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:49.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:49.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:49.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:49.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:49.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:49.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:49 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] up:boot 2024-06-06T03:34:49.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:49 smithi131 ceph-mon[36687]: fsmap 9 up:standby 2024-06-06T03:34:49.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:49 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T03:34:49.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:49 smithi104 ceph-mon[36569]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:49.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:49.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:49.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:49.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:49.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:49.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:49.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:49 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] up:boot 2024-06-06T03:34:49.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:49 smithi104 ceph-mon[36569]: fsmap 9 up:standby 2024-06-06T03:34:49.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:49 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T03:34:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.878+0000 7f4dfd039700 1 -- 172.21.15.40:0/3183475987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4df8100a80 msgr2=0x7f4df8100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.878+0000 7f4dfd039700 1 --2- 172.21.15.40:0/3183475987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4df8100a80 0x7f4df8100f00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009990 tx=0x7f4de0009ca0 comp rx=0 tx=0).stop 2024-06-06T03:34:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.878+0000 7f4dfd039700 1 -- 172.21.15.40:0/3183475987 shutdown_connections 2024-06-06T03:34:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.878+0000 7f4dfd039700 1 --2- 172.21.15.40:0/3183475987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4df8108510 0x7f4df81088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.878+0000 7f4dfd039700 1 --2- 172.21.15.40:0/3183475987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df8101440 0x7f4df810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.878+0000 7f4dfd039700 1 --2- 172.21.15.40:0/3183475987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4df8100a80 0x7f4df8100f00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.878+0000 7f4dfd039700 1 -- 172.21.15.40:0/3183475987 >> 172.21.15.40:0/3183475987 conn(0x7f4df8077a00 msgr2=0x7f4df80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.878+0000 7f4dfd039700 1 -- 172.21.15.40:0/3183475987 shutdown_connections 2024-06-06T03:34:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.878+0000 7f4dfd039700 1 -- 172.21.15.40:0/3183475987 wait complete. 2024-06-06T03:34:49.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.879+0000 7f4dfd039700 1 Processor -- start 2024-06-06T03:34:49.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.879+0000 7f4dfd039700 1 -- start start 2024-06-06T03:34:49.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dfd039700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df8100a80 0x7f4df8101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:49.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dfd039700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4df8101440 0x7f4df81021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:49.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dfd039700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4df8108510 0x7f4df81026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:49.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dfd039700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81127f0 con 0x7f4df8101440 2024-06-06T03:34:49.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dfd039700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8112670 con 0x7f4df8100a80 2024-06-06T03:34:49.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dfd039700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8112970 con 0x7f4df8108510 2024-06-06T03:34:49.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dee59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4df8101440 0x7f4df81021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:49.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dee59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4df8101440 0x7f4df81021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59206/0 (socket says 172.21.15.40:59206) 2024-06-06T03:34:49.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dee59c700 1 -- 172.21.15.40:0/646893357 learned_addr learned my addr 172.21.15.40:0/646893357 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:34:49.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4df6d9d700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df8100a80 0x7f4df8101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4df759e700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4df8108510 0x7f4df81026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dee59c700 1 -- 172.21.15.40:0/646893357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4df8108510 msgr2=0x7f4df81026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dee59c700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4df8108510 0x7f4df81026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dee59c700 1 -- 172.21.15.40:0/646893357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df8100a80 msgr2=0x7f4df8101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dee59c700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df8100a80 0x7f4df8101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dee59c700 1 -- 172.21.15.40:0/646893357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df81a7130 con 0x7f4df8101440 2024-06-06T03:34:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4df6d9d700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df8100a80 0x7f4df8101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:34:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4df759e700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4df8108510 0x7f4df81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:34:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4dee59c700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4df8101440 0x7f4df81021a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4de000bff0 tx=0x7f4de0005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4df4d99700 1 -- 172.21.15.40:0/646893357 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0027070 con 0x7f4df8101440 2024-06-06T03:34:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.880+0000 7f4df4d99700 1 -- 172.21.15.40:0/646893357 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de0004150 con 0x7f4df8101440 2024-06-06T03:34:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.881+0000 7f4df4d99700 1 -- 172.21.15.40:0/646893357 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de000f460 con 0x7f4df8101440 2024-06-06T03:34:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.881+0000 7f4dfd039700 1 -- 172.21.15.40:0/646893357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df81a73c0 con 0x7f4df8101440 2024-06-06T03:34:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.881+0000 7f4dfd039700 1 -- 172.21.15.40:0/646893357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df81a78a0 con 0x7f4df8101440 2024-06-06T03:34:49.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.881+0000 7f4dfd039700 1 -- 172.21.15.40:0/646893357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df8069180 con 0x7f4df8101440 2024-06-06T03:34:49.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.883+0000 7f4df4d99700 1 -- 172.21.15.40:0/646893357 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4de002a070 con 0x7f4df8101440 2024-06-06T03:34:49.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.883+0000 7f4df4d99700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4de40a2c20 0x7f4de40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:49.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.884+0000 7f4df6d9d700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4de40a2c20 0x7f4de40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:49.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.884+0000 7f4df4d99700 1 -- 172.21.15.40:0/646893357 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4de00d0260 con 0x7f4df8101440 2024-06-06T03:34:49.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.884+0000 7f4df6d9d700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4de40a2c20 0x7f4de40a50e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4de8005fd0 tx=0x7f4de8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:49.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:49.887+0000 7f4df4d99700 1 -- 172.21.15.40:0/646893357 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4de0031020 con 0x7f4df8101440 2024-06-06T03:34:50.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:50.140+0000 7f4dfd039700 1 -- 172.21.15.40:0/646893357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4df802dca0 con 0x7f4df8101440 2024-06-06T03:34:50.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:50.141+0000 7f4df4d99700 1 -- 172.21.15.40:0/646893357 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7f4de0013070 con 0x7f4df8101440 2024-06-06T03:34:50.140 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:34:50.140 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24358,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24376,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-06-06T03:34:50.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:50.143+0000 7f4dfd039700 1 -- 172.21.15.40:0/646893357 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4de40a2c20 msgr2=0x7f4de40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:50.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:50.143+0000 7f4dfd039700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4de40a2c20 0x7f4de40a50e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4de8005fd0 tx=0x7f4de8005e30 comp rx=0 tx=0).stop 2024-06-06T03:34:50.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:50.143+0000 7f4dfd039700 1 -- 172.21.15.40:0/646893357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4df8101440 msgr2=0x7f4df81021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:50.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:50.143+0000 7f4dfd039700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4df8101440 0x7f4df81021a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4de000bff0 tx=0x7f4de0005640 comp rx=0 tx=0).stop 2024-06-06T03:34:50.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:50.143+0000 7f4dfd039700 1 -- 172.21.15.40:0/646893357 shutdown_connections 2024-06-06T03:34:50.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:50.143+0000 7f4dfd039700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f4de40a2c20 0x7f4de40a50e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:50.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:50.143+0000 7f4dfd039700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4df8108510 0x7f4df81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:50.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:50.143+0000 7f4dfd039700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df8100a80 0x7f4df8101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:50.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:50.143+0000 7f4dfd039700 1 --2- 172.21.15.40:0/646893357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4df8101440 0x7f4df81021a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:50.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:50.143+0000 7f4dfd039700 1 -- 172.21.15.40:0/646893357 >> 172.21.15.40:0/646893357 conn(0x7f4df8077a00 msgr2=0x7f4df80fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:50.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:50.143+0000 7f4dfd039700 1 -- 172.21.15.40:0/646893357 shutdown_connections 2024-06-06T03:34:50.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:50.144+0000 7f4dfd039700 1 -- 172.21.15.40:0/646893357 wait complete. 2024-06-06T03:34:50.143 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 10 2024-06-06T03:34:50.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:50 smithi131 ceph-mon[36687]: Deploying daemon mds.g on smithi040 2024-06-06T03:34:50.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:50 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/646893357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:50.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:50 smithi040 ceph-mon[32220]: Deploying daemon mds.g on smithi040 2024-06-06T03:34:50.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:50 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/646893357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:50.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:50 smithi104 ceph-mon[36569]: Deploying daemon mds.g on smithi040 2024-06-06T03:34:50.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:50 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/646893357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:51.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:51 smithi131 ceph-mon[36687]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:51.819 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:51 smithi040 ceph-mon[32220]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:51.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:51 smithi104 ceph-mon[36569]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:53.080 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jun 06 03:34:52 smithi040 systemd[1]: Started Ceph mds.g for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:34:53.257 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:34:53.553 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:53 smithi040 ceph-mon[32220]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:53.553 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:53 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:53.553 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:53 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:53.553 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:53 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:53.553 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:53 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:53.554 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:53 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:53.554 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:53 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:53.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:53 smithi131 ceph-mon[36687]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:53.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:53 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:53.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:53 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:53.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:53 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:53.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:53 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:53.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:53 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:53.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:53 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:53.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:53 smithi104 ceph-mon[36569]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:53.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:53 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:53.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:53 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:53.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:53 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:53.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:53 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:53.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:53 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:53.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:53 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:34:54.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:54 smithi040 ceph-mon[32220]: Deploying daemon mds.a on smithi040 2024-06-06T03:34:54.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:54 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.40:6838/1930069210,v1:172.21.15.40:6839/1930069210] up:boot 2024-06-06T03:34:54.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:54 smithi040 ceph-mon[32220]: fsmap 10 up:standby 2024-06-06T03:34:54.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:54 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T03:34:54.781 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:54 smithi131 ceph-mon[36687]: Deploying daemon mds.a on smithi040 2024-06-06T03:34:54.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:54 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.40:6838/1930069210,v1:172.21.15.40:6839/1930069210] up:boot 2024-06-06T03:34:54.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:54 smithi131 ceph-mon[36687]: fsmap 10 up:standby 2024-06-06T03:34:54.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:54 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T03:34:54.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:54 smithi104 ceph-mon[36569]: Deploying daemon mds.a on smithi040 2024-06-06T03:34:54.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:54 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.40:6838/1930069210,v1:172.21.15.40:6839/1930069210] up:boot 2024-06-06T03:34:54.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:54 smithi104 ceph-mon[36569]: fsmap 10 up:standby 2024-06-06T03:34:54.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:54 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T03:34:55.229 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:34:55 smithi040 systemd[1]: Starting Ceph mds.a for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:34:55.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:55 smithi131 ceph-mon[36687]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:55.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:55 smithi040 ceph-mon[32220]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:55.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:55 smithi104 ceph-mon[36569]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:56.484 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:34:56 smithi040 podman[74713]: 2024-06-06 03:34:56.239443983 +0000 UTC m=+0.029047144 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:34:56.485 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:34:56 smithi040 podman[74713]: 2024-06-06 03:34:56.340565857 +0000 UTC m=+0.130169067 container create 8f73f8f00067447387e4ab151be22708f3699a840d83ec470ea5125335c0cb87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:34:56.830 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:34:56 smithi040 podman[74713]: 2024-06-06 03:34:56.576718454 +0000 UTC m=+0.366321610 container init 8f73f8f00067447387e4ab151be22708f3699a840d83ec470ea5125335c0cb87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-06T03:34:56.830 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:34:56 smithi040 podman[74713]: 2024-06-06 03:34:56.57973407 +0000 UTC m=+0.369337219 container start 8f73f8f00067447387e4ab151be22708f3699a840d83ec470ea5125335c0cb87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-a, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-06T03:34:56.830 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:34:56 smithi040 ceph-mds[74737]: -- 172.21.15.40:0/1326401697 <== mon.0 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ed6c653a40 con 0x55ed6c626c00 2024-06-06T03:34:56.830 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:34:56 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-a[74733]: starting mds.a at 2024-06-06T03:34:56.830 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:34:56 smithi040 bash[74713]: 8f73f8f00067447387e4ab151be22708f3699a840d83ec470ea5125335c0cb87 2024-06-06T03:34:56.831 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:34:56 smithi040 systemd[1]: Started Ceph mds.a for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:34:56.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.894+0000 7fcac0a0a700 1 -- 172.21.15.40:0/2091054696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc106c00 msgr2=0x7fcabc106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:56.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.894+0000 7fcac0a0a700 1 --2- 172.21.15.40:0/2091054696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc106c00 0x7fcabc106fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00b0d0 tx=0x7fcaac00b3e0 comp rx=0 tx=0).stop 2024-06-06T03:34:56.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.894+0000 7fcac0a0a700 1 -- 172.21.15.40:0/2091054696 shutdown_connections 2024-06-06T03:34:56.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.894+0000 7fcac0a0a700 1 --2- 172.21.15.40:0/2091054696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcabc101510 0x7fcabc10d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:56.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.894+0000 7fcac0a0a700 1 --2- 172.21.15.40:0/2091054696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc106c00 0x7fcabc106fe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:56.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.894+0000 7fcac0a0a700 1 --2- 172.21.15.40:0/2091054696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcabc100b50 0x7fcabc100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.894+0000 7fcac0a0a700 1 -- 172.21.15.40:0/2091054696 >> 172.21.15.40:0/2091054696 conn(0x7fcabc077990 msgr2=0x7fcabc0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.894+0000 7fcac0a0a700 1 -- 172.21.15.40:0/2091054696 shutdown_connections 2024-06-06T03:34:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.894+0000 7fcac0a0a700 1 -- 172.21.15.40:0/2091054696 wait complete. 2024-06-06T03:34:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.895+0000 7fcac0a0a700 1 Processor -- start 2024-06-06T03:34:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.895+0000 7fcac0a0a700 1 -- start start 2024-06-06T03:34:56.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcac0a0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc100b50 0x7fcabc19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcac0a0a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcabc101510 0x7fcabc19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcac0a0a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcabc106c00 0x7fcabc1a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcac0a0a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1105e0 con 0x7fcabc100b50 2024-06-06T03:34:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcac0a0a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc110460 con 0x7fcabc106c00 2024-06-06T03:34:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcac0a0a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc110760 con 0x7fcabc101510 2024-06-06T03:34:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcaba59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc100b50 0x7fcabc19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcaba59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc100b50 0x7fcabc19f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59250/0 (socket says 172.21.15.40:59250) 2024-06-06T03:34:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcaba59c700 1 -- 172.21.15.40:0/4059646077 learned_addr learned my addr 172.21.15.40:0/4059646077 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:34:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcab9d9b700 1 --2- 172.21.15.40:0/4059646077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcabc101510 0x7fcabc19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcabad9d700 1 --2- 172.21.15.40:0/4059646077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcabc106c00 0x7fcabc1a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcaba59c700 1 -- 172.21.15.40:0/4059646077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcabc101510 msgr2=0x7fcabc19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcaba59c700 1 --2- 172.21.15.40:0/4059646077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcabc101510 0x7fcabc19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcaba59c700 1 -- 172.21.15.40:0/4059646077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcabc106c00 msgr2=0x7fcabc1a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcaba59c700 1 --2- 172.21.15.40:0/4059646077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcabc106c00 0x7fcabc1a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcaba59c700 1 -- 172.21.15.40:0/4059646077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc1a41d0 con 0x7fcabc100b50 2024-06-06T03:34:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.896+0000 7fcaba59c700 1 --2- 172.21.15.40:0/4059646077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc100b50 0x7fcabc19f1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00d010 tx=0x7fcaac007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.897+0000 7fcab37fe700 1 -- 172.21.15.40:0/4059646077 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac021070 con 0x7fcabc100b50 2024-06-06T03:34:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.897+0000 7fcac0a0a700 1 -- 172.21.15.40:0/4059646077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc1a4460 con 0x7fcabc100b50 2024-06-06T03:34:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.897+0000 7fcab37fe700 1 -- 172.21.15.40:0/4059646077 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaac01db30 con 0x7fcabc100b50 2024-06-06T03:34:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.897+0000 7fcab37fe700 1 -- 172.21.15.40:0/4059646077 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac01bb00 con 0x7fcabc100b50 2024-06-06T03:34:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.897+0000 7fcac0a0a700 1 -- 172.21.15.40:0/4059646077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc1abd40 con 0x7fcabc100b50 2024-06-06T03:34:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.898+0000 7fcac0a0a700 1 -- 172.21.15.40:0/4059646077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcabc109090 con 0x7fcabc100b50 2024-06-06T03:34:56.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.899+0000 7fcab37fe700 1 -- 172.21.15.40:0/4059646077 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcaac024070 con 0x7fcabc100b50 2024-06-06T03:34:56.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.900+0000 7fcab37fe700 1 --2- 172.21.15.40:0/4059646077 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fcaa80a2c20 0x7fcaa80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:34:56.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.900+0000 7fcab9d9b700 1 --2- 172.21.15.40:0/4059646077 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fcaa80a2c20 0x7fcaa80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:34:56.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.900+0000 7fcab37fe700 1 -- 172.21.15.40:0/4059646077 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fcaac00f020 con 0x7fcabc100b50 2024-06-06T03:34:56.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.900+0000 7fcab9d9b700 1 --2- 172.21.15.40:0/4059646077 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fcaa80a2c20 0x7fcaa80a50e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcabc1047e0 tx=0x7fcaa400b4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:34:56.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:56.904+0000 7fcab37fe700 1 -- 172.21.15.40:0/4059646077 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcaac02b030 con 0x7fcabc100b50 2024-06-06T03:34:57.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:57.157+0000 7fcac0a0a700 1 -- 172.21.15.40:0/4059646077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcabc1a0520 con 0x7fcabc100b50 2024-06-06T03:34:57.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:57.158+0000 7fcab37fe700 1 -- 172.21.15.40:0/4059646077 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7fcaac09dc90 con 0x7fcabc100b50 2024-06-06T03:34:57.157 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:34:57.158 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24358,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24376,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-06-06T03:34:57.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:57.160+0000 7fcac0a0a700 1 -- 172.21.15.40:0/4059646077 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fcaa80a2c20 msgr2=0x7fcaa80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:57.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:57.160+0000 7fcac0a0a700 1 --2- 172.21.15.40:0/4059646077 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fcaa80a2c20 0x7fcaa80a50e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcabc1047e0 tx=0x7fcaa400b4c0 comp rx=0 tx=0).stop 2024-06-06T03:34:57.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:57.160+0000 7fcac0a0a700 1 -- 172.21.15.40:0/4059646077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc100b50 msgr2=0x7fcabc19f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:34:57.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:57.160+0000 7fcac0a0a700 1 --2- 172.21.15.40:0/4059646077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc100b50 0x7fcabc19f1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00d010 tx=0x7fcaac007a20 comp rx=0 tx=0).stop 2024-06-06T03:34:57.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:57.161+0000 7fcac0a0a700 1 -- 172.21.15.40:0/4059646077 shutdown_connections 2024-06-06T03:34:57.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:57.161+0000 7fcac0a0a700 1 --2- 172.21.15.40:0/4059646077 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fcaa80a2c20 0x7fcaa80a50e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:57.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:57.161+0000 7fcac0a0a700 1 --2- 172.21.15.40:0/4059646077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcabc106c00 0x7fcabc1a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:57.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:57.161+0000 7fcac0a0a700 1 --2- 172.21.15.40:0/4059646077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc100b50 0x7fcabc19f1c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:57.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:57.161+0000 7fcac0a0a700 1 --2- 172.21.15.40:0/4059646077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcabc101510 0x7fcabc19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:34:57.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:57.161+0000 7fcac0a0a700 1 -- 172.21.15.40:0/4059646077 >> 172.21.15.40:0/4059646077 conn(0x7fcabc077990 msgr2=0x7fcabc0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:34:57.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:57.161+0000 7fcac0a0a700 1 -- 172.21.15.40:0/4059646077 shutdown_connections 2024-06-06T03:34:57.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:34:57.161+0000 7fcac0a0a700 1 -- 172.21.15.40:0/4059646077 wait complete. 2024-06-06T03:34:57.161 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 11 2024-06-06T03:34:57.672 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:57 smithi104 ceph-mon[36569]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:57.672 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:57.673 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:57.673 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:57.673 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:57.673 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:57.673 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:57 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:57.673 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:57 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/4059646077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:57.790 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:57 smithi040 ceph-mon[32220]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:57.790 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:57.790 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:57.790 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:57.790 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:57.790 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:57.790 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:57 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:57.791 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:57 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/4059646077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:58.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:57 smithi131 ceph-mon[36687]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:58.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:58.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:58.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:34:58.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T03:34:58.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T03:34:58.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:57 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:34:58.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:57 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/4059646077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:34:58.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:58 smithi040 ceph-mon[32220]: Deploying daemon mds.b on smithi104 2024-06-06T03:34:58.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:58 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.40:6840/2460301066,v1:172.21.15.40:6841/2460301066] up:boot 2024-06-06T03:34:58.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:58 smithi040 ceph-mon[32220]: fsmap 11 up:standby 2024-06-06T03:34:58.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:58 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T03:34:58.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:34:58 smithi040 ceph-mon[32220]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:58.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:58 smithi104 ceph-mon[36569]: Deploying daemon mds.b on smithi104 2024-06-06T03:34:58.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:58 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.40:6840/2460301066,v1:172.21.15.40:6841/2460301066] up:boot 2024-06-06T03:34:58.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:58 smithi104 ceph-mon[36569]: fsmap 11 up:standby 2024-06-06T03:34:58.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:58 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T03:34:58.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:34:58 smithi104 ceph-mon[36569]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:59.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:58 smithi131 ceph-mon[36687]: Deploying daemon mds.b on smithi104 2024-06-06T03:34:59.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:58 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.40:6840/2460301066,v1:172.21.15.40:6841/2460301066] up:boot 2024-06-06T03:34:59.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:58 smithi131 ceph-mon[36687]: fsmap 11 up:standby 2024-06-06T03:34:59.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:58 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T03:34:59.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:34:58 smithi131 ceph-mon[36687]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:34:59.251 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:34:59 smithi104 systemd[1]: Starting Ceph mds.b for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:34:59.525 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:34:59 smithi104 podman[66915]: 2024-06-06 03:34:59.155209239 +0000 UTC m=+0.016782529 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:34:59.525 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:34:59 smithi104 podman[66915]: 2024-06-06 03:34:59.281280735 +0000 UTC m=+0.142854003 container create 0226ae581893891d254fe2536e97a551b0d053a1f131818065eafa570270466a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-06T03:34:59.525 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:34:59 smithi104 podman[66915]: 2024-06-06 03:34:59.526093281 +0000 UTC m=+0.387666550 container init 0226ae581893891d254fe2536e97a551b0d053a1f131818065eafa570270466a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-06T03:34:59.821 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:34:59.840 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:34:59 smithi104 podman[66915]: 2024-06-06 03:34:59.529067279 +0000 UTC m=+0.390640557 container start 0226ae581893891d254fe2536e97a551b0d053a1f131818065eafa570270466a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-06-06T03:34:59.840 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:34:59 smithi104 ceph-mds[66934]: -- 172.21.15.104:0/2422543088 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a51c7b9a40 con 0x55a51c78c800 2024-06-06T03:34:59.840 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:34:59 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b[66930]: starting mds.b at 2024-06-06T03:34:59.840 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:34:59 smithi104 bash[66915]: 0226ae581893891d254fe2536e97a551b0d053a1f131818065eafa570270466a 2024-06-06T03:34:59.841 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:34:59 smithi104 systemd[1]: Started Ceph mds.b for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:35:00.088 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:00.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:00 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] up:boot 2024-06-06T03:35:00.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:00 smithi040 ceph-mon[32220]: fsmap 12 up:standby 2024-06-06T03:35:00.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T03:35:00.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:00 smithi040 ceph-mon[32220]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:00.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:00 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:35:00.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:00 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] up:boot 2024-06-06T03:35:00.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:00 smithi131 ceph-mon[36687]: fsmap 12 up:standby 2024-06-06T03:35:00.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:00 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T03:35:00.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:00 smithi131 ceph-mon[36687]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:00.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:00 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:00 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:00 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:00 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:00 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:00 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:35:00.899 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:00 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] up:boot 2024-06-06T03:35:00.899 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:00 smithi104 ceph-mon[36569]: fsmap 12 up:standby 2024-06-06T03:35:00.899 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:00 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T03:35:00.899 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:00 smithi104 ceph-mon[36569]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:00.899 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:00 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.899 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:00 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.899 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:00 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.900 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:00 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.900 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:00 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:00.900 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:00 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:35:01.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.802+0000 7ff00078c700 1 -- 172.21.15.40:0/2250811407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff8100b50 msgr2=0x7feff8100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:01.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.802+0000 7ff00078c700 1 --2- 172.21.15.40:0/2250811407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff8100b50 0x7feff8100fd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fefec0099c0 tx=0x7fefec009cd0 comp rx=0 tx=0).stop 2024-06-06T03:35:01.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.802+0000 7ff00078c700 1 -- 172.21.15.40:0/2250811407 shutdown_connections 2024-06-06T03:35:01.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.802+0000 7ff00078c700 1 --2- 172.21.15.40:0/2250811407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff8101510 0x7feff810d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:01.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.802+0000 7ff00078c700 1 --2- 172.21.15.40:0/2250811407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feff8106c00 0x7feff8106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:01.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.802+0000 7ff00078c700 1 --2- 172.21.15.40:0/2250811407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff8100b50 0x7feff8100fd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:01.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.802+0000 7ff00078c700 1 -- 172.21.15.40:0/2250811407 >> 172.21.15.40:0/2250811407 conn(0x7feff8077990 msgr2=0x7feff80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:01.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.803+0000 7ff00078c700 1 -- 172.21.15.40:0/2250811407 shutdown_connections 2024-06-06T03:35:01.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.803+0000 7ff00078c700 1 -- 172.21.15.40:0/2250811407 wait complete. 2024-06-06T03:35:01.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.803+0000 7ff00078c700 1 Processor -- start 2024-06-06T03:35:01.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.803+0000 7ff00078c700 1 -- start start 2024-06-06T03:35:01.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.803+0000 7ff00078c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff8100b50 0x7feff819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:01.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.803+0000 7ff00078c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feff8101510 0x7feff819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:01.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.803+0000 7ff00078c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff8106c00 0x7feff81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:01.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.803+0000 7ff00078c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7feff81116f0 con 0x7feff8106c00 2024-06-06T03:35:01.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.803+0000 7ff00078c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feff8111570 con 0x7feff8101510 2024-06-06T03:35:01.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.803+0000 7ff00078c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feff8111870 con 0x7feff8100b50 2024-06-06T03:35:01.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.803+0000 7feffed29700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff8106c00 0x7feff81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:01.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7feffed29700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff8106c00 0x7feff81989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34670/0 (socket says 172.21.15.40:34670) 2024-06-06T03:35:01.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7feffed29700 1 -- 172.21.15.40:0/1103413573 learned_addr learned my addr 172.21.15.40:0/1103413573 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:01.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7feffdd27700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feff8101510 0x7feff819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:01.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7feffe528700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff8100b50 0x7feff819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:01.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7feffed29700 1 -- 172.21.15.40:0/1103413573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff8100b50 msgr2=0x7feff819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:01.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7feffed29700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff8100b50 0x7feff819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:01.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7feffed29700 1 -- 172.21.15.40:0/1103413573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feff8101510 msgr2=0x7feff819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:01.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7feffed29700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feff8101510 0x7feff819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:01.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7feffed29700 1 -- 172.21.15.40:0/1103413573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff8199140 con 0x7feff8106c00 2024-06-06T03:35:01.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7feffdd27700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feff8101510 0x7feff819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:35:01.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7feffed29700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff8106c00 0x7feff81989e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feff000e970 tx=0x7feff000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:01.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7feff77fe700 1 -- 172.21.15.40:0/1103413573 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0004a30 con 0x7feff8106c00 2024-06-06T03:35:01.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7feff77fe700 1 -- 172.21.15.40:0/1103413573 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff0005500 con 0x7feff8106c00 2024-06-06T03:35:01.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7ff00078c700 1 -- 172.21.15.40:0/1103413573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff8199430 con 0x7feff8106c00 2024-06-06T03:35:01.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7feff77fe700 1 -- 172.21.15.40:0/1103413573 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0010ce0 con 0x7feff8106c00 2024-06-06T03:35:01.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.804+0000 7ff00078c700 1 -- 172.21.15.40:0/1103413573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff81a56f0 con 0x7feff8106c00 2024-06-06T03:35:01.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.806+0000 7ff00078c700 1 -- 172.21.15.40:0/1103413573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff8069180 con 0x7feff8106c00 2024-06-06T03:35:01.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.807+0000 7feff77fe700 1 -- 172.21.15.40:0/1103413573 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7feff0005020 con 0x7feff8106c00 2024-06-06T03:35:01.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.808+0000 7feff77fe700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fefe00a2c20 0x7fefe00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:01.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.808+0000 7feffe528700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fefe00a2c20 0x7fefe00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:01.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.808+0000 7feff77fe700 1 -- 172.21.15.40:0/1103413573 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7feff00ceed0 con 0x7feff8106c00 2024-06-06T03:35:01.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.808+0000 7feffe528700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fefe00a2c20 0x7fefe00a50e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feff8199b20 tx=0x7fefe800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:01.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:01.812+0000 7feff77fe700 1 -- 172.21.15.40:0/1103413573 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feff0016330 con 0x7feff8106c00 2024-06-06T03:35:02.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:02.068+0000 7ff00078c700 1 -- 172.21.15.40:0/1103413573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feff81a5980 con 0x7feff8106c00 2024-06-06T03:35:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:02.069+0000 7feff77fe700 1 -- 172.21.15.40:0/1103413573 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10482 (secure 0 0 0) 0x7feff009c230 con 0x7feff8106c00 2024-06-06T03:35:02.068 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:35:02.068 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24358,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24376,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-06-06T03:35:02.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:02.072+0000 7ff00078c700 1 -- 172.21.15.40:0/1103413573 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fefe00a2c20 msgr2=0x7fefe00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:02.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:02.072+0000 7ff00078c700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fefe00a2c20 0x7fefe00a50e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feff8199b20 tx=0x7fefe800b040 comp rx=0 tx=0).stop 2024-06-06T03:35:02.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:02.072+0000 7ff00078c700 1 -- 172.21.15.40:0/1103413573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff8106c00 msgr2=0x7feff81989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:02.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:02.072+0000 7ff00078c700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff8106c00 0x7feff81989e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feff000e970 tx=0x7feff000ec80 comp rx=0 tx=0).stop 2024-06-06T03:35:02.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:02.072+0000 7ff00078c700 1 -- 172.21.15.40:0/1103413573 shutdown_connections 2024-06-06T03:35:02.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:02.072+0000 7ff00078c700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fefe00a2c20 0x7fefe00a50e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:02.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:02.072+0000 7ff00078c700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff8100b50 0x7feff819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:02.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:02.072+0000 7ff00078c700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff8106c00 0x7feff81989e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:02.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:02.072+0000 7ff00078c700 1 --2- 172.21.15.40:0/1103413573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feff8101510 0x7feff819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:02.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:02.072+0000 7ff00078c700 1 -- 172.21.15.40:0/1103413573 >> 172.21.15.40:0/1103413573 conn(0x7feff8077990 msgr2=0x7feff80ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:02.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:02.072+0000 7ff00078c700 1 -- 172.21.15.40:0/1103413573 shutdown_connections 2024-06-06T03:35:02.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:02.072+0000 7ff00078c700 1 -- 172.21.15.40:0/1103413573 wait complete. 2024-06-06T03:35:02.072 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 13 2024-06-06T03:35:02.761 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:02 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:02.761 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:02 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:02.761 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:02 smithi040 ceph-mon[32220]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:02.761 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:02 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:02.761 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:02 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:02.761 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:02 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1103413573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:35:02.793 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-06T03:35:03.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:02 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:03.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:02 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:03.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:02 smithi131 ceph-mon[36687]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:03.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:02 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:03.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:02 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:03.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:02 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1103413573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:35:03.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:02 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:03.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:02 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:03.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:02 smithi104 ceph-mon[36569]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:03.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:02 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:03.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:02 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:03.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:02 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1103413573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:35:03.181 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:03 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:04.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:03 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:04.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:03 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:04.873 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:04 smithi040 ceph-mon[32220]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:04.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.874+0000 7fa8e8637700 1 -- 172.21.15.40:0/2698951848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e0103120 msgr2=0x7fa8e0107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:04.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.874+0000 7fa8e8637700 1 --2- 172.21.15.40:0/2698951848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e0103120 0x7fa8e0107130 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00099c0 tx=0x7fa8d0009cd0 comp rx=0 tx=0).stop 2024-06-06T03:35:04.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.874+0000 7fa8e8637700 1 -- 172.21.15.40:0/2698951848 shutdown_connections 2024-06-06T03:35:04.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.874+0000 7fa8e8637700 1 --2- 172.21.15.40:0/2698951848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e0102770 0x7fa8e0102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:04.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.874+0000 7fa8e8637700 1 --2- 172.21.15.40:0/2698951848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e0107860 0x7fa8e010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:04.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.874+0000 7fa8e8637700 1 --2- 172.21.15.40:0/2698951848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e0103120 0x7fa8e0107130 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:04.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.874+0000 7fa8e8637700 1 -- 172.21.15.40:0/2698951848 >> 172.21.15.40:0/2698951848 conn(0x7fa8e0079600 msgr2=0x7fa8e0079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:04.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.874+0000 7fa8e8637700 1 -- 172.21.15.40:0/2698951848 shutdown_connections 2024-06-06T03:35:04.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.874+0000 7fa8e8637700 1 -- 172.21.15.40:0/2698951848 wait complete. 2024-06-06T03:35:04.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e8637700 1 Processor -- start 2024-06-06T03:35:04.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e8637700 1 -- start start 2024-06-06T03:35:04.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e8637700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e0102770 0x7fa8e019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:04.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e8637700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e0103120 0x7fa8e019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:04.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e8637700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e0107860 0x7fa8e01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:04.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e8637700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e0112840 con 0x7fa8e0107860 2024-06-06T03:35:04.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e8637700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e01126c0 con 0x7fa8e0103120 2024-06-06T03:35:04.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e8637700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e01129c0 con 0x7fa8e0102770 2024-06-06T03:35:04.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e63d3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e0102770 0x7fa8e019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:04.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e6bd4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e0107860 0x7fa8e01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:04.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e63d3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e0102770 0x7fa8e019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.40:52634/0 (socket says 172.21.15.40:52634) 2024-06-06T03:35:04.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e6bd4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e0107860 0x7fa8e01989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34684/0 (socket says 172.21.15.40:34684) 2024-06-06T03:35:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e63d3700 1 -- 172.21.15.40:0/2546292194 learned_addr learned my addr 172.21.15.40:0/2546292194 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e5bd2700 1 --2- 172.21.15.40:0/2546292194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e0103120 0x7fa8e019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e6bd4700 1 -- 172.21.15.40:0/2546292194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e0102770 msgr2=0x7fa8e019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e6bd4700 1 --2- 172.21.15.40:0/2546292194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e0102770 0x7fa8e019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e6bd4700 1 -- 172.21.15.40:0/2546292194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e0103120 msgr2=0x7fa8e019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e6bd4700 1 --2- 172.21.15.40:0/2546292194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e0103120 0x7fa8e019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.875+0000 7fa8e6bd4700 1 -- 172.21.15.40:0/2546292194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e0199140 con 0x7fa8e0107860 2024-06-06T03:35:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.876+0000 7fa8e63d3700 1 --2- 172.21.15.40:0/2546292194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e0102770 0x7fa8e019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:35:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.876+0000 7fa8e6bd4700 1 --2- 172.21.15.40:0/2546292194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e0107860 0x7fa8e01989e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800be20 tx=0x7fa8d8009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.876+0000 7fa8d77fe700 1 -- 172.21.15.40:0/2546292194 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8019070 con 0x7fa8e0107860 2024-06-06T03:35:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.876+0000 7fa8d77fe700 1 -- 172.21.15.40:0/2546292194 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d800cb90 con 0x7fa8e0107860 2024-06-06T03:35:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.876+0000 7fa8e8637700 1 -- 172.21.15.40:0/2546292194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e0199430 con 0x7fa8e0107860 2024-06-06T03:35:04.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.876+0000 7fa8d77fe700 1 -- 172.21.15.40:0/2546292194 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d80125a0 con 0x7fa8e0107860 2024-06-06T03:35:04.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.876+0000 7fa8e8637700 1 -- 172.21.15.40:0/2546292194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e01a56f0 con 0x7fa8e0107860 2024-06-06T03:35:04.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.878+0000 7fa8e8637700 1 -- 172.21.15.40:0/2546292194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e0069180 con 0x7fa8e0107860 2024-06-06T03:35:04.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.878+0000 7fa8d77fe700 1 -- 172.21.15.40:0/2546292194 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa8d801c070 con 0x7fa8e0107860 2024-06-06T03:35:04.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.879+0000 7fa8d77fe700 1 --2- 172.21.15.40:0/2546292194 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa8cc0a2ca0 0x7fa8cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:04.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.879+0000 7fa8d77fe700 1 -- 172.21.15.40:0/2546292194 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa8d80ce0c0 con 0x7fa8e0107860 2024-06-06T03:35:04.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.879+0000 7fa8e63d3700 1 --2- 172.21.15.40:0/2546292194 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa8cc0a2ca0 0x7fa8cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:04.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.880+0000 7fa8e63d3700 1 --2- 172.21.15.40:0/2546292194 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa8cc0a2ca0 0x7fa8cc0a5160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0199b20 tx=0x7fa8dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:04.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:04.884+0000 7fa8d77fe700 1 -- 172.21.15.40:0/2546292194 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa8d809b4c0 con 0x7fa8e0107860 2024-06-06T03:35:05.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:04 smithi131 ceph-mon[36687]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:05.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:04 smithi104 ceph-mon[36569]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:05.127+0000 7fa8e8637700 1 -- 172.21.15.40:0/2546292194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa8e01a5980 con 0x7fa8e0107860 2024-06-06T03:35:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:05.127+0000 7fa8d77fe700 1 -- 172.21.15.40:0/2546292194 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7fa8d809adc0 con 0x7fa8e0107860 2024-06-06T03:35:05.126 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:35:05.126 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-06-06T03:35:05.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:35:05.126 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-06-06T03:35:05.126 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-06-06T03:35:05.126 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:35:05.126 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-06-06T03:35:05.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:05.129+0000 7fa8e8637700 1 -- 172.21.15.40:0/2546292194 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa8cc0a2ca0 msgr2=0x7fa8cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:05.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:05.129+0000 7fa8e8637700 1 --2- 172.21.15.40:0/2546292194 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa8cc0a2ca0 0x7fa8cc0a5160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0199b20 tx=0x7fa8dc005f90 comp rx=0 tx=0).stop 2024-06-06T03:35:05.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:05.129+0000 7fa8e8637700 1 -- 172.21.15.40:0/2546292194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e0107860 msgr2=0x7fa8e01989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:05.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:05.129+0000 7fa8e8637700 1 --2- 172.21.15.40:0/2546292194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e0107860 0x7fa8e01989e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800be20 tx=0x7fa8d8009ed0 comp rx=0 tx=0).stop 2024-06-06T03:35:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:05.129+0000 7fa8e8637700 1 -- 172.21.15.40:0/2546292194 shutdown_connections 2024-06-06T03:35:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:05.129+0000 7fa8e8637700 1 --2- 172.21.15.40:0/2546292194 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa8cc0a2ca0 0x7fa8cc0a5160 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:05.129+0000 7fa8e8637700 1 --2- 172.21.15.40:0/2546292194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e0102770 0x7fa8e019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:05.129+0000 7fa8e8637700 1 --2- 172.21.15.40:0/2546292194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e0107860 0x7fa8e01989e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:05.129+0000 7fa8e8637700 1 --2- 172.21.15.40:0/2546292194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e0103120 0x7fa8e019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:05.129+0000 7fa8e8637700 1 -- 172.21.15.40:0/2546292194 >> 172.21.15.40:0/2546292194 conn(0x7fa8e0079600 msgr2=0x7fa8e0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:05.130+0000 7fa8e8637700 1 -- 172.21.15.40:0/2546292194 shutdown_connections 2024-06-06T03:35:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:05.130+0000 7fa8e8637700 1 -- 172.21.15.40:0/2546292194 wait complete. 2024-06-06T03:35:05.864 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-06T03:35:05.864 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-06T03:35:06.113 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:06.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:06 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:06.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:06 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:06.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:06 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2546292194' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T03:35:06.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:06 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:35:06.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:06 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:35:06.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:06 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:06.364 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:06 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:06.364 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:06 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:06.364 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:06 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2546292194' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T03:35:06.364 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:06 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:35:06.364 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:06 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:35:06.364 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:06 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:06.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:06 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:06.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:06 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:06.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:06 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2546292194' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T03:35:06.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:06 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:35:06.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:06 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:35:06.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:06 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:07.266 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:07 smithi040 ceph-mon[32220]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:07.267 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:07 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:35:07.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:07 smithi131 ceph-mon[36687]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:07.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:07 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:35:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:07 smithi104 ceph-mon[36569]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:07 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:35:07.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.861+0000 7f39cf649700 1 -- 172.21.15.40:0/1889948660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c81084f0 msgr2=0x7f39c81088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:07.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.861+0000 7f39cf649700 1 --2- 172.21.15.40:0/1889948660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c81084f0 0x7f39c81088d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f39b800b0a0 tx=0x7f39b800b3b0 comp rx=0 tx=0).stop 2024-06-06T03:35:07.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.861+0000 7f39cf649700 1 -- 172.21.15.40:0/1889948660 shutdown_connections 2024-06-06T03:35:07.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.861+0000 7f39cf649700 1 --2- 172.21.15.40:0/1889948660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39c806b3c0 0x7f39c810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:07.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.861+0000 7f39cf649700 1 --2- 172.21.15.40:0/1889948660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c81084f0 0x7f39c81088d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:07.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.861+0000 7f39cf649700 1 --2- 172.21.15.40:0/1889948660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39c806aa00 0x7f39c806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:07.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.861+0000 7f39cf649700 1 -- 172.21.15.40:0/1889948660 >> 172.21.15.40:0/1889948660 conn(0x7f39c8078300 msgr2=0x7f39c8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:07.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.861+0000 7f39cf649700 1 -- 172.21.15.40:0/1889948660 shutdown_connections 2024-06-06T03:35:07.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.861+0000 7f39cf649700 1 -- 172.21.15.40:0/1889948660 wait complete. 2024-06-06T03:35:07.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.862+0000 7f39cf649700 1 Processor -- start 2024-06-06T03:35:07.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.862+0000 7f39cf649700 1 -- start start 2024-06-06T03:35:07.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39cf649700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39c806aa00 0x7f39c819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:07.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39cf649700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c806b3c0 0x7f39c819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:07.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39cf649700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39c81084f0 0x7f39c819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:07.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39cf649700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8114950 con 0x7f39c806b3c0 2024-06-06T03:35:07.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39cf649700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81147d0 con 0x7f39c81084f0 2024-06-06T03:35:07.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39cf649700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8114ad0 con 0x7f39c806aa00 2024-06-06T03:35:07.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39ccbe4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c806b3c0 0x7f39c819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:07.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39ccbe4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c806b3c0 0x7f39c819b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34704/0 (socket says 172.21.15.40:34704) 2024-06-06T03:35:07.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39ccbe4700 1 -- 172.21.15.40:0/3468869174 learned_addr learned my addr 172.21.15.40:0/3468869174 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:07.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39cdbe6700 1 --2- 172.21.15.40:0/3468869174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39c81084f0 0x7f39c819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:07.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39cd3e5700 1 --2- 172.21.15.40:0/3468869174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39c806aa00 0x7f39c819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:07.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39ccbe4700 1 -- 172.21.15.40:0/3468869174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39c806aa00 msgr2=0x7f39c819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:07.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39ccbe4700 1 --2- 172.21.15.40:0/3468869174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39c806aa00 0x7f39c819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:07.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39ccbe4700 1 -- 172.21.15.40:0/3468869174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39c81084f0 msgr2=0x7f39c819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:07.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39ccbe4700 1 --2- 172.21.15.40:0/3468869174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39c81084f0 0x7f39c819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:07.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39ccbe4700 1 -- 172.21.15.40:0/3468869174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c819fd70 con 0x7f39c806b3c0 2024-06-06T03:35:07.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39ccbe4700 1 --2- 172.21.15.40:0/3468869174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c806b3c0 0x7f39c819b2a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00c920 tx=0x7f39bc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:07.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39c67fc700 1 -- 172.21.15.40:0/3468869174 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc007b00 con 0x7f39c806b3c0 2024-06-06T03:35:07.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39cf649700 1 -- 172.21.15.40:0/3468869174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c81a0060 con 0x7f39c806b3c0 2024-06-06T03:35:07.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39c67fc700 1 -- 172.21.15.40:0/3468869174 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39bc004d10 con 0x7f39c806b3c0 2024-06-06T03:35:07.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39c67fc700 1 -- 172.21.15.40:0/3468869174 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc005630 con 0x7f39c806b3c0 2024-06-06T03:35:07.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.863+0000 7f39cf649700 1 -- 172.21.15.40:0/3468869174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c81a78e0 con 0x7f39c806b3c0 2024-06-06T03:35:07.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.864+0000 7f39cf649700 1 -- 172.21.15.40:0/3468869174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c810a9d0 con 0x7f39c806b3c0 2024-06-06T03:35:07.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.865+0000 7f39c67fc700 1 -- 172.21.15.40:0/3468869174 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f39bc004750 con 0x7f39c806b3c0 2024-06-06T03:35:07.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.866+0000 7f39c67fc700 1 --2- 172.21.15.40:0/3468869174 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f39b00a2c20 0x7f39b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:07.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.866+0000 7f39c67fc700 1 -- 172.21.15.40:0/3468869174 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f39bc0ce730 con 0x7f39c806b3c0 2024-06-06T03:35:07.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.866+0000 7f39cd3e5700 1 --2- 172.21.15.40:0/3468869174 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f39b00a2c20 0x7f39b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:07.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.867+0000 7f39cd3e5700 1 --2- 172.21.15.40:0/3468869174 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f39b00a2c20 0x7f39b00a50e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f39b8006130 tx=0x7f39b8006080 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:07.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:07.871+0000 7f39c67fc700 1 -- 172.21.15.40:0/3468869174 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f39bc014030 con 0x7f39c806b3c0 2024-06-06T03:35:08.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:08.083+0000 7f39cf649700 1 -- 172.21.15.40:0/3468869174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f39c8069180 con 0x7f39c806b3c0 2024-06-06T03:35:08.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:08.805+0000 7f39c67fc700 1 -- 172.21.15.40:0/3468869174 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f39bc09bb30 con 0x7f39c806b3c0 2024-06-06T03:35:08.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:08.808+0000 7f39cf649700 1 -- 172.21.15.40:0/3468869174 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f39b00a2c20 msgr2=0x7f39b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:08.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:08.808+0000 7f39cf649700 1 --2- 172.21.15.40:0/3468869174 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f39b00a2c20 0x7f39b00a50e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f39b8006130 tx=0x7f39b8006080 comp rx=0 tx=0).stop 2024-06-06T03:35:08.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:08.808+0000 7f39cf649700 1 -- 172.21.15.40:0/3468869174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c806b3c0 msgr2=0x7f39c819b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:08.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:08.808+0000 7f39cf649700 1 --2- 172.21.15.40:0/3468869174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c806b3c0 0x7f39c819b2a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00c920 tx=0x7f39bc00cc30 comp rx=0 tx=0).stop 2024-06-06T03:35:08.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:08.808+0000 7f39cf649700 1 -- 172.21.15.40:0/3468869174 shutdown_connections 2024-06-06T03:35:08.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:08.808+0000 7f39cf649700 1 --2- 172.21.15.40:0/3468869174 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f39b00a2c20 0x7f39b00a50e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:08.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:08.808+0000 7f39cf649700 1 --2- 172.21.15.40:0/3468869174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39c806aa00 0x7f39c819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:08.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:08.808+0000 7f39cf649700 1 --2- 172.21.15.40:0/3468869174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39c81084f0 0x7f39c819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:08.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:08.808+0000 7f39cf649700 1 --2- 172.21.15.40:0/3468869174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c806b3c0 0x7f39c819b2a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:08.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:08.808+0000 7f39cf649700 1 -- 172.21.15.40:0/3468869174 >> 172.21.15.40:0/3468869174 conn(0x7f39c8078300 msgr2=0x7f39c80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:08.809+0000 7f39cf649700 1 -- 172.21.15.40:0/3468869174 shutdown_connections 2024-06-06T03:35:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:08.809+0000 7f39cf649700 1 -- 172.21.15.40:0/3468869174 wait complete. 2024-06-06T03:35:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-06-06T03:35:09.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:08 smithi040 ceph-mon[32220]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:09.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:09.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:09.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:09.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:08 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:09.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:08 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3468869174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T03:35:09.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:08 smithi104 ceph-mon[36569]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:09.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:09.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:09.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:09.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:08 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:09.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:08 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3468869174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T03:35:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:08 smithi131 ceph-mon[36687]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:08 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:08 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3468869174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T03:35:09.545 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-06-06T03:35:09.824 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:10.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:09 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3468869174' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T03:35:10.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:09 smithi040 ceph-mon[32220]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T03:35:10.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:09 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3468869174' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T03:35:10.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:09 smithi104 ceph-mon[36569]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T03:35:10.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:09 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3468869174' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T03:35:10.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:09 smithi131 ceph-mon[36687]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T03:35:11.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:10 smithi040 ceph-mon[32220]: pgmap v293: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:11.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:10 smithi040 ceph-mon[32220]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T03:35:11.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:10 smithi040 ceph-mon[32220]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T03:35:11.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:10 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:11.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:10 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:11.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:10 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:35:11.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:10 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:35:11.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:10 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:11.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:10 smithi104 ceph-mon[36569]: pgmap v293: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:11.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:10 smithi104 ceph-mon[36569]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T03:35:11.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:10 smithi104 ceph-mon[36569]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T03:35:11.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:10 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:11.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:10 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:11.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:10 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:35:11.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:10 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:35:11.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:10 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:10 smithi131 ceph-mon[36687]: pgmap v293: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:10 smithi131 ceph-mon[36687]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T03:35:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:10 smithi131 ceph-mon[36687]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T03:35:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:10 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:10 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:10 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:35:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:10 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:35:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:10 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:35:11.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.715+0000 7f8020206700 1 -- 172.21.15.40:0/2571875152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80181017f0 msgr2=0x7f801810f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:11.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.715+0000 7f8020206700 1 --2- 172.21.15.40:0/2571875152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80181017f0 0x7f801810f260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f801000b0a0 tx=0x7f801000b3b0 comp rx=0 tx=0).stop 2024-06-06T03:35:11.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.715+0000 7f8020206700 1 -- 172.21.15.40:0/2571875152 shutdown_connections 2024-06-06T03:35:11.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.715+0000 7f8020206700 1 --2- 172.21.15.40:0/2571875152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80181017f0 0x7f801810f260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:11.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.715+0000 7f8020206700 1 --2- 172.21.15.40:0/2571875152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8018107550 0x7f8018107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:11.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.715+0000 7f8020206700 1 --2- 172.21.15.40:0/2571875152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8018100e30 0x7f80181012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:11.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.715+0000 7f8020206700 1 -- 172.21.15.40:0/2571875152 >> 172.21.15.40:0/2571875152 conn(0x7f80180766b0 msgr2=0x7f8018076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:11.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.715+0000 7f8020206700 1 -- 172.21.15.40:0/2571875152 shutdown_connections 2024-06-06T03:35:11.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.715+0000 7f8020206700 1 -- 172.21.15.40:0/2571875152 wait complete. 2024-06-06T03:35:11.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.716+0000 7f8020206700 1 Processor -- start 2024-06-06T03:35:11.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.716+0000 7f8020206700 1 -- start start 2024-06-06T03:35:11.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.716+0000 7f8020206700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8018100e30 0x7f8018102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:11.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.716+0000 7f8020206700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80181017f0 0x7f8018102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:11.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.716+0000 7f8020206700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8018107550 0x7f8018102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.716+0000 7f8020206700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f80181127b0 con 0x7f8018107550 2024-06-06T03:35:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.716+0000 7f8020206700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8018112630 con 0x7f8018100e30 2024-06-06T03:35:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.716+0000 7f8020206700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8018112930 con 0x7f80181017f0 2024-06-06T03:35:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.716+0000 7f801e7a3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8018107550 0x7f8018102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.716+0000 7f801e7a3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8018107550 0x7f8018102a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54014/0 (socket says 172.21.15.40:54014) 2024-06-06T03:35:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.716+0000 7f801e7a3700 1 -- 172.21.15.40:0/3876414562 learned_addr learned my addr 172.21.15.40:0/3876414562 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.716+0000 7f801d7a1700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80181017f0 0x7f8018102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.716+0000 7f801dfa2700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8018100e30 0x7f8018102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.717+0000 7f801d7a1700 1 -- 172.21.15.40:0/3876414562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8018100e30 msgr2=0x7f8018102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.717+0000 7f801d7a1700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8018100e30 0x7f8018102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.717+0000 7f801d7a1700 1 -- 172.21.15.40:0/3876414562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8018107550 msgr2=0x7f8018102a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.717+0000 7f801d7a1700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8018107550 0x7f8018102a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.717+0000 7f801d7a1700 1 -- 172.21.15.40:0/3876414562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80181a70a0 con 0x7f80181017f0 2024-06-06T03:35:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.717+0000 7f801e7a3700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8018107550 0x7f8018102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:35:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.717+0000 7f801d7a1700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80181017f0 0x7f8018102550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f800800d750 tx=0x7f800800da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.717+0000 7f800effd700 1 -- 172.21.15.40:0/3876414562 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008016070 con 0x7f80181017f0 2024-06-06T03:35:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.717+0000 7f8020206700 1 -- 172.21.15.40:0/3876414562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80181a7390 con 0x7f80181017f0 2024-06-06T03:35:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.717+0000 7f800effd700 1 -- 172.21.15.40:0/3876414562 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8008005020 con 0x7f80181017f0 2024-06-06T03:35:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.717+0000 7f800effd700 1 -- 172.21.15.40:0/3876414562 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800800fb30 con 0x7f80181017f0 2024-06-06T03:35:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.718+0000 7f8020206700 1 -- 172.21.15.40:0/3876414562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80181a7880 con 0x7f80181017f0 2024-06-06T03:35:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.718+0000 7f8020206700 1 -- 172.21.15.40:0/3876414562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8018069180 con 0x7f80181017f0 2024-06-06T03:35:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.720+0000 7f800effd700 1 -- 172.21.15.40:0/3876414562 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f80080051d0 con 0x7f80181017f0 2024-06-06T03:35:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.720+0000 7f800effd700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f80040a2c20 0x7f80040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.721+0000 7f801dfa2700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f80040a2c20 0x7f80040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.721+0000 7f800effd700 1 -- 172.21.15.40:0/3876414562 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7f80080ce810 con 0x7f80181017f0 2024-06-06T03:35:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.721+0000 7f801dfa2700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f80040a2c20 0x7f80040a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8018077c30 tx=0x7f8014007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:11.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.724+0000 7f800effd700 1 -- 172.21.15.40:0/3876414562 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f800809ba00 con 0x7f80181017f0 2024-06-06T03:35:11.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:11.932+0000 7f8020206700 1 -- 172.21.15.40:0/3876414562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f801802dca0 con 0x7f80181017f0 2024-06-06T03:35:12.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:11 smithi040 ceph-mon[32220]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T03:35:12.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:11 smithi104 ceph-mon[36569]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T03:35:12.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:11 smithi131 ceph-mon[36687]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T03:35:12.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:12.831+0000 7f800effd700 1 -- 172.21.15.40:0/3876414562 <== mon.1 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7f800809b300 con 0x7f80181017f0 2024-06-06T03:35:12.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:12.835+0000 7f8020206700 1 -- 172.21.15.40:0/3876414562 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f80040a2c20 msgr2=0x7f80040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:12.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:12.835+0000 7f8020206700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f80040a2c20 0x7f80040a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8018077c30 tx=0x7f8014007400 comp rx=0 tx=0).stop 2024-06-06T03:35:12.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:12.835+0000 7f8020206700 1 -- 172.21.15.40:0/3876414562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80181017f0 msgr2=0x7f8018102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:12.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:12.835+0000 7f8020206700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80181017f0 0x7f8018102550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f800800d750 tx=0x7f800800da60 comp rx=0 tx=0).stop 2024-06-06T03:35:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:12.838+0000 7f8020206700 1 -- 172.21.15.40:0/3876414562 shutdown_connections 2024-06-06T03:35:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:12.838+0000 7f8020206700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f80040a2c20 0x7f80040a50e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:12.838+0000 7f8020206700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8018107550 0x7f8018102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:12.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:12.838+0000 7f8020206700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8018100e30 0x7f8018102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:12.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:12.838+0000 7f8020206700 1 --2- 172.21.15.40:0/3876414562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80181017f0 0x7f8018102550 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:12.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:12.838+0000 7f8020206700 1 -- 172.21.15.40:0/3876414562 >> 172.21.15.40:0/3876414562 conn(0x7f80180766b0 msgr2=0x7f80180ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:12.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:12.839+0000 7f8020206700 1 -- 172.21.15.40:0/3876414562 shutdown_connections 2024-06-06T03:35:12.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:12.839+0000 7f8020206700 1 -- 172.21.15.40:0/3876414562 wait complete. 2024-06-06T03:35:12.840 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-06-06T03:35:13.012 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:12 smithi040 ceph-mon[32220]: pgmap v296: 33 pgs: 16 active+clean, 1 creating+activating, 8 creating+peering, 8 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:13.012 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:12 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3876414562' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T03:35:13.012 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:12 smithi040 ceph-mon[32220]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T03:35:13.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:12 smithi104 ceph-mon[36569]: pgmap v296: 33 pgs: 16 active+clean, 1 creating+activating, 8 creating+peering, 8 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:13.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:12 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3876414562' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T03:35:13.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:12 smithi104 ceph-mon[36569]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T03:35:13.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:12 smithi131 ceph-mon[36687]: pgmap v296: 33 pgs: 16 active+clean, 1 creating+activating, 8 creating+peering, 8 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:13.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:12 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3876414562' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T03:35:13.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:12 smithi131 ceph-mon[36687]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T03:35:13.627 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-06T03:35:13.905 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:14.016 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:13 smithi040 ceph-mon[32220]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T03:35:14.017 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:13 smithi040 ceph-mon[32220]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T03:35:14.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:13 smithi131 ceph-mon[36687]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T03:35:14.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:13 smithi131 ceph-mon[36687]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T03:35:14.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:13 smithi104 ceph-mon[36569]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T03:35:14.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:13 smithi104 ceph-mon[36569]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T03:35:15.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:14 smithi040 ceph-mon[32220]: pgmap v298: 97 pgs: 24 active+clean, 1 creating+activating, 22 creating+peering, 50 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:15.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:14 smithi040 ceph-mon[32220]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T03:35:15.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:14 smithi040 ceph-mon[32220]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T03:35:15.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:14 smithi131 ceph-mon[36687]: pgmap v298: 97 pgs: 24 active+clean, 1 creating+activating, 22 creating+peering, 50 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:15.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:14 smithi131 ceph-mon[36687]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T03:35:15.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:14 smithi131 ceph-mon[36687]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T03:35:15.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:14 smithi104 ceph-mon[36569]: pgmap v298: 97 pgs: 24 active+clean, 1 creating+activating, 22 creating+peering, 50 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:15.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:14 smithi104 ceph-mon[36569]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T03:35:15.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:14 smithi104 ceph-mon[36569]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T03:35:15.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.696+0000 7f471f067700 1 -- 172.21.15.40:0/3128057564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4718101fd0 msgr2=0x7f4718105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:15.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.696+0000 7f471f067700 1 --2- 172.21.15.40:0/3128057564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4718101fd0 0x7f4718105fd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f47000099c0 tx=0x7f4700009cd0 comp rx=0 tx=0).stop 2024-06-06T03:35:15.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.696+0000 7f471f067700 1 -- 172.21.15.40:0/3128057564 shutdown_connections 2024-06-06T03:35:15.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.696+0000 7f471f067700 1 --2- 172.21.15.40:0/3128057564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47181066b0 0x7f471810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:15.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.696+0000 7f471f067700 1 --2- 172.21.15.40:0/3128057564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4718101620 0x7f4718101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:15.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.696+0000 7f471f067700 1 --2- 172.21.15.40:0/3128057564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4718101fd0 0x7f4718105fd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:15.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.696+0000 7f471f067700 1 -- 172.21.15.40:0/3128057564 >> 172.21.15.40:0/3128057564 conn(0x7f4718078340 msgr2=0x7f4718078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:15.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.696+0000 7f471f067700 1 -- 172.21.15.40:0/3128057564 shutdown_connections 2024-06-06T03:35:15.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.696+0000 7f471f067700 1 -- 172.21.15.40:0/3128057564 wait complete. 2024-06-06T03:35:15.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.697+0000 7f471f067700 1 Processor -- start 2024-06-06T03:35:15.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.697+0000 7f471f067700 1 -- start start 2024-06-06T03:35:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.697+0000 7f471f067700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4718101620 0x7f4718199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.697+0000 7f471f067700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4718101fd0 0x7f4718199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.697+0000 7f471f067700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47181066b0 0x7f47181a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.697+0000 7f471f067700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4718111690 con 0x7f47181066b0 2024-06-06T03:35:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.697+0000 7f471f067700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4718111510 con 0x7f4718101620 2024-06-06T03:35:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.697+0000 7f471f067700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4718111810 con 0x7f4718101fd0 2024-06-06T03:35:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.697+0000 7f471d604700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47181066b0 0x7f47181a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.697+0000 7f471d604700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47181066b0 0x7f47181a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54032/0 (socket says 172.21.15.40:54032) 2024-06-06T03:35:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.697+0000 7f471d604700 1 -- 172.21.15.40:0/2381879466 learned_addr learned my addr 172.21.15.40:0/2381879466 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.697+0000 7f471ce03700 1 --2- 172.21.15.40:0/2381879466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4718101620 0x7f4718199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.698+0000 7f4717fff700 1 --2- 172.21.15.40:0/2381879466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4718101fd0 0x7f4718199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.698+0000 7f471d604700 1 -- 172.21.15.40:0/2381879466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4718101fd0 msgr2=0x7f4718199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:15.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.698+0000 7f471d604700 1 --2- 172.21.15.40:0/2381879466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4718101fd0 0x7f4718199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:15.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.698+0000 7f471d604700 1 -- 172.21.15.40:0/2381879466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4718101620 msgr2=0x7f4718199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:15.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.698+0000 7f471d604700 1 --2- 172.21.15.40:0/2381879466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4718101620 0x7f4718199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:15.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.698+0000 7f471d604700 1 -- 172.21.15.40:0/2381879466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47181a5070 con 0x7f47181066b0 2024-06-06T03:35:15.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.698+0000 7f471d604700 1 --2- 172.21.15.40:0/2381879466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47181066b0 0x7f47181a4b30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f471000ada0 tx=0x7f471000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:15.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.698+0000 7f4715ffb700 1 -- 172.21.15.40:0/2381879466 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710017020 con 0x7f47181066b0 2024-06-06T03:35:15.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.698+0000 7f4715ffb700 1 -- 172.21.15.40:0/2381879466 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f471000cdb0 con 0x7f47181066b0 2024-06-06T03:35:15.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.698+0000 7f471f067700 1 -- 172.21.15.40:0/2381879466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47181a52a0 con 0x7f47181066b0 2024-06-06T03:35:15.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.698+0000 7f4715ffb700 1 -- 172.21.15.40:0/2381879466 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47100053e0 con 0x7f47181066b0 2024-06-06T03:35:15.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.698+0000 7f471f067700 1 -- 172.21.15.40:0/2381879466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47181a57b0 con 0x7f47181066b0 2024-06-06T03:35:15.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.700+0000 7f471f067700 1 -- 172.21.15.40:0/2381879466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4718069180 con 0x7f47181066b0 2024-06-06T03:35:15.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.701+0000 7f4715ffb700 1 -- 172.21.15.40:0/2381879466 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f471000c8d0 con 0x7f47181066b0 2024-06-06T03:35:15.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.702+0000 7f4715ffb700 1 --2- 172.21.15.40:0/2381879466 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f47040a2c20 0x7f47040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:15.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.702+0000 7f471ce03700 1 --2- 172.21.15.40:0/2381879466 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f47040a2c20 0x7f47040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:15.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.702+0000 7f4715ffb700 1 -- 172.21.15.40:0/2381879466 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7879+0+0 (secure 0 0 0) 0x7f47100ce4e0 con 0x7f47181066b0 2024-06-06T03:35:15.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.702+0000 7f471ce03700 1 --2- 172.21.15.40:0/2381879466 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f47040a2c20 0x7f47040a50e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4708005950 tx=0x7f470800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:15.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.707+0000 7f4715ffb700 1 -- 172.21.15.40:0/2381879466 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4710021020 con 0x7f47181066b0 2024-06-06T03:35:15.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:15.959+0000 7f471f067700 1 -- 172.21.15.40:0/2381879466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f471802ddc0 con 0x7f47181066b0 2024-06-06T03:35:16.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:16.857+0000 7f4715ffb700 1 -- 172.21.15.40:0/2381879466 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f471009b680 con 0x7f47181066b0 2024-06-06T03:35:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:16.860+0000 7f471f067700 1 -- 172.21.15.40:0/2381879466 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f47040a2c20 msgr2=0x7f47040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:16.860+0000 7f471f067700 1 --2- 172.21.15.40:0/2381879466 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f47040a2c20 0x7f47040a50e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4708005950 tx=0x7f470800a300 comp rx=0 tx=0).stop 2024-06-06T03:35:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:16.860+0000 7f471f067700 1 -- 172.21.15.40:0/2381879466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47181066b0 msgr2=0x7f47181a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:16.860+0000 7f471f067700 1 --2- 172.21.15.40:0/2381879466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47181066b0 0x7f47181a4b30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f471000ada0 tx=0x7f471000c330 comp rx=0 tx=0).stop 2024-06-06T03:35:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:16.860+0000 7f471f067700 1 -- 172.21.15.40:0/2381879466 shutdown_connections 2024-06-06T03:35:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:16.860+0000 7f471f067700 1 --2- 172.21.15.40:0/2381879466 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f47040a2c20 0x7f47040a50e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:16.860+0000 7f471f067700 1 --2- 172.21.15.40:0/2381879466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47181066b0 0x7f47181a4b30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:16.860+0000 7f471f067700 1 --2- 172.21.15.40:0/2381879466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4718101620 0x7f4718199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:16.860+0000 7f471f067700 1 --2- 172.21.15.40:0/2381879466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4718101fd0 0x7f4718199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:16.860+0000 7f471f067700 1 -- 172.21.15.40:0/2381879466 >> 172.21.15.40:0/2381879466 conn(0x7f4718078340 msgr2=0x7f47180ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:16.860+0000 7f471f067700 1 -- 172.21.15.40:0/2381879466 shutdown_connections 2024-06-06T03:35:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:16.860+0000 7f471f067700 1 -- 172.21.15.40:0/2381879466 wait complete. 2024-06-06T03:35:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 2 and data pool 3 2024-06-06T03:35:17.191 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:16 smithi040 ceph-mon[32220]: pgmap v301: 97 pgs: 60 active+clean, 14 creating+peering, 23 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:17.191 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:16 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2381879466' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T03:35:17.191 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:16 smithi040 ceph-mon[32220]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-06T03:35:17.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:16 smithi131 ceph-mon[36687]: pgmap v301: 97 pgs: 60 active+clean, 14 creating+peering, 23 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:17.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:16 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2381879466' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T03:35:17.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:16 smithi131 ceph-mon[36687]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-06T03:35:17.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:16 smithi104 ceph-mon[36569]: pgmap v301: 97 pgs: 60 active+clean, 14 creating+peering, 23 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:17.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:16 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2381879466' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T03:35:17.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:16 smithi104 ceph-mon[36569]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-06T03:35:17.645 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T03:35:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:18.027 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:17 smithi040 ceph-mon[32220]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T03:35:18.028 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:17 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2381879466' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T03:35:18.028 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:17 smithi040 ceph-mon[32220]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-06T03:35:18.028 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:17 smithi040 ceph-mon[32220]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-06T03:35:18.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:17 smithi131 ceph-mon[36687]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T03:35:18.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:17 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2381879466' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T03:35:18.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:17 smithi131 ceph-mon[36687]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-06T03:35:18.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:17 smithi131 ceph-mon[36687]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-06T03:35:18.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:17 smithi104 ceph-mon[36569]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T03:35:18.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:17 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2381879466' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T03:35:18.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:17 smithi104 ceph-mon[36569]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-06T03:35:18.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:17 smithi104 ceph-mon[36569]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-06T03:35:19.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:18 smithi040 ceph-mon[32220]: pgmap v303: 97 pgs: 83 active+clean, 14 creating+peering; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:19.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:18 smithi040 ceph-mon[32220]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T03:35:19.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:18 smithi040 ceph-mon[32220]: Cluster is now healthy 2024-06-06T03:35:19.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:18 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] up:active 2024-06-06T03:35:19.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:18 smithi040 ceph-mon[32220]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T03:35:19.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:18 smithi040 ceph-mon[32220]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T03:35:19.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:18 smithi131 ceph-mon[36687]: pgmap v303: 97 pgs: 83 active+clean, 14 creating+peering; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:19.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:18 smithi131 ceph-mon[36687]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T03:35:19.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:18 smithi131 ceph-mon[36687]: Cluster is now healthy 2024-06-06T03:35:19.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:18 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] up:active 2024-06-06T03:35:19.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:18 smithi131 ceph-mon[36687]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T03:35:19.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:18 smithi131 ceph-mon[36687]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T03:35:19.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:18 smithi104 ceph-mon[36569]: pgmap v303: 97 pgs: 83 active+clean, 14 creating+peering; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:35:19.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:18 smithi104 ceph-mon[36569]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T03:35:19.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:18 smithi104 ceph-mon[36569]: Cluster is now healthy 2024-06-06T03:35:19.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:18 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] up:active 2024-06-06T03:35:19.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:18 smithi104 ceph-mon[36569]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T03:35:19.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:18 smithi104 ceph-mon[36569]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T03:35:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.649+0000 7f22c2cae700 1 -- 172.21.15.40:0/1937639839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc06aac0 msgr2=0x7f22bc10ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.649+0000 7f22c2cae700 1 --2- 172.21.15.40:0/1937639839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc06aac0 0x7f22bc10ea00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009990 tx=0x7f22ac009ca0 comp rx=0 tx=0).stop 2024-06-06T03:35:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.649+0000 7f22c2cae700 1 -- 172.21.15.40:0/1937639839 shutdown_connections 2024-06-06T03:35:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.649+0000 7f22c2cae700 1 --2- 172.21.15.40:0/1937639839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22bc10ef40 0x7f22bc111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.649+0000 7f22c2cae700 1 --2- 172.21.15.40:0/1937639839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc06a1a0 0x7f22bc06a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.649+0000 7f22c2cae700 1 --2- 172.21.15.40:0/1937639839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc06aac0 0x7f22bc10ea00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.649+0000 7f22c2cae700 1 -- 172.21.15.40:0/1937639839 >> 172.21.15.40:0/1937639839 conn(0x7f22bc078340 msgr2=0x7f22bc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:19.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.650+0000 7f22c2cae700 1 -- 172.21.15.40:0/1937639839 shutdown_connections 2024-06-06T03:35:19.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.650+0000 7f22c2cae700 1 -- 172.21.15.40:0/1937639839 wait complete. 2024-06-06T03:35:19.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.650+0000 7f22c2cae700 1 Processor -- start 2024-06-06T03:35:19.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c2cae700 1 -- start start 2024-06-06T03:35:19.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c2cae700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc06a1a0 0x7f22bc19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:19.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c2cae700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22bc06aac0 0x7f22bc19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:19.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c2cae700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc10ef40 0x7f22bc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:19.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c2cae700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc114a10 con 0x7f22bc10ef40 2024-06-06T03:35:19.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c2cae700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc114890 con 0x7f22bc06a1a0 2024-06-06T03:35:19.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c2cae700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc114b90 con 0x7f22bc06aac0 2024-06-06T03:35:19.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c124b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc10ef40 0x7f22bc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:19.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c124b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc10ef40 0x7f22bc19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46064/0 (socket says 172.21.15.40:46064) 2024-06-06T03:35:19.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c124b700 1 -- 172.21.15.40:0/2196141604 learned_addr learned my addr 172.21.15.40:0/2196141604 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:19.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22bbfff700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22bc06aac0 0x7f22bc19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:19.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c0a4a700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc06a1a0 0x7f22bc19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:19.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c124b700 1 -- 172.21.15.40:0/2196141604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22bc06aac0 msgr2=0x7f22bc19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:19.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c124b700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22bc06aac0 0x7f22bc19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:19.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c124b700 1 -- 172.21.15.40:0/2196141604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc06a1a0 msgr2=0x7f22bc19af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:19.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c124b700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc06a1a0 0x7f22bc19af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:19.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c124b700 1 -- 172.21.15.40:0/2196141604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc19ff10 con 0x7f22bc10ef40 2024-06-06T03:35:19.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c0a4a700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc06a1a0 0x7f22bc19af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:35:19.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.651+0000 7f22c124b700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc10ef40 0x7f22bc19f7d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22b400b9e0 tx=0x7f22b400bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:19.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.652+0000 7f22b9ffb700 1 -- 172.21.15.40:0/2196141604 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b4019070 con 0x7f22bc10ef40 2024-06-06T03:35:19.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.652+0000 7f22b9ffb700 1 -- 172.21.15.40:0/2196141604 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22b400ce10 con 0x7f22bc10ef40 2024-06-06T03:35:19.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.652+0000 7f22c2cae700 1 -- 172.21.15.40:0/2196141604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22bc1a0200 con 0x7f22bc10ef40 2024-06-06T03:35:19.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.652+0000 7f22b9ffb700 1 -- 172.21.15.40:0/2196141604 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b4012360 con 0x7f22bc10ef40 2024-06-06T03:35:19.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.652+0000 7f22c2cae700 1 -- 172.21.15.40:0/2196141604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22bc1a7a80 con 0x7f22bc10ef40 2024-06-06T03:35:19.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.653+0000 7f22c2cae700 1 -- 172.21.15.40:0/2196141604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22bc069180 con 0x7f22bc10ef40 2024-06-06T03:35:19.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.654+0000 7f22b9ffb700 1 -- 172.21.15.40:0/2196141604 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f22b401c070 con 0x7f22bc10ef40 2024-06-06T03:35:19.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.655+0000 7f22b9ffb700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f22a80a2c20 0x7f22a80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:19.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.655+0000 7f22c0a4a700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f22a80a2c20 0x7f22a80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:19.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.655+0000 7f22b9ffb700 1 -- 172.21.15.40:0/2196141604 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f22b40ce360 con 0x7f22bc10ef40 2024-06-06T03:35:19.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.655+0000 7f22c0a4a700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f22a80a2c20 0x7f22a80a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005950 tx=0x7f22a40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:19.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.659+0000 7f22b9ffb700 1 -- 172.21.15.40:0/2196141604 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f22b409b370 con 0x7f22bc10ef40 2024-06-06T03:35:19.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.867+0000 7f22c2cae700 1 -- 172.21.15.40:0/2196141604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22bc19c260 con 0x7f22bc10ef40 2024-06-06T03:35:19.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.868+0000 7f22b9ffb700 1 -- 172.21.15.40:0/2196141604 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22242 (secure 0 0 0) 0x7f22b409ac70 con 0x7f22bc10ef40 2024-06-06T03:35:19.868 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:35:19.869 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":88,"fsid":"77c25d68-23b4-11ef-bc9b-c7b262605968","created":"2024-06-06T03:26:53.165489+0000","modified":"2024-06-06T03:35:15.961722+0000","last_up_change":"2024-06-06T03:34:07.114448+0000","last_in_change":"2024-06-06T03:33:46.152948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T03:30:14.102785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T03:35:08.521025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T03:35:12.369035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63c0a320-a2f2-4b84-87e7-68501d335285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6803","nonce":3453526793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6805","nonce":3453526793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6809","nonce":3453526793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6807","nonce":3453526793}]},"public_addr":"172.21.15.40:6803/3453526793","cluster_addr":"172.21.15.40:6805/3453526793","heartbeat_back_addr":"172.21.15.40:6809/3453526793","heartbeat_front_addr":"172.21.15.40:6807/3453526793","state":["exists","up"]},{"osd":1,"uuid":"70a31534-06a1-4ffa-a54d-577d45bccf98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6801","nonce":405854252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6803","nonce":405854252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6807","nonce":405854252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6805","nonce":405854252}]},"public_addr":"172.21.15.104:6801/405854252","cluster_addr":"172.21.15.104:6803/405854252","heartbeat_back_addr":"172.21.15.104:6807/405854252","heartbeat_front_addr":"172.21.15.104:6805/405854252","state":["exists","up"]},{"osd":2,"uuid":"f5a89f43-e533-45f0-a777-bd3e9bfa8072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6801","nonce":3919221141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6803","nonce":3919221141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6807","nonce":3919221141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6805","nonce":3919221141}]},"public_addr":"172.21.15.131:6801/3919221141","cluster_addr":"172.21.15.131:6803/3919221141","heartbeat_back_addr":"172.21.15.131:6807/3919221141","heartbeat_front_addr":"172.21.15.131:6805/3919221141","state":["exists","up"]},{"osd":3,"uuid":"4df19390-6378-4346-a2be-29bce1dd2380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6811","nonce":2008541522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6813","nonce":2008541522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6817","nonce":2008541522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6815","nonce":2008541522}]},"public_addr":"172.21.15.40:6811/2008541522","cluster_addr":"172.21.15.40:6813/2008541522","heartbeat_back_addr":"172.21.15.40:6817/2008541522","heartbeat_front_addr":"172.21.15.40:6815/2008541522","state":["exists","up"]},{"osd":4,"uuid":"508de408-1cb1-4423-8199-db6bde9034f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6809","nonce":1153394847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6811","nonce":1153394847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6815","nonce":1153394847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6813","nonce":1153394847}]},"public_addr":"172.21.15.104:6809/1153394847","cluster_addr":"172.21.15.104:6811/1153394847","heartbeat_back_addr":"172.21.15.104:6815/1153394847","heartbeat_front_addr":"172.21.15.104:6813/1153394847","state":["exists","up"]},{"osd":5,"uuid":"a8c0cec9-a8ad-4a5d-800b-8b891ca6d836","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6809","nonce":1051933303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6811","nonce":1051933303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6815","nonce":1051933303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6813","nonce":1051933303}]},"public_addr":"172.21.15.131:6809/1051933303","cluster_addr":"172.21.15.131:6811/1051933303","heartbeat_back_addr":"172.21.15.131:6815/1051933303","heartbeat_front_addr":"172.21.15.131:6813/1051933303","state":["exists","up"]},{"osd":6,"uuid":"5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6819","nonce":2670922011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6821","nonce":2670922011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6825","nonce":2670922011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6823","nonce":2670922011}]},"public_addr":"172.21.15.40:6819/2670922011","cluster_addr":"172.21.15.40:6821/2670922011","heartbeat_back_addr":"172.21.15.40:6825/2670922011","heartbeat_front_addr":"172.21.15.40:6823/2670922011","state":["exists","up"]},{"osd":7,"uuid":"bfc536bb-6ae4-459e-afd9-8197c7420bd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6817","nonce":2353571051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6819","nonce":2353571051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6823","nonce":2353571051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6821","nonce":2353571051}]},"public_addr":"172.21.15.104:6817/2353571051","cluster_addr":"172.21.15.104:6819/2353571051","heartbeat_back_addr":"172.21.15.104:6823/2353571051","heartbeat_front_addr":"172.21.15.104:6821/2353571051","state":["exists","up"]},{"osd":8,"uuid":"1fe214f2-5ea3-4fc5-83bf-d9bccafe1529","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6817","nonce":1364533794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6819","nonce":1364533794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6823","nonce":1364533794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6821","nonce":1364533794}]},"public_addr":"172.21.15.131:6817/1364533794","cluster_addr":"172.21.15.131:6819/1364533794","heartbeat_back_addr":"172.21.15.131:6823/1364533794","heartbeat_front_addr":"172.21.15.131:6821/1364533794","state":["exists","up"]},{"osd":9,"uuid":"4b2b3555-6870-4ead-be50-b635799265a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6827","nonce":2604133190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6829","nonce":2604133190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6833","nonce":2604133190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6831","nonce":2604133190}]},"public_addr":"172.21.15.40:6827/2604133190","cluster_addr":"172.21.15.40:6829/2604133190","heartbeat_back_addr":"172.21.15.40:6833/2604133190","heartbeat_front_addr":"172.21.15.40:6831/2604133190","state":["exists","up"]},{"osd":10,"uuid":"70049e24-94ce-458e-97e3-dbcf9de05444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6825","nonce":2231164832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6827","nonce":2231164832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6831","nonce":2231164832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6829","nonce":2231164832}]},"public_addr":"172.21.15.104:6825/2231164832","cluster_addr":"172.21.15.104:6827/2231164832","heartbeat_back_addr":"172.21.15.104:6831/2231164832","heartbeat_front_addr":"172.21.15.104:6829/2231164832","state":["exists","up"]},{"osd":11,"uuid":"80447548-1fbe-4892-ac13-eb4bb6acf10d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6825","nonce":4128185670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6827","nonce":4128185670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6831","nonce":4128185670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6829","nonce":4128185670}]},"public_addr":"172.21.15.131:6825/4128185670","cluster_addr":"172.21.15.131:6827/4128185670","heartbeat_back_addr":"172.21.15.131:6831/4128185670","heartbeat_front_addr":"172.21.15.131:6829/4128185670","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:23.289784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:47.271677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:10.866792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:36.781924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:01.995848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:26.904606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:52.997438+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:19.221567+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:44.543977+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:11.948545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:38.402943+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:34:04.712200+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/23942578":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2234980855":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6801/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6800/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2301695357":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/1676880727":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6801/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6800/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/1597665640":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/513375601":"2024-06-07T03:27:19.799499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T03:35:19.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.871+0000 7f22c2cae700 1 -- 172.21.15.40:0/2196141604 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f22a80a2c20 msgr2=0x7f22a80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.871+0000 7f22c2cae700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f22a80a2c20 0x7f22a80a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005950 tx=0x7f22a40058e0 comp rx=0 tx=0).stop 2024-06-06T03:35:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.871+0000 7f22c2cae700 1 -- 172.21.15.40:0/2196141604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc10ef40 msgr2=0x7f22bc19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.871+0000 7f22c2cae700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc10ef40 0x7f22bc19f7d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22b400b9e0 tx=0x7f22b400bcf0 comp rx=0 tx=0).stop 2024-06-06T03:35:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.871+0000 7f22c2cae700 1 -- 172.21.15.40:0/2196141604 shutdown_connections 2024-06-06T03:35:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.871+0000 7f22c2cae700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f22a80a2c20 0x7f22a80a50e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.871+0000 7f22c2cae700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc10ef40 0x7f22bc19f7d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.871+0000 7f22c2cae700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22bc06a1a0 0x7f22bc19af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.871+0000 7f22c2cae700 1 --2- 172.21.15.40:0/2196141604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22bc06aac0 0x7f22bc19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.871+0000 7f22c2cae700 1 -- 172.21.15.40:0/2196141604 >> 172.21.15.40:0/2196141604 conn(0x7f22bc078340 msgr2=0x7f22bc1103f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.871+0000 7f22c2cae700 1 -- 172.21.15.40:0/2196141604 shutdown_connections 2024-06-06T03:35:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:19.871+0000 7f22c2cae700 1 -- 172.21.15.40:0/2196141604 wait complete. 2024-06-06T03:35:20.090 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:35:19 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b[66930]: 2024-06-06T03:35:19.614+0000 7f16ce78f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-06T03:35:20.571 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T03:35:20.821 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:21.196 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:20 smithi040 ceph-mon[32220]: pgmap v304: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-06-06T03:35:21.197 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:20 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2196141604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:35:21.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:20 smithi131 ceph-mon[36687]: pgmap v304: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-06-06T03:35:21.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:20 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2196141604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:35:21.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:20 smithi104 ceph-mon[36569]: pgmap v304: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-06-06T03:35:21.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:20 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2196141604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:35:22.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.556+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1733941415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb34102e80 msgr2=0x7fcb3410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:22.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.556+0000 7fcb38d0b700 1 --2- 172.21.15.40:0/1733941415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb34102e80 0x7fcb3410f330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00b0a0 tx=0x7fcb2c00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:35:22.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.556+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1733941415 shutdown_connections 2024-06-06T03:35:22.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.556+0000 7fcb38d0b700 1 --2- 172.21.15.40:0/1733941415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb341084f0 0x7fcb341088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:22.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.556+0000 7fcb38d0b700 1 --2- 172.21.15.40:0/1733941415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb34102e80 0x7fcb3410f330 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:22.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.556+0000 7fcb38d0b700 1 --2- 172.21.15.40:0/1733941415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb341024e0 0x7fcb34102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.556+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1733941415 >> 172.21.15.40:0/1733941415 conn(0x7fcb34079530 msgr2=0x7fcb34079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.556+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1733941415 shutdown_connections 2024-06-06T03:35:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.556+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1733941415 wait complete. 2024-06-06T03:35:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.557+0000 7fcb38d0b700 1 Processor -- start 2024-06-06T03:35:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.557+0000 7fcb38d0b700 1 -- start start 2024-06-06T03:35:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.557+0000 7fcb38d0b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb341024e0 0x7fcb34198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.557+0000 7fcb38d0b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb34102e80 0x7fcb341992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.557+0000 7fcb38d0b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb341084f0 0x7fcb3419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.557+0000 7fcb38d0b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcb34112830 con 0x7fcb341024e0 2024-06-06T03:35:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.557+0000 7fcb38d0b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb341126b0 con 0x7fcb34102e80 2024-06-06T03:35:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.557+0000 7fcb38d0b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcb341129b0 con 0x7fcb341084f0 2024-06-06T03:35:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.557+0000 7fcb3259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb341024e0 0x7fcb34198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.557+0000 7fcb3259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb341024e0 0x7fcb34198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46074/0 (socket says 172.21.15.40:46074) 2024-06-06T03:35:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.557+0000 7fcb3259c700 1 -- 172.21.15.40:0/1382245096 learned_addr learned my addr 172.21.15.40:0/1382245096 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.557+0000 7fcb31d9b700 1 --2- 172.21.15.40:0/1382245096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb34102e80 0x7fcb341992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.558+0000 7fcb3259c700 1 -- 172.21.15.40:0/1382245096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb341084f0 msgr2=0x7fcb3419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.558+0000 7fcb3259c700 1 --2- 172.21.15.40:0/1382245096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb341084f0 0x7fcb3419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.558+0000 7fcb3259c700 1 -- 172.21.15.40:0/1382245096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb34102e80 msgr2=0x7fcb341992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.558+0000 7fcb3259c700 1 --2- 172.21.15.40:0/1382245096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb34102e80 0x7fcb341992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.558+0000 7fcb3259c700 1 -- 172.21.15.40:0/1382245096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3419dd20 con 0x7fcb341024e0 2024-06-06T03:35:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.558+0000 7fcb31d9b700 1 --2- 172.21.15.40:0/1382245096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb34102e80 0x7fcb341992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:35:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.558+0000 7fcb3259c700 1 --2- 172.21.15.40:0/1382245096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb341024e0 0x7fcb34198d60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400b530 tx=0x7fcb2400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.558+0000 7fcb237fe700 1 -- 172.21.15.40:0/1382245096 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24015020 con 0x7fcb341024e0 2024-06-06T03:35:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.558+0000 7fcb237fe700 1 -- 172.21.15.40:0/1382245096 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb24011d10 con 0x7fcb341024e0 2024-06-06T03:35:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.558+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1382245096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3419e010 con 0x7fcb341024e0 2024-06-06T03:35:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.558+0000 7fcb237fe700 1 -- 172.21.15.40:0/1382245096 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24010e50 con 0x7fcb341024e0 2024-06-06T03:35:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.558+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1382245096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb341a5890 con 0x7fcb341024e0 2024-06-06T03:35:22.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.559+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1382245096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb34069180 con 0x7fcb341024e0 2024-06-06T03:35:22.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.560+0000 7fcb237fe700 1 -- 172.21.15.40:0/1382245096 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcb24018020 con 0x7fcb341024e0 2024-06-06T03:35:22.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.561+0000 7fcb237fe700 1 --2- 172.21.15.40:0/1382245096 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fcb1c0a2dc0 0x7fcb1c0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:22.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.561+0000 7fcb31d9b700 1 --2- 172.21.15.40:0/1382245096 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fcb1c0a2dc0 0x7fcb1c0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:22.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.561+0000 7fcb237fe700 1 -- 172.21.15.40:0/1382245096 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fcb240cede0 con 0x7fcb341024e0 2024-06-06T03:35:22.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.561+0000 7fcb31d9b700 1 --2- 172.21.15.40:0/1382245096 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fcb1c0a2dc0 0x7fcb1c0a5280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcb34079b30 tx=0x7fcb28008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:22.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.565+0000 7fcb237fe700 1 -- 172.21.15.40:0/1382245096 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcb2401f020 con 0x7fcb341024e0 2024-06-06T03:35:22.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.773+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1382245096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb3402dca0 con 0x7fcb341024e0 2024-06-06T03:35:22.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.774+0000 7fcb237fe700 1 -- 172.21.15.40:0/1382245096 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22242 (secure 0 0 0) 0x7fcb2409bdf0 con 0x7fcb341024e0 2024-06-06T03:35:22.774 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:35:22.775 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":88,"fsid":"77c25d68-23b4-11ef-bc9b-c7b262605968","created":"2024-06-06T03:26:53.165489+0000","modified":"2024-06-06T03:35:15.961722+0000","last_up_change":"2024-06-06T03:34:07.114448+0000","last_in_change":"2024-06-06T03:33:46.152948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T03:30:14.102785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T03:35:08.521025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T03:35:12.369035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63c0a320-a2f2-4b84-87e7-68501d335285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6803","nonce":3453526793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6805","nonce":3453526793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6809","nonce":3453526793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6807","nonce":3453526793}]},"public_addr":"172.21.15.40:6803/3453526793","cluster_addr":"172.21.15.40:6805/3453526793","heartbeat_back_addr":"172.21.15.40:6809/3453526793","heartbeat_front_addr":"172.21.15.40:6807/3453526793","state":["exists","up"]},{"osd":1,"uuid":"70a31534-06a1-4ffa-a54d-577d45bccf98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6801","nonce":405854252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6803","nonce":405854252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6807","nonce":405854252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6805","nonce":405854252}]},"public_addr":"172.21.15.104:6801/405854252","cluster_addr":"172.21.15.104:6803/405854252","heartbeat_back_addr":"172.21.15.104:6807/405854252","heartbeat_front_addr":"172.21.15.104:6805/405854252","state":["exists","up"]},{"osd":2,"uuid":"f5a89f43-e533-45f0-a777-bd3e9bfa8072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6801","nonce":3919221141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6803","nonce":3919221141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6807","nonce":3919221141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6805","nonce":3919221141}]},"public_addr":"172.21.15.131:6801/3919221141","cluster_addr":"172.21.15.131:6803/3919221141","heartbeat_back_addr":"172.21.15.131:6807/3919221141","heartbeat_front_addr":"172.21.15.131:6805/3919221141","state":["exists","up"]},{"osd":3,"uuid":"4df19390-6378-4346-a2be-29bce1dd2380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6811","nonce":2008541522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6813","nonce":2008541522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6817","nonce":2008541522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6815","nonce":2008541522}]},"public_addr":"172.21.15.40:6811/2008541522","cluster_addr":"172.21.15.40:6813/2008541522","heartbeat_back_addr":"172.21.15.40:6817/2008541522","heartbeat_front_addr":"172.21.15.40:6815/2008541522","state":["exists","up"]},{"osd":4,"uuid":"508de408-1cb1-4423-8199-db6bde9034f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6809","nonce":1153394847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6811","nonce":1153394847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6815","nonce":1153394847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6813","nonce":1153394847}]},"public_addr":"172.21.15.104:6809/1153394847","cluster_addr":"172.21.15.104:6811/1153394847","heartbeat_back_addr":"172.21.15.104:6815/1153394847","heartbeat_front_addr":"172.21.15.104:6813/1153394847","state":["exists","up"]},{"osd":5,"uuid":"a8c0cec9-a8ad-4a5d-800b-8b891ca6d836","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6809","nonce":1051933303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6811","nonce":1051933303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6815","nonce":1051933303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6813","nonce":1051933303}]},"public_addr":"172.21.15.131:6809/1051933303","cluster_addr":"172.21.15.131:6811/1051933303","heartbeat_back_addr":"172.21.15.131:6815/1051933303","heartbeat_front_addr":"172.21.15.131:6813/1051933303","state":["exists","up"]},{"osd":6,"uuid":"5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6819","nonce":2670922011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6821","nonce":2670922011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6825","nonce":2670922011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6823","nonce":2670922011}]},"public_addr":"172.21.15.40:6819/2670922011","cluster_addr":"172.21.15.40:6821/2670922011","heartbeat_back_addr":"172.21.15.40:6825/2670922011","heartbeat_front_addr":"172.21.15.40:6823/2670922011","state":["exists","up"]},{"osd":7,"uuid":"bfc536bb-6ae4-459e-afd9-8197c7420bd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6817","nonce":2353571051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6819","nonce":2353571051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6823","nonce":2353571051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6821","nonce":2353571051}]},"public_addr":"172.21.15.104:6817/2353571051","cluster_addr":"172.21.15.104:6819/2353571051","heartbeat_back_addr":"172.21.15.104:6823/2353571051","heartbeat_front_addr":"172.21.15.104:6821/2353571051","state":["exists","up"]},{"osd":8,"uuid":"1fe214f2-5ea3-4fc5-83bf-d9bccafe1529","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6817","nonce":1364533794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6819","nonce":1364533794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6823","nonce":1364533794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6821","nonce":1364533794}]},"public_addr":"172.21.15.131:6817/1364533794","cluster_addr":"172.21.15.131:6819/1364533794","heartbeat_back_addr":"172.21.15.131:6823/1364533794","heartbeat_front_addr":"172.21.15.131:6821/1364533794","state":["exists","up"]},{"osd":9,"uuid":"4b2b3555-6870-4ead-be50-b635799265a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6827","nonce":2604133190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6829","nonce":2604133190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6833","nonce":2604133190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6831","nonce":2604133190}]},"public_addr":"172.21.15.40:6827/2604133190","cluster_addr":"172.21.15.40:6829/2604133190","heartbeat_back_addr":"172.21.15.40:6833/2604133190","heartbeat_front_addr":"172.21.15.40:6831/2604133190","state":["exists","up"]},{"osd":10,"uuid":"70049e24-94ce-458e-97e3-dbcf9de05444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6825","nonce":2231164832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6827","nonce":2231164832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6831","nonce":2231164832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6829","nonce":2231164832}]},"public_addr":"172.21.15.104:6825/2231164832","cluster_addr":"172.21.15.104:6827/2231164832","heartbeat_back_addr":"172.21.15.104:6831/2231164832","heartbeat_front_addr":"172.21.15.104:6829/2231164832","state":["exists","up"]},{"osd":11,"uuid":"80447548-1fbe-4892-ac13-eb4bb6acf10d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6825","nonce":4128185670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6827","nonce":4128185670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6831","nonce":4128185670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6829","nonce":4128185670}]},"public_addr":"172.21.15.131:6825/4128185670","cluster_addr":"172.21.15.131:6827/4128185670","heartbeat_back_addr":"172.21.15.131:6831/4128185670","heartbeat_front_addr":"172.21.15.131:6829/4128185670","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:23.289784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:47.271677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:10.866792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:36.781924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:01.995848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:26.904606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:52.997438+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:19.221567+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:44.543977+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:11.948545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:38.402943+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:34:04.712200+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/23942578":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2234980855":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6801/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6800/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2301695357":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/1676880727":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6801/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6800/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/1597665640":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/513375601":"2024-06-07T03:27:19.799499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T03:35:22.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.776+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1382245096 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fcb1c0a2dc0 msgr2=0x7fcb1c0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:22.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.776+0000 7fcb38d0b700 1 --2- 172.21.15.40:0/1382245096 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fcb1c0a2dc0 0x7fcb1c0a5280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcb34079b30 tx=0x7fcb28008040 comp rx=0 tx=0).stop 2024-06-06T03:35:22.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.777+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1382245096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb341024e0 msgr2=0x7fcb34198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:22.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.777+0000 7fcb38d0b700 1 --2- 172.21.15.40:0/1382245096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb341024e0 0x7fcb34198d60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400b530 tx=0x7fcb2400b8f0 comp rx=0 tx=0).stop 2024-06-06T03:35:22.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.777+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1382245096 shutdown_connections 2024-06-06T03:35:22.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.777+0000 7fcb38d0b700 1 --2- 172.21.15.40:0/1382245096 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fcb1c0a2dc0 0x7fcb1c0a5280 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:22.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.777+0000 7fcb38d0b700 1 --2- 172.21.15.40:0/1382245096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb341084f0 0x7fcb3419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:22.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.777+0000 7fcb38d0b700 1 --2- 172.21.15.40:0/1382245096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb341024e0 0x7fcb34198d60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:22.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.777+0000 7fcb38d0b700 1 --2- 172.21.15.40:0/1382245096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb34102e80 0x7fcb341992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:22.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.777+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1382245096 >> 172.21.15.40:0/1382245096 conn(0x7fcb34079530 msgr2=0x7fcb341005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:22.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.777+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1382245096 shutdown_connections 2024-06-06T03:35:22.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:22.777+0000 7fcb38d0b700 1 -- 172.21.15.40:0/1382245096 wait complete. 2024-06-06T03:35:23.118 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:22 smithi040 ceph-mon[32220]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-06-06T03:35:23.118 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:22 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1382245096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:35:23.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:22 smithi131 ceph-mon[36687]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-06-06T03:35:23.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:22 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1382245096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:35:23.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:22 smithi104 ceph-mon[36569]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-06-06T03:35:23.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:22 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1382245096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:35:23.585 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-06T03:35:23.840 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:25.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:24 smithi131 ceph-mon[36687]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-06-06T03:35:25.321 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:24 smithi040 ceph-mon[32220]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-06-06T03:35:25.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:24 smithi104 ceph-mon[36569]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-06-06T03:35:25.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.877+0000 7fd71b3de700 1 -- 172.21.15.40:0/8536687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd714108510 msgr2=0x7fd7141088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:25.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.877+0000 7fd71b3de700 1 --2- 172.21.15.40:0/8536687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd714108510 0x7fd7141088f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd710009990 tx=0x7fd710009ca0 comp rx=0 tx=0).stop 2024-06-06T03:35:25.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.877+0000 7fd71b3de700 1 -- 172.21.15.40:0/8536687 shutdown_connections 2024-06-06T03:35:25.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.877+0000 7fd71b3de700 1 --2- 172.21.15.40:0/8536687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd714102ed0 0x7fd71410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:25.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.877+0000 7fd71b3de700 1 --2- 172.21.15.40:0/8536687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd714108510 0x7fd7141088f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:25.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.877+0000 7fd71b3de700 1 --2- 172.21.15.40:0/8536687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714102510 0x7fd714102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:25.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.877+0000 7fd71b3de700 1 -- 172.21.15.40:0/8536687 >> 172.21.15.40:0/8536687 conn(0x7fd7140795b0 msgr2=0x7fd7140799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:25.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.877+0000 7fd71b3de700 1 -- 172.21.15.40:0/8536687 shutdown_connections 2024-06-06T03:35:25.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.877+0000 7fd71b3de700 1 -- 172.21.15.40:0/8536687 wait complete. 2024-06-06T03:35:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd71b3de700 1 Processor -- start 2024-06-06T03:35:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd71b3de700 1 -- start start 2024-06-06T03:35:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd71b3de700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd714102510 0x7fd71419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd71b3de700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd714102ed0 0x7fd71419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd71b3de700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714108510 0x7fd714198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd71b3de700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd714112760 con 0x7fd714102ed0 2024-06-06T03:35:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd71b3de700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd7141125e0 con 0x7fd714102510 2024-06-06T03:35:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd71b3de700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd7141128e0 con 0x7fd714108510 2024-06-06T03:35:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd718979700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd714102ed0 0x7fd71419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd718979700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd714102ed0 0x7fd71419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46098/0 (socket says 172.21.15.40:46098) 2024-06-06T03:35:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd718979700 1 -- 172.21.15.40:0/1283081160 learned_addr learned my addr 172.21.15.40:0/1283081160 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:25.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd71917a700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd714102510 0x7fd71419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:25.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd71997b700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714108510 0x7fd714198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:25.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd718979700 1 -- 172.21.15.40:0/1283081160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714108510 msgr2=0x7fd714198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:25.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd718979700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714108510 0x7fd714198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:25.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd718979700 1 -- 172.21.15.40:0/1283081160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd714102510 msgr2=0x7fd71419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:25.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd718979700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd714102510 0x7fd71419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:25.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd718979700 1 -- 172.21.15.40:0/1283081160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd714199160 con 0x7fd714102ed0 2024-06-06T03:35:25.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.878+0000 7fd71997b700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714108510 0x7fd714198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:35:25.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.879+0000 7fd718979700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd714102ed0 0x7fd71419ed80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd70800c970 tx=0x7fd70800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:25.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.879+0000 7fd7067fc700 1 -- 172.21.15.40:0/1283081160 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708007aa0 con 0x7fd714102ed0 2024-06-06T03:35:25.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.879+0000 7fd7067fc700 1 -- 172.21.15.40:0/1283081160 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd708004d10 con 0x7fd714102ed0 2024-06-06T03:35:25.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.879+0000 7fd71b3de700 1 -- 172.21.15.40:0/1283081160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd714199450 con 0x7fd714102ed0 2024-06-06T03:35:25.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.879+0000 7fd7067fc700 1 -- 172.21.15.40:0/1283081160 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70800f450 con 0x7fd714102ed0 2024-06-06T03:35:25.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.879+0000 7fd71b3de700 1 -- 172.21.15.40:0/1283081160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7141a5710 con 0x7fd714102ed0 2024-06-06T03:35:25.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.880+0000 7fd71b3de700 1 -- 172.21.15.40:0/1283081160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd71410aa10 con 0x7fd714102ed0 2024-06-06T03:35:25.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.881+0000 7fd7067fc700 1 -- 172.21.15.40:0/1283081160 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd708004750 con 0x7fd714102ed0 2024-06-06T03:35:25.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.882+0000 7fd7067fc700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd7000a2ca0 0x7fd7000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:25.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.882+0000 7fd71917a700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd7000a2ca0 0x7fd7000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:25.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.882+0000 7fd7067fc700 1 -- 172.21.15.40:0/1283081160 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd7080cec50 con 0x7fd714102ed0 2024-06-06T03:35:25.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.882+0000 7fd71917a700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd7000a2ca0 0x7fd7000a5160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd71000b5c0 tx=0x7fd710005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:25.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:25.886+0000 7fd7067fc700 1 -- 172.21.15.40:0/1283081160 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd7080d4020 con 0x7fd714102ed0 2024-06-06T03:35:26.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:26.138+0000 7fd71b3de700 1 -- 172.21.15.40:0/1283081160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd714069180 con 0x7fd714102ed0 2024-06-06T03:35:26.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:26.875+0000 7fd7067fc700 1 -- 172.21.15.40:0/1283081160 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fd70809bd90 con 0x7fd714102ed0 2024-06-06T03:35:26.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:26.879+0000 7fd71b3de700 1 -- 172.21.15.40:0/1283081160 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd7000a2ca0 msgr2=0x7fd7000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:26.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:26.879+0000 7fd71b3de700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd7000a2ca0 0x7fd7000a5160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd71000b5c0 tx=0x7fd710005960 comp rx=0 tx=0).stop 2024-06-06T03:35:26.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:26.879+0000 7fd71b3de700 1 -- 172.21.15.40:0/1283081160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd714102ed0 msgr2=0x7fd71419ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:26.879+0000 7fd71b3de700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd714102ed0 0x7fd71419ed80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd70800c970 tx=0x7fd70800cc80 comp rx=0 tx=0).stop 2024-06-06T03:35:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:26.879+0000 7fd71b3de700 1 -- 172.21.15.40:0/1283081160 shutdown_connections 2024-06-06T03:35:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:26.879+0000 7fd71b3de700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd7000a2ca0 0x7fd7000a5160 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:26.879+0000 7fd71b3de700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd714108510 0x7fd714198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:26.879+0000 7fd71b3de700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd714102510 0x7fd71419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:26.879+0000 7fd71b3de700 1 --2- 172.21.15.40:0/1283081160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd714102ed0 0x7fd71419ed80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:26.879+0000 7fd71b3de700 1 -- 172.21.15.40:0/1283081160 >> 172.21.15.40:0/1283081160 conn(0x7fd7140795b0 msgr2=0x7fd7141011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:26.879+0000 7fd71b3de700 1 -- 172.21.15.40:0/1283081160 shutdown_connections 2024-06-06T03:35:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:26.879+0000 7fd71b3de700 1 -- 172.21.15.40:0/1283081160 wait complete. 2024-06-06T03:35:27.213 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:26 smithi040 ceph-mon[32220]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-06-06T03:35:27.213 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:26 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1283081160' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T03:35:27.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:26 smithi131 ceph-mon[36687]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-06-06T03:35:27.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:26 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1283081160' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T03:35:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:26 smithi104 ceph-mon[36569]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-06-06T03:35:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:26 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1283081160' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T03:35:27.641 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-06-06T03:35:27.642 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-06-06T03:35:27.904 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:28.157 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:27 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1283081160' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T03:35:28.157 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:27 smithi040 ceph-mon[32220]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T03:35:28.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:27 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1283081160' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T03:35:28.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:27 smithi131 ceph-mon[36687]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T03:35:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:27 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1283081160' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T03:35:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:27 smithi104 ceph-mon[36569]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T03:35:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:28 smithi040 ceph-mon[32220]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-06-06T03:35:29.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:28 smithi131 ceph-mon[36687]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-06-06T03:35:29.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:28 smithi104 ceph-mon[36569]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-06-06T03:35:29.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.697+0000 7f232f38c700 1 -- 172.21.15.40:0/3781027064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2328102ef0 msgr2=0x7f232810f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:29.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.697+0000 7f232f38c700 1 --2- 172.21.15.40:0/3781027064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2328102ef0 0x7f232810f3a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f232000b0d0 tx=0x7f232000b3e0 comp rx=0 tx=0).stop 2024-06-06T03:35:29.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.697+0000 7f232f38c700 1 -- 172.21.15.40:0/3781027064 shutdown_connections 2024-06-06T03:35:29.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.697+0000 7f232f38c700 1 --2- 172.21.15.40:0/3781027064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23281085d0 0x7f23281089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:29.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.697+0000 7f232f38c700 1 --2- 172.21.15.40:0/3781027064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2328102ef0 0x7f232810f3a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:29.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.697+0000 7f232f38c700 1 --2- 172.21.15.40:0/3781027064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2328102530 0x7f23281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:29.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.697+0000 7f232f38c700 1 -- 172.21.15.40:0/3781027064 >> 172.21.15.40:0/3781027064 conn(0x7f2328079640 msgr2=0x7f2328079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:29.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.697+0000 7f232f38c700 1 -- 172.21.15.40:0/3781027064 shutdown_connections 2024-06-06T03:35:29.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.697+0000 7f232f38c700 1 -- 172.21.15.40:0/3781027064 wait complete. 2024-06-06T03:35:29.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.697+0000 7f232f38c700 1 Processor -- start 2024-06-06T03:35:29.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232f38c700 1 -- start start 2024-06-06T03:35:29.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232f38c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2328102530 0x7f2328074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:29.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232f38c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2328102ef0 0x7f2328074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232f38c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23281085d0 0x7f2328075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232f38c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2328112830 con 0x7f2328102ef0 2024-06-06T03:35:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232f38c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f23281126b0 con 0x7f2328102530 2024-06-06T03:35:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232f38c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f23281129b0 con 0x7f23281085d0 2024-06-06T03:35:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232c927700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2328102ef0 0x7f2328074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232c927700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2328102ef0 0x7f2328074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37120/0 (socket says 172.21.15.40:37120) 2024-06-06T03:35:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232c927700 1 -- 172.21.15.40:0/3982715985 learned_addr learned my addr 172.21.15.40:0/3982715985 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232d128700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2328102530 0x7f2328074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232d929700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23281085d0 0x7f2328075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232c927700 1 -- 172.21.15.40:0/3982715985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23281085d0 msgr2=0x7f2328075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232c927700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23281085d0 0x7f2328075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232c927700 1 -- 172.21.15.40:0/3982715985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2328102530 msgr2=0x7f2328074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232c927700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2328102530 0x7f2328074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232c927700 1 -- 172.21.15.40:0/3982715985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23281a70f0 con 0x7f2328102ef0 2024-06-06T03:35:29.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.698+0000 7f232d929700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23281085d0 0x7f2328075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:35:29.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.699+0000 7f232c927700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2328102ef0 0x7f2328074cd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f231800d980 tx=0x7f231800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:29.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.699+0000 7f231e7fc700 1 -- 172.21.15.40:0/3982715985 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318016070 con 0x7f2328102ef0 2024-06-06T03:35:29.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.699+0000 7f231e7fc700 1 -- 172.21.15.40:0/3982715985 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2318004d10 con 0x7f2328102ef0 2024-06-06T03:35:29.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.699+0000 7f232f38c700 1 -- 172.21.15.40:0/3982715985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23281a73e0 con 0x7f2328102ef0 2024-06-06T03:35:29.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.699+0000 7f231e7fc700 1 -- 172.21.15.40:0/3982715985 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23180055f0 con 0x7f2328102ef0 2024-06-06T03:35:29.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.699+0000 7f232f38c700 1 -- 172.21.15.40:0/3982715985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23281a7970 con 0x7f2328102ef0 2024-06-06T03:35:29.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.700+0000 7f232f38c700 1 -- 172.21.15.40:0/3982715985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2328103c40 con 0x7f2328102ef0 2024-06-06T03:35:29.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.701+0000 7f231e7fc700 1 -- 172.21.15.40:0/3982715985 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2318004750 con 0x7f2328102ef0 2024-06-06T03:35:29.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.702+0000 7f231e7fc700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f23140a2ca0 0x7f23140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:29.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.702+0000 7f232d128700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f23140a2ca0 0x7f23140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:29.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.702+0000 7f231e7fc700 1 -- 172.21.15.40:0/3982715985 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f23180cecc0 con 0x7f2328102ef0 2024-06-06T03:35:29.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.702+0000 7f232d128700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f23140a2ca0 0x7f23140a5160 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2324005950 tx=0x7f232400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:29.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.707+0000 7f231e7fc700 1 -- 172.21.15.40:0/3982715985 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f231801f020 con 0x7f2328102ef0 2024-06-06T03:35:29.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:29.965+0000 7f232f38c700 1 -- 172.21.15.40:0/3982715985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f23281a7ce0 con 0x7f2328102ef0 2024-06-06T03:35:30.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:30.891+0000 7f231e7fc700 1 -- 172.21.15.40:0/3982715985 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f231809bd70 con 0x7f2328102ef0 2024-06-06T03:35:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:30.894+0000 7f232f38c700 1 -- 172.21.15.40:0/3982715985 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f23140a2ca0 msgr2=0x7f23140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:30.894+0000 7f232f38c700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f23140a2ca0 0x7f23140a5160 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2324005950 tx=0x7f232400a300 comp rx=0 tx=0).stop 2024-06-06T03:35:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:30.894+0000 7f232f38c700 1 -- 172.21.15.40:0/3982715985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2328102ef0 msgr2=0x7f2328074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:30.894+0000 7f232f38c700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2328102ef0 0x7f2328074cd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f231800d980 tx=0x7f231800dc90 comp rx=0 tx=0).stop 2024-06-06T03:35:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:30.894+0000 7f232f38c700 1 -- 172.21.15.40:0/3982715985 shutdown_connections 2024-06-06T03:35:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:30.894+0000 7f232f38c700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f23140a2ca0 0x7f23140a5160 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:30.894+0000 7f232f38c700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23281085d0 0x7f2328075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:30.894+0000 7f232f38c700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2328102530 0x7f2328074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:30.894+0000 7f232f38c700 1 --2- 172.21.15.40:0/3982715985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2328102ef0 0x7f2328074cd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:30.894+0000 7f232f38c700 1 -- 172.21.15.40:0/3982715985 >> 172.21.15.40:0/3982715985 conn(0x7f2328079640 msgr2=0x7f2328100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:30.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:30.894+0000 7f232f38c700 1 -- 172.21.15.40:0/3982715985 shutdown_connections 2024-06-06T03:35:30.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:30.894+0000 7f232f38c700 1 -- 172.21.15.40:0/3982715985 wait complete. 2024-06-06T03:35:31.142 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:30 smithi104 ceph-mon[36569]: pgmap v309: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-06-06T03:35:31.142 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:30 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3982715985' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-06T03:35:31.222 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:30 smithi040 ceph-mon[32220]: pgmap v309: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-06-06T03:35:31.223 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:30 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3982715985' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-06T03:35:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:30 smithi131 ceph-mon[36687]: pgmap v309: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-06-06T03:35:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:30 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3982715985' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-06T03:35:31.693 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-06T03:35:31.959 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:31 smithi040 ceph-mon[32220]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T03:35:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:31 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3982715985' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-06T03:35:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:31 smithi040 ceph-mon[32220]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T03:35:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:31 smithi040 ceph-mon[32220]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T03:35:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:31 smithi040 ceph-mon[32220]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-06-06T03:35:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:31 smithi040 ceph-mon[32220]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-06-06T03:35:32.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:31 smithi131 ceph-mon[36687]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T03:35:32.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:31 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3982715985' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-06T03:35:32.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:31 smithi131 ceph-mon[36687]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T03:35:32.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:31 smithi131 ceph-mon[36687]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T03:35:32.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:31 smithi131 ceph-mon[36687]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-06-06T03:35:32.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:31 smithi131 ceph-mon[36687]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-06-06T03:35:32.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:31 smithi104 ceph-mon[36569]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T03:35:32.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:31 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3982715985' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-06T03:35:32.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:31 smithi104 ceph-mon[36569]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T03:35:32.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:31 smithi104 ceph-mon[36569]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T03:35:32.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:31 smithi104 ceph-mon[36569]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-06-06T03:35:32.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:31 smithi104 ceph-mon[36569]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-06-06T03:35:33.183 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:32 smithi040 ceph-mon[32220]: pgmap v310: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 2 op/s 2024-06-06T03:35:33.184 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:32 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] up:active 2024-06-06T03:35:33.184 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:32 smithi040 ceph-mon[32220]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T03:35:33.184 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:32 smithi040 ceph-mon[32220]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-06-06T03:35:33.184 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:32 smithi040 ceph-mon[32220]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-06-06T03:35:33.185 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:32 smithi040 ceph-mon[32220]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-06-06T03:35:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:32 smithi131 ceph-mon[36687]: pgmap v310: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 2 op/s 2024-06-06T03:35:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:32 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] up:active 2024-06-06T03:35:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:32 smithi131 ceph-mon[36687]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T03:35:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:32 smithi131 ceph-mon[36687]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-06-06T03:35:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:32 smithi131 ceph-mon[36687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-06-06T03:35:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:32 smithi131 ceph-mon[36687]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-06-06T03:35:33.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:32 smithi104 ceph-mon[36569]: pgmap v310: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 2 op/s 2024-06-06T03:35:33.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:32 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] up:active 2024-06-06T03:35:33.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:32 smithi104 ceph-mon[36569]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T03:35:33.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:32 smithi104 ceph-mon[36569]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-06-06T03:35:33.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:32 smithi104 ceph-mon[36569]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-06-06T03:35:33.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:32 smithi104 ceph-mon[36569]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-06-06T03:35:33.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.740+0000 7f089143a700 1 -- 172.21.15.40:0/2102284497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f088c06b580 msgr2=0x7f088c10f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:33.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.740+0000 7f089143a700 1 --2- 172.21.15.40:0/2102284497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f088c06b580 0x7f088c10f370 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f088400b0d0 tx=0x7f088400b3e0 comp rx=0 tx=0).stop 2024-06-06T03:35:33.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.740+0000 7f089143a700 1 -- 172.21.15.40:0/2102284497 shutdown_connections 2024-06-06T03:35:33.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.740+0000 7f089143a700 1 --2- 172.21.15.40:0/2102284497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f088c1085a0 0x7f088c108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:33.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.740+0000 7f089143a700 1 --2- 172.21.15.40:0/2102284497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f088c06b580 0x7f088c10f370 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:33.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.740+0000 7f089143a700 1 --2- 172.21.15.40:0/2102284497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f088c06abc0 0x7f088c06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:33.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.740+0000 7f089143a700 1 -- 172.21.15.40:0/2102284497 >> 172.21.15.40:0/2102284497 conn(0x7f088c077200 msgr2=0x7f088c077610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:33.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.740+0000 7f089143a700 1 -- 172.21.15.40:0/2102284497 shutdown_connections 2024-06-06T03:35:33.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.740+0000 7f089143a700 1 -- 172.21.15.40:0/2102284497 wait complete. 2024-06-06T03:35:33.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.740+0000 7f089143a700 1 Processor -- start 2024-06-06T03:35:33.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.740+0000 7f089143a700 1 -- start start 2024-06-06T03:35:33.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f089143a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f088c06abc0 0x7f088c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:33.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f089143a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f088c06b580 0x7f088c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:33.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f089143a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f088c1085a0 0x7f088c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:33.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f089143a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f088c112860 con 0x7f088c06abc0 2024-06-06T03:35:33.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f089143a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f088c1126e0 con 0x7f088c06b580 2024-06-06T03:35:33.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f089143a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f088c1129e0 con 0x7f088c1085a0 2024-06-06T03:35:33.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f088affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f088c06abc0 0x7f088c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:33.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f088affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f088c06abc0 0x7f088c19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37142/0 (socket says 172.21.15.40:37142) 2024-06-06T03:35:33.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f088affd700 1 -- 172.21.15.40:0/3995784475 learned_addr learned my addr 172.21.15.40:0/3995784475 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:33.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f088b7fe700 1 --2- 172.21.15.40:0/3995784475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f088c1085a0 0x7f088c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:33.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f088a7fc700 1 --2- 172.21.15.40:0/3995784475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f088c06b580 0x7f088c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:33.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f088affd700 1 -- 172.21.15.40:0/3995784475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f088c1085a0 msgr2=0x7f088c198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:33.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f088affd700 1 --2- 172.21.15.40:0/3995784475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f088c1085a0 0x7f088c198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:33.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f088affd700 1 -- 172.21.15.40:0/3995784475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f088c06b580 msgr2=0x7f088c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:33.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f088affd700 1 --2- 172.21.15.40:0/3995784475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f088c06b580 0x7f088c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:33.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f088affd700 1 -- 172.21.15.40:0/3995784475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c199180 con 0x7f088c06abc0 2024-06-06T03:35:33.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f088affd700 1 --2- 172.21.15.40:0/3995784475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f088c06abc0 0x7f088c19e860 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f087400b530 tx=0x7f087400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:33.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.741+0000 7f0883fff700 1 -- 172.21.15.40:0/3995784475 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874015020 con 0x7f088c06abc0 2024-06-06T03:35:33.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.742+0000 7f089143a700 1 -- 172.21.15.40:0/3995784475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088c199470 con 0x7f088c06abc0 2024-06-06T03:35:33.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.742+0000 7f0883fff700 1 -- 172.21.15.40:0/3995784475 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0874011d10 con 0x7f088c06abc0 2024-06-06T03:35:33.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.742+0000 7f0883fff700 1 -- 172.21.15.40:0/3995784475 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087400f2e0 con 0x7f088c06abc0 2024-06-06T03:35:33.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.742+0000 7f089143a700 1 -- 172.21.15.40:0/3995784475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f088c1a5730 con 0x7f088c06abc0 2024-06-06T03:35:33.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.743+0000 7f089143a700 1 -- 172.21.15.40:0/3995784475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088c103bf0 con 0x7f088c06abc0 2024-06-06T03:35:33.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.744+0000 7f0883fff700 1 -- 172.21.15.40:0/3995784475 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0874018020 con 0x7f088c06abc0 2024-06-06T03:35:33.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.745+0000 7f0883fff700 1 --2- 172.21.15.40:0/3995784475 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f08780a2c20 0x7f08780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:33.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.745+0000 7f088a7fc700 1 --2- 172.21.15.40:0/3995784475 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f08780a2c20 0x7f08780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:33.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.745+0000 7f0883fff700 1 -- 172.21.15.40:0/3995784475 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f08740ced30 con 0x7f088c06abc0 2024-06-06T03:35:33.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.745+0000 7f088a7fc700 1 --2- 172.21.15.40:0/3995784475 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f08780a2c20 0x7f08780a50e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f087c005ea0 tx=0x7f087c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:33.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:33.748+0000 7f0883fff700 1 -- 172.21.15.40:0/3995784475 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f087401f020 con 0x7f088c06abc0 2024-06-06T03:35:34.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:34.007+0000 7f089143a700 1 -- 172.21.15.40:0/3995784475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f088c02dca0 con 0x7f088c06abc0 2024-06-06T03:35:34.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:33 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] up:active 2024-06-06T03:35:34.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:33 smithi131 ceph-mon[36687]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-06T03:35:34.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:33 smithi131 ceph-mon[36687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-06-06T03:35:34.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:33 smithi131 ceph-mon[36687]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=i=up:creating} 8 up:standby 2024-06-06T03:35:34.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:33 smithi131 ceph-mon[36687]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-06-06T03:35:34.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:33 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] up:active 2024-06-06T03:35:34.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:33 smithi040 ceph-mon[32220]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-06T03:35:34.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:33 smithi040 ceph-mon[32220]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-06-06T03:35:34.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:33 smithi040 ceph-mon[32220]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=i=up:creating} 8 up:standby 2024-06-06T03:35:34.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:33 smithi040 ceph-mon[32220]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-06-06T03:35:34.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:33 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] up:active 2024-06-06T03:35:34.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:33 smithi104 ceph-mon[36569]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-06T03:35:34.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:33 smithi104 ceph-mon[36569]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-06-06T03:35:34.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:33 smithi104 ceph-mon[36569]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=i=up:creating} 8 up:standby 2024-06-06T03:35:34.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:33 smithi104 ceph-mon[36569]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-06-06T03:35:34.924 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:35:34 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b[66930]: 2024-06-06T03:35:34.615+0000 7f16ce78f700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-06-06T03:35:34.924 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:35:34 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b[66930]: 2024-06-06T03:35:34.615+0000 7f16ce78f700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-06-06T03:35:34.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:34.930+0000 7f0883fff700 1 -- 172.21.15.40:0/3995784475 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v26) ==== 121+0+0 (secure 0 0 0) 0x7f087409bde0 con 0x7f088c06abc0 2024-06-06T03:35:34.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:34.933+0000 7f089143a700 1 -- 172.21.15.40:0/3995784475 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f08780a2c20 msgr2=0x7f08780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:34.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:34.933+0000 7f089143a700 1 --2- 172.21.15.40:0/3995784475 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f08780a2c20 0x7f08780a50e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f087c005ea0 tx=0x7f087c005e30 comp rx=0 tx=0).stop 2024-06-06T03:35:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:34.933+0000 7f089143a700 1 -- 172.21.15.40:0/3995784475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f088c06abc0 msgr2=0x7f088c19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:34.933+0000 7f089143a700 1 --2- 172.21.15.40:0/3995784475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f088c06abc0 0x7f088c19e860 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f087400b530 tx=0x7f087400b840 comp rx=0 tx=0).stop 2024-06-06T03:35:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:34.933+0000 7f089143a700 1 -- 172.21.15.40:0/3995784475 shutdown_connections 2024-06-06T03:35:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:34.933+0000 7f089143a700 1 --2- 172.21.15.40:0/3995784475 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f08780a2c20 0x7f08780a50e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:34.933+0000 7f089143a700 1 --2- 172.21.15.40:0/3995784475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f088c1085a0 0x7f088c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:34.933+0000 7f089143a700 1 --2- 172.21.15.40:0/3995784475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f088c06abc0 0x7f088c19e860 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:34.933+0000 7f089143a700 1 --2- 172.21.15.40:0/3995784475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f088c06b580 0x7f088c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:34.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:34.933+0000 7f089143a700 1 -- 172.21.15.40:0/3995784475 >> 172.21.15.40:0/3995784475 conn(0x7f088c077200 msgr2=0x7f088c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:34.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:34.934+0000 7f089143a700 1 -- 172.21.15.40:0/3995784475 shutdown_connections 2024-06-06T03:35:34.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:34.934+0000 7f089143a700 1 -- 172.21.15.40:0/3995784475 wait complete. 2024-06-06T03:35:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:34 smithi131 ceph-mon[36687]: pgmap v311: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-06-06T03:35:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:34 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] up:active 2024-06-06T03:35:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:34 smithi131 ceph-mon[36687]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-06T03:35:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:34 smithi131 ceph-mon[36687]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T03:35:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:34 smithi131 ceph-mon[36687]: Cluster is now healthy 2024-06-06T03:35:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:34 smithi131 ceph-mon[36687]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=i=up:active} 8 up:standby 2024-06-06T03:35:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:34 smithi131 ceph-mon[36687]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-06T03:35:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:34 smithi131 ceph-mon[36687]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-06-06T03:35:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:34 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3995784475' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T03:35:35.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:34 smithi040 ceph-mon[32220]: pgmap v311: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-06-06T03:35:35.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:34 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] up:active 2024-06-06T03:35:35.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:34 smithi040 ceph-mon[32220]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-06T03:35:35.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:34 smithi040 ceph-mon[32220]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T03:35:35.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:34 smithi040 ceph-mon[32220]: Cluster is now healthy 2024-06-06T03:35:35.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:34 smithi040 ceph-mon[32220]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=i=up:active} 8 up:standby 2024-06-06T03:35:35.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:34 smithi040 ceph-mon[32220]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-06T03:35:35.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:34 smithi040 ceph-mon[32220]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-06-06T03:35:35.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:34 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3995784475' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T03:35:35.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:34 smithi104 ceph-mon[36569]: pgmap v311: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-06-06T03:35:35.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:34 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] up:active 2024-06-06T03:35:35.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:34 smithi104 ceph-mon[36569]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-06T03:35:35.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:34 smithi104 ceph-mon[36569]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T03:35:35.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:34 smithi104 ceph-mon[36569]: Cluster is now healthy 2024-06-06T03:35:35.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:34 smithi104 ceph-mon[36569]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=i=up:active} 8 up:standby 2024-06-06T03:35:35.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:34 smithi104 ceph-mon[36569]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-06T03:35:35.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:34 smithi104 ceph-mon[36569]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-06-06T03:35:35.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:34 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3995784475' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T03:35:35.773 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-06T03:35:36.027 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:36.276 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:35 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3995784475' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T03:35:36.277 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:35 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] up:active 2024-06-06T03:35:36.277 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:35 smithi040 ceph-mon[32220]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-06T03:35:36.277 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:35 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:35:36.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:35 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3995784475' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T03:35:36.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:35 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] up:active 2024-06-06T03:35:36.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:35 smithi131 ceph-mon[36687]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-06T03:35:36.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:35 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:35:36.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:35 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3995784475' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T03:35:36.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:35 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] up:active 2024-06-06T03:35:36.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:35 smithi104 ceph-mon[36569]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-06T03:35:36.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:35 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:35:37.227 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:36 smithi040 ceph-mon[32220]: pgmap v312: 97 pgs: 97 active+clean; 583 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 2.9 KiB/s wr, 8 op/s 2024-06-06T03:35:37.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:36 smithi131 ceph-mon[36687]: pgmap v312: 97 pgs: 97 active+clean; 583 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 2.9 KiB/s wr, 8 op/s 2024-06-06T03:35:37.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:36 smithi104 ceph-mon[36569]: pgmap v312: 97 pgs: 97 active+clean; 583 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 2.9 KiB/s wr, 8 op/s 2024-06-06T03:35:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.786+0000 7fc7e7859700 1 -- 172.21.15.40:0/2996009186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7e0101390 msgr2=0x7fc7e0101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.786+0000 7fc7e7859700 1 --2- 172.21.15.40:0/2996009186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7e0101390 0x7fc7e0101810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0009990 tx=0x7fc7d0009ca0 comp rx=0 tx=0).stop 2024-06-06T03:35:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.786+0000 7fc7e7859700 1 -- 172.21.15.40:0/2996009186 shutdown_connections 2024-06-06T03:35:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.786+0000 7fc7e7859700 1 --2- 172.21.15.40:0/2996009186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e0101d50 0x7fc7e010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.786+0000 7fc7e7859700 1 --2- 172.21.15.40:0/2996009186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7e0107400 0x7fc7e01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.786+0000 7fc7e7859700 1 --2- 172.21.15.40:0/2996009186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7e0101390 0x7fc7e0101810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.786+0000 7fc7e7859700 1 -- 172.21.15.40:0/2996009186 >> 172.21.15.40:0/2996009186 conn(0x7fc7e0078340 msgr2=0x7fc7e0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.786+0000 7fc7e7859700 1 -- 172.21.15.40:0/2996009186 shutdown_connections 2024-06-06T03:35:37.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.786+0000 7fc7e7859700 1 -- 172.21.15.40:0/2996009186 wait complete. 2024-06-06T03:35:37.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.786+0000 7fc7e7859700 1 Processor -- start 2024-06-06T03:35:37.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e7859700 1 -- start start 2024-06-06T03:35:37.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e7859700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e0101390 0x7fc7e019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:37.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e7859700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7e0101d50 0x7fc7e019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:37.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e7859700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7e0107400 0x7fc7e0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e7859700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01116f0 con 0x7fc7e0101d50 2024-06-06T03:35:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e7859700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0111570 con 0x7fc7e0107400 2024-06-06T03:35:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e7859700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0111870 con 0x7fc7e0101390 2024-06-06T03:35:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e4df4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7e0101d50 0x7fc7e019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e4df4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7e0101d50 0x7fc7e019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37158/0 (socket says 172.21.15.40:37158) 2024-06-06T03:35:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e4df4700 1 -- 172.21.15.40:0/2190418873 learned_addr learned my addr 172.21.15.40:0/2190418873 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e55f5700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e0101390 0x7fc7e019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e5df6700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7e0107400 0x7fc7e0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e4df4700 1 -- 172.21.15.40:0/2190418873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e0101390 msgr2=0x7fc7e019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e4df4700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e0101390 0x7fc7e019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e4df4700 1 -- 172.21.15.40:0/2190418873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7e0107400 msgr2=0x7fc7e0198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e4df4700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7e0107400 0x7fc7e0198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e4df4700 1 -- 172.21.15.40:0/2190418873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e01991f0 con 0x7fc7e0101d50 2024-06-06T03:35:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e5df6700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7e0107400 0x7fc7e0198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:35:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.787+0000 7fc7e4df4700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7e0101d50 0x7fc7e019ee10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0000c00 tx=0x7fc7d0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.788+0000 7fc7d67fc700 1 -- 172.21.15.40:0/2190418873 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0027070 con 0x7fc7e0101d50 2024-06-06T03:35:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.788+0000 7fc7d67fc700 1 -- 172.21.15.40:0/2190418873 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d0004110 con 0x7fc7e0101d50 2024-06-06T03:35:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.788+0000 7fc7d67fc700 1 -- 172.21.15.40:0/2190418873 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d000f460 con 0x7fc7e0101d50 2024-06-06T03:35:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.788+0000 7fc7e7859700 1 -- 172.21.15.40:0/2190418873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e0199480 con 0x7fc7e0101d50 2024-06-06T03:35:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.788+0000 7fc7e7859700 1 -- 172.21.15.40:0/2190418873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e01a57a0 con 0x7fc7e0101d50 2024-06-06T03:35:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.789+0000 7fc7e7859700 1 -- 172.21.15.40:0/2190418873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e0069180 con 0x7fc7e0101d50 2024-06-06T03:35:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.790+0000 7fc7d67fc700 1 -- 172.21.15.40:0/2190418873 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc7d002a070 con 0x7fc7e0101d50 2024-06-06T03:35:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.791+0000 7fc7d67fc700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc7cc0a2ca0 0x7fc7cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.791+0000 7fc7e55f5700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc7cc0a2ca0 0x7fc7cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.791+0000 7fc7d67fc700 1 -- 172.21.15.40:0/2190418873 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fc7d00d0290 con 0x7fc7e0101d50 2024-06-06T03:35:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.791+0000 7fc7e55f5700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc7cc0a2ca0 0x7fc7cc0a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0199bd0 tx=0x7fc7dc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:37.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:37.794+0000 7fc7d67fc700 1 -- 172.21.15.40:0/2190418873 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc7d0031070 con 0x7fc7e0101d50 2024-06-06T03:35:38.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:38.051+0000 7fc7e7859700 1 -- 172.21.15.40:0/2190418873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc7e01a5970 con 0x7fc7e0101d50 2024-06-06T03:35:38.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:38.946+0000 7fc7d67fc700 1 -- 172.21.15.40:0/2190418873 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7fc7d0013070 con 0x7fc7e0101d50 2024-06-06T03:35:38.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:38.949+0000 7fc7e7859700 1 -- 172.21.15.40:0/2190418873 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc7cc0a2ca0 msgr2=0x7fc7cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:38.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:38.949+0000 7fc7e7859700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc7cc0a2ca0 0x7fc7cc0a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0199bd0 tx=0x7fc7dc009190 comp rx=0 tx=0).stop 2024-06-06T03:35:38.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:38.949+0000 7fc7e7859700 1 -- 172.21.15.40:0/2190418873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7e0101d50 msgr2=0x7fc7e019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:38.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:38.949+0000 7fc7e7859700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7e0101d50 0x7fc7e019ee10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0000c00 tx=0x7fc7d0003680 comp rx=0 tx=0).stop 2024-06-06T03:35:38.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:38.949+0000 7fc7e7859700 1 -- 172.21.15.40:0/2190418873 shutdown_connections 2024-06-06T03:35:38.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:38.949+0000 7fc7e7859700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc7cc0a2ca0 0x7fc7cc0a5160 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:38.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:38.949+0000 7fc7e7859700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e0101390 0x7fc7e019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:38.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:38.949+0000 7fc7e7859700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7e0107400 0x7fc7e0198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:38.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:38.949+0000 7fc7e7859700 1 --2- 172.21.15.40:0/2190418873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7e0101d50 0x7fc7e019ee10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:38.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:38.949+0000 7fc7e7859700 1 -- 172.21.15.40:0/2190418873 >> 172.21.15.40:0/2190418873 conn(0x7fc7e0078340 msgr2=0x7fc7e0100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:38.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:38.949+0000 7fc7e7859700 1 -- 172.21.15.40:0/2190418873 shutdown_connections 2024-06-06T03:35:38.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:38.949+0000 7fc7e7859700 1 -- 172.21.15.40:0/2190418873 wait complete. 2024-06-06T03:35:39.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:38 smithi131 ceph-mon[36687]: pgmap v313: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-06-06T03:35:39.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:38 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2190418873' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T03:35:39.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:38 smithi040 ceph-mon[32220]: pgmap v313: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-06-06T03:35:39.398 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:38 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2190418873' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T03:35:39.398 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:38 smithi104 ceph-mon[36569]: pgmap v313: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-06-06T03:35:39.399 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:38 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2190418873' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T03:35:39.789 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-06-06T03:35:39.789 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-06T03:35:40.047 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:40.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:39 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2190418873' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T03:35:40.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:39 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:35:40.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:39 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2190418873' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T03:35:40.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:39 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:35:40.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:39 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2190418873' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T03:35:40.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:39 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:35:41.221 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:40 smithi040 ceph-mon[32220]: pgmap v314: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-06T03:35:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:40 smithi131 ceph-mon[36687]: pgmap v314: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-06T03:35:41.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:40 smithi104 ceph-mon[36569]: pgmap v314: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-06T03:35:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.802+0000 7f35a3574700 1 -- 172.21.15.40:0/3288528009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c06ac60 msgr2=0x7f359c06b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.802+0000 7f35a3574700 1 --2- 172.21.15.40:0/3288528009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c06ac60 0x7f359c06b0e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f358c0099c0 tx=0x7f358c009cd0 comp rx=0 tx=0).stop 2024-06-06T03:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.803+0000 7f35a3574700 1 -- 172.21.15.40:0/3288528009 shutdown_connections 2024-06-06T03:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.803+0000 7f35a3574700 1 --2- 172.21.15.40:0/3288528009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f359c06b620 0x7f359c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.803+0000 7f35a3574700 1 --2- 172.21.15.40:0/3288528009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f359c108580 0x7f359c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.803+0000 7f35a3574700 1 --2- 172.21.15.40:0/3288528009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c06ac60 0x7f359c06b0e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.803+0000 7f35a3574700 1 -- 172.21.15.40:0/3288528009 >> 172.21.15.40:0/3288528009 conn(0x7f359c078430 msgr2=0x7f359c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.803+0000 7f35a3574700 1 -- 172.21.15.40:0/3288528009 shutdown_connections 2024-06-06T03:35:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.803+0000 7f35a3574700 1 -- 172.21.15.40:0/3288528009 wait complete. 2024-06-06T03:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.803+0000 7f35a3574700 1 Processor -- start 2024-06-06T03:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.803+0000 7f35a3574700 1 -- start start 2024-06-06T03:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a3574700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f359c06ac60 0x7f359c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a3574700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f359c06b620 0x7f359c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a3574700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c108580 0x7f359c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a3574700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1127d0 con 0x7f359c108580 2024-06-06T03:35:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a3574700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f359c112650 con 0x7f359c06b620 2024-06-06T03:35:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a3574700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f359c112950 con 0x7f359c06ac60 2024-06-06T03:35:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a1b11700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c108580 0x7f359c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a1b11700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c108580 0x7f359c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48582/0 (socket says 172.21.15.40:48582) 2024-06-06T03:35:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a1b11700 1 -- 172.21.15.40:0/3121917875 learned_addr learned my addr 172.21.15.40:0/3121917875 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a1310700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f359c06ac60 0x7f359c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a0b0f700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f359c06b620 0x7f359c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a1b11700 1 -- 172.21.15.40:0/3121917875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f359c06ac60 msgr2=0x7f359c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a1b11700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f359c06ac60 0x7f359c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a1b11700 1 -- 172.21.15.40:0/3121917875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f359c06b620 msgr2=0x7f359c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a1b11700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f359c06b620 0x7f359c19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:41.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a1b11700 1 -- 172.21.15.40:0/3121917875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c1991b0 con 0x7f359c108580 2024-06-06T03:35:41.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a1310700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f359c06ac60 0x7f359c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:35:41.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.804+0000 7f35a1b11700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c108580 0x7f359c198a50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3594007ee0 tx=0x7f359400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:41.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.805+0000 7f35927fc700 1 -- 172.21.15.40:0/3121917875 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359400dbd0 con 0x7f359c108580 2024-06-06T03:35:41.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.805+0000 7f35927fc700 1 -- 172.21.15.40:0/3121917875 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f359400dd70 con 0x7f359c108580 2024-06-06T03:35:41.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.805+0000 7f35927fc700 1 -- 172.21.15.40:0/3121917875 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35940133f0 con 0x7f359c108580 2024-06-06T03:35:41.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.805+0000 7f35a3574700 1 -- 172.21.15.40:0/3121917875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c1994a0 con 0x7f359c108580 2024-06-06T03:35:41.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.805+0000 7f35a3574700 1 -- 172.21.15.40:0/3121917875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c1a5760 con 0x7f359c108580 2024-06-06T03:35:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.806+0000 7f35a3574700 1 -- 172.21.15.40:0/3121917875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359c069180 con 0x7f359c108580 2024-06-06T03:35:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.807+0000 7f35927fc700 1 -- 172.21.15.40:0/3121917875 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f359401a020 con 0x7f359c108580 2024-06-06T03:35:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.808+0000 7f35927fc700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f35880a2c20 0x7f35880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.808+0000 7f35a1310700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f35880a2c20 0x7f35880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.808+0000 7f35927fc700 1 -- 172.21.15.40:0/3121917875 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f35940ce7b0 con 0x7f359c108580 2024-06-06T03:35:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.808+0000 7f35a1310700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f35880a2c20 0x7f35880a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3598005e10 tx=0x7f3598005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:41.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:41.812+0000 7f35927fc700 1 -- 172.21.15.40:0/3121917875 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3594010070 con 0x7f359c108580 2024-06-06T03:35:42.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:42.052+0000 7f35a3574700 1 -- 172.21.15.40:0/3121917875 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f359c0628d0 con 0x7f35880a2c20 2024-06-06T03:35:42.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:42.080+0000 7f35927fc700 1 -- 172.21.15.40:0/3121917875 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f359c0628d0 con 0x7f35880a2c20 2024-06-06T03:35:42.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:42.082+0000 7f35a3574700 1 -- 172.21.15.40:0/3121917875 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f35880a2c20 msgr2=0x7f35880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:42.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:42.082+0000 7f35a3574700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f35880a2c20 0x7f35880a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3598005e10 tx=0x7f3598005d60 comp rx=0 tx=0).stop 2024-06-06T03:35:42.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:42.082+0000 7f35a3574700 1 -- 172.21.15.40:0/3121917875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c108580 msgr2=0x7f359c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:42.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:42.082+0000 7f35a3574700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c108580 0x7f359c198a50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3594007ee0 tx=0x7f359400d3b0 comp rx=0 tx=0).stop 2024-06-06T03:35:42.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:42.082+0000 7f35a3574700 1 -- 172.21.15.40:0/3121917875 shutdown_connections 2024-06-06T03:35:42.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:42.082+0000 7f35a3574700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f35880a2c20 0x7f35880a50e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:42.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:42.082+0000 7f35a3574700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f359c06ac60 0x7f359c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:42.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:42.082+0000 7f35a3574700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c108580 0x7f359c198a50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:42.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:42.082+0000 7f35a3574700 1 --2- 172.21.15.40:0/3121917875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f359c06b620 0x7f359c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:42.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:42.082+0000 7f35a3574700 1 -- 172.21.15.40:0/3121917875 >> 172.21.15.40:0/3121917875 conn(0x7f359c078430 msgr2=0x7f359c0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:42.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:42.082+0000 7f35a3574700 1 -- 172.21.15.40:0/3121917875 shutdown_connections 2024-06-06T03:35:42.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:42.082+0000 7f35a3574700 1 -- 172.21.15.40:0/3121917875 wait complete. 2024-06-06T03:35:42.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:35:42 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:35:42.056+0000 7f4999e8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:35:42.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:35:42 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:35:42.056+0000 7f4999e8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:35:42.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:35:42 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:35:42.056+0000 7f4999e8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:35:42.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:35:42 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:35:42.056+0000 7f4999e8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:35:42.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:35:42 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:35:42.056+0000 7f4999e8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:35:42.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-06T03:35:43.161 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:42 smithi131 ceph-mon[36687]: pgmap v315: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-06-06T03:35:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:42 smithi131 ceph-mon[36687]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:43.287 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:42 smithi040 ceph-mon[32220]: pgmap v315: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-06-06T03:35:43.287 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:42 smithi040 ceph-mon[32220]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:43.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:42 smithi104 ceph-mon[36569]: pgmap v315: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-06-06T03:35:43.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:42 smithi104 ceph-mon[36569]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.863+0000 7f8d5a313700 1 -- 172.21.15.40:0/202425355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d54107570 msgr2=0x7f8d54107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.863+0000 7f8d5a313700 1 --2- 172.21.15.40:0/202425355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d54107570 0x7f8d54107950 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009990 tx=0x7f8d44009ca0 comp rx=0 tx=0).stop 2024-06-06T03:35:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.863+0000 7f8d5a313700 1 -- 172.21.15.40:0/202425355 shutdown_connections 2024-06-06T03:35:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.863+0000 7f8d5a313700 1 --2- 172.21.15.40:0/202425355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d541017d0 0x7f8d5410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.863+0000 7f8d5a313700 1 --2- 172.21.15.40:0/202425355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d54107570 0x7f8d54107950 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.863+0000 7f8d5a313700 1 --2- 172.21.15.40:0/202425355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d54100e30 0x7f8d54101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.863+0000 7f8d5a313700 1 -- 172.21.15.40:0/202425355 >> 172.21.15.40:0/202425355 conn(0x7f8d54076680 msgr2=0x7f8d54076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.863+0000 7f8d5a313700 1 -- 172.21.15.40:0/202425355 shutdown_connections 2024-06-06T03:35:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.864+0000 7f8d5a313700 1 -- 172.21.15.40:0/202425355 wait complete. 2024-06-06T03:35:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.864+0000 7f8d5a313700 1 Processor -- start 2024-06-06T03:35:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.864+0000 7f8d5a313700 1 -- start start 2024-06-06T03:35:44.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d5a313700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d54100e30 0x7f8d5419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:44.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d5a313700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d541017d0 0x7f8d5419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:44.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d5a313700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d54107570 0x7f8d54198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:44.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d5a313700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8d54112800 con 0x7f8d54100e30 2024-06-06T03:35:44.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d5a313700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8d54112680 con 0x7f8d541017d0 2024-06-06T03:35:44.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d5a313700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8d54112980 con 0x7f8d54107570 2024-06-06T03:35:44.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d53fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d54100e30 0x7f8d5419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:44.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d53fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d54100e30 0x7f8d5419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48614/0 (socket says 172.21.15.40:48614) 2024-06-06T03:35:44.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d53fff700 1 -- 172.21.15.40:0/14483647 learned_addr learned my addr 172.21.15.40:0/14483647 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d537fe700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d541017d0 0x7f8d5419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d588b0700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d54107570 0x7f8d54198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d53fff700 1 -- 172.21.15.40:0/14483647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d54107570 msgr2=0x7f8d54198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d53fff700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d54107570 0x7f8d54198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d53fff700 1 -- 172.21.15.40:0/14483647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d541017d0 msgr2=0x7f8d5419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d53fff700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d541017d0 0x7f8d5419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d53fff700 1 -- 172.21.15.40:0/14483647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d541991c0 con 0x7f8d54100e30 2024-06-06T03:35:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.865+0000 7f8d588b0700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d54107570 0x7f8d54198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:35:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.866+0000 7f8d53fff700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d54100e30 0x7f8d5419e850 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8d44000c00 tx=0x7f8d44003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.866+0000 7f8d517fa700 1 -- 172.21.15.40:0/14483647 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44027070 con 0x7f8d54100e30 2024-06-06T03:35:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.866+0000 7f8d517fa700 1 -- 172.21.15.40:0/14483647 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d44004110 con 0x7f8d54100e30 2024-06-06T03:35:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.866+0000 7f8d5a313700 1 -- 172.21.15.40:0/14483647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d54199450 con 0x7f8d54100e30 2024-06-06T03:35:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.866+0000 7f8d517fa700 1 -- 172.21.15.40:0/14483647 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400f460 con 0x7f8d54100e30 2024-06-06T03:35:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.866+0000 7f8d5a313700 1 -- 172.21.15.40:0/14483647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d541a5720 con 0x7f8d54100e30 2024-06-06T03:35:44.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.867+0000 7f8d5a313700 1 -- 172.21.15.40:0/14483647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d54069180 con 0x7f8d54100e30 2024-06-06T03:35:44.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.868+0000 7f8d517fa700 1 -- 172.21.15.40:0/14483647 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8d4402a070 con 0x7f8d54100e30 2024-06-06T03:35:44.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.869+0000 7f8d517fa700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8d3c0a2cb0 0x7f8d3c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:44.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.869+0000 7f8d537fe700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8d3c0a2cb0 0x7f8d3c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:44.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.869+0000 7f8d517fa700 1 -- 172.21.15.40:0/14483647 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f8d440d00c0 con 0x7f8d54100e30 2024-06-06T03:35:44.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.869+0000 7f8d537fe700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8d3c0a2cb0 0x7f8d3c0a5170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8d54076ba0 tx=0x7f8d4800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:44.873+0000 7f8d517fa700 1 -- 172.21.15.40:0/14483647 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8d4405e850 con 0x7f8d54100e30 2024-06-06T03:35:45.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:45.114+0000 7f8d5a313700 1 -- 172.21.15.40:0/14483647 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8d540628d0 con 0x7f8d3c0a2cb0 2024-06-06T03:35:45.114 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:44 smithi040 ceph-mon[32220]: pgmap v316: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-06-06T03:35:45.114 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:44 smithi040 ceph-mon[32220]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-06-06T03:35:45.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:45.141+0000 7f8d517fa700 1 -- 172.21.15.40:0/14483647 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8d540628d0 con 0x7f8d3c0a2cb0 2024-06-06T03:35:45.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:45.143+0000 7f8d5a313700 1 -- 172.21.15.40:0/14483647 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8d3c0a2cb0 msgr2=0x7f8d3c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:45.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:45.143+0000 7f8d5a313700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8d3c0a2cb0 0x7f8d3c0a5170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8d54076ba0 tx=0x7f8d4800a380 comp rx=0 tx=0).stop 2024-06-06T03:35:45.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:45.143+0000 7f8d5a313700 1 -- 172.21.15.40:0/14483647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d54100e30 msgr2=0x7f8d5419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:45.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:45.143+0000 7f8d5a313700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d54100e30 0x7f8d5419e850 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8d44000c00 tx=0x7f8d44003680 comp rx=0 tx=0).stop 2024-06-06T03:35:45.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:45.143+0000 7f8d5a313700 1 -- 172.21.15.40:0/14483647 shutdown_connections 2024-06-06T03:35:45.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:45.143+0000 7f8d5a313700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8d3c0a2cb0 0x7f8d3c0a5170 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:45.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:45.143+0000 7f8d5a313700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d54107570 0x7f8d54198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:45.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:45.143+0000 7f8d5a313700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d54100e30 0x7f8d5419e850 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:45.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:45.143+0000 7f8d5a313700 1 --2- 172.21.15.40:0/14483647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d541017d0 0x7f8d5419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:45.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:45.143+0000 7f8d5a313700 1 -- 172.21.15.40:0/14483647 >> 172.21.15.40:0/14483647 conn(0x7f8d54076680 msgr2=0x7f8d540ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:45.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:45.143+0000 7f8d5a313700 1 -- 172.21.15.40:0/14483647 shutdown_connections 2024-06-06T03:35:45.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:45.143+0000 7f8d5a313700 1 -- 172.21.15.40:0/14483647 wait complete. 2024-06-06T03:35:45.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:44 smithi131 ceph-mon[36687]: pgmap v316: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-06-06T03:35:45.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:44 smithi131 ceph-mon[36687]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-06-06T03:35:45.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:44 smithi104 ceph-mon[36569]: pgmap v316: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-06-06T03:35:45.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:44 smithi104 ceph-mon[36569]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-06-06T03:35:45.830 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-06T03:35:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:45 smithi040 ceph-mon[32220]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:46.091 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:46.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:45 smithi131 ceph-mon[36687]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:46.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:45 smithi104 ceph-mon[36569]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:47.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:46 smithi040 ceph-mon[32220]: pgmap v317: 97 pgs: 97 active+clean; 585 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.1 KiB/s wr, 20 op/s 2024-06-06T03:35:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:46 smithi131 ceph-mon[36687]: pgmap v317: 97 pgs: 97 active+clean; 585 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.1 KiB/s wr, 20 op/s 2024-06-06T03:35:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:46 smithi104 ceph-mon[36569]: pgmap v317: 97 pgs: 97 active+clean; 585 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.1 KiB/s wr, 20 op/s 2024-06-06T03:35:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.766+0000 7f13a67d4700 1 -- 172.21.15.40:0/1048259367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13a006ac60 msgr2=0x7f13a006b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.766+0000 7f13a67d4700 1 --2- 172.21.15.40:0/1048259367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13a006ac60 0x7f13a006b0e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f13940099c0 tx=0x7f1394009cd0 comp rx=0 tx=0).stop 2024-06-06T03:35:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.766+0000 7f13a67d4700 1 -- 172.21.15.40:0/1048259367 shutdown_connections 2024-06-06T03:35:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.766+0000 7f13a67d4700 1 --2- 172.21.15.40:0/1048259367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13a01085b0 0x7f13a0108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.766+0000 7f13a67d4700 1 --2- 172.21.15.40:0/1048259367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13a006b620 0x7f13a010f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.766+0000 7f13a67d4700 1 --2- 172.21.15.40:0/1048259367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13a006ac60 0x7f13a006b0e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.766+0000 7f13a67d4700 1 -- 172.21.15.40:0/1048259367 >> 172.21.15.40:0/1048259367 conn(0x7f13a00772a0 msgr2=0x7f13a00776b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.766+0000 7f13a67d4700 1 -- 172.21.15.40:0/1048259367 shutdown_connections 2024-06-06T03:35:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.766+0000 7f13a67d4700 1 -- 172.21.15.40:0/1048259367 wait complete. 2024-06-06T03:35:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.767+0000 7f13a67d4700 1 Processor -- start 2024-06-06T03:35:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.767+0000 7f13a67d4700 1 -- start start 2024-06-06T03:35:47.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.767+0000 7f13a67d4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13a006ac60 0x7f13a010e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:47.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.767+0000 7f13a67d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13a006b620 0x7f13a010eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:47.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.767+0000 7f13a67d4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13a01085b0 0x7f13a01a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:47.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.767+0000 7f13a67d4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f13a0112880 con 0x7f13a006b620 2024-06-06T03:35:47.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.767+0000 7f13a67d4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f13a0112700 con 0x7f13a006ac60 2024-06-06T03:35:47.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.767+0000 7f13a67d4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f13a0112a00 con 0x7f13a01085b0 2024-06-06T03:35:47.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.767+0000 7f139f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13a006b620 0x7f13a010eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:47.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.767+0000 7f139f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13a006b620 0x7f13a010eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48640/0 (socket says 172.21.15.40:48640) 2024-06-06T03:35:47.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.767+0000 7f139f7fe700 1 -- 172.21.15.40:0/3032068139 learned_addr learned my addr 172.21.15.40:0/3032068139 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:47.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.767+0000 7f139ffff700 1 --2- 172.21.15.40:0/3032068139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13a006ac60 0x7f13a010e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:47.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.768+0000 7f13a4d71700 1 --2- 172.21.15.40:0/3032068139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13a01085b0 0x7f13a01a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:47.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.768+0000 7f139f7fe700 1 -- 172.21.15.40:0/3032068139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13a01085b0 msgr2=0x7f13a01a3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:47.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.768+0000 7f139f7fe700 1 --2- 172.21.15.40:0/3032068139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13a01085b0 0x7f13a01a3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:47.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.768+0000 7f139f7fe700 1 -- 172.21.15.40:0/3032068139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13a006ac60 msgr2=0x7f13a010e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.768+0000 7f139f7fe700 1 --2- 172.21.15.40:0/3032068139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13a006ac60 0x7f13a010e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.768+0000 7f139f7fe700 1 -- 172.21.15.40:0/3032068139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a01a42d0 con 0x7f13a006b620 2024-06-06T03:35:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.768+0000 7f139f7fe700 1 --2- 172.21.15.40:0/3032068139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13a006b620 0x7f13a010eb90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1394009fd0 tx=0x7f1394003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.768+0000 7f139d7fa700 1 -- 172.21.15.40:0/3032068139 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394027070 con 0x7f13a006b620 2024-06-06T03:35:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.768+0000 7f139d7fa700 1 -- 172.21.15.40:0/3032068139 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13940040d0 con 0x7f13a006b620 2024-06-06T03:35:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.768+0000 7f13a67d4700 1 -- 172.21.15.40:0/3032068139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a01a4560 con 0x7f13a006b620 2024-06-06T03:35:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.768+0000 7f139d7fa700 1 -- 172.21.15.40:0/3032068139 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139400f460 con 0x7f13a006b620 2024-06-06T03:35:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.768+0000 7f13a67d4700 1 -- 172.21.15.40:0/3032068139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a01abe40 con 0x7f13a006b620 2024-06-06T03:35:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.769+0000 7f13a67d4700 1 -- 172.21.15.40:0/3032068139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a0103c20 con 0x7f13a006b620 2024-06-06T03:35:47.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.770+0000 7f139d7fa700 1 -- 172.21.15.40:0/3032068139 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f139402a070 con 0x7f13a006b620 2024-06-06T03:35:47.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.771+0000 7f139d7fa700 1 --2- 172.21.15.40:0/3032068139 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f13880a2cb0 0x7f13880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:47.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.771+0000 7f139ffff700 1 --2- 172.21.15.40:0/3032068139 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f13880a2cb0 0x7f13880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:47.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.771+0000 7f139d7fa700 1 -- 172.21.15.40:0/3032068139 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f13940d0750 con 0x7f13a006b620 2024-06-06T03:35:47.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.772+0000 7f139ffff700 1 --2- 172.21.15.40:0/3032068139 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f13880a2cb0 0x7f13880a5170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1390005950 tx=0x7f139000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:47.775+0000 7f139d7fa700 1 -- 172.21.15.40:0/3032068139 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1394031070 con 0x7f13a006b620 2024-06-06T03:35:48.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:48.023+0000 7f13a67d4700 1 -- 172.21.15.40:0/3032068139 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f13a0100640 con 0x7f13880a2cb0 2024-06-06T03:35:48.023 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-06-06T03:35:48.023 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-06-06T03:35:48.023 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-06-06T03:35:48.023 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-06-06T03:35:48.023 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:47 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-06-06T03:35:48.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:48.029+0000 7f139d7fa700 1 -- 172.21.15.40:0/3032068139 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f13a0100640 con 0x7f13880a2cb0 2024-06-06T03:35:48.029 INFO:teuthology.orchestra.run.smithi040.stdout:/volumes/qa/sv_0/db17e9ff-20dc-46c4-ab09-5fd823744903 2024-06-06T03:35:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:48.031+0000 7f13a67d4700 1 -- 172.21.15.40:0/3032068139 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f13880a2cb0 msgr2=0x7f13880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:48.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:48.031+0000 7f13a67d4700 1 --2- 172.21.15.40:0/3032068139 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f13880a2cb0 0x7f13880a5170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1390005950 tx=0x7f139000a300 comp rx=0 tx=0).stop 2024-06-06T03:35:48.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:48.031+0000 7f13a67d4700 1 -- 172.21.15.40:0/3032068139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13a006b620 msgr2=0x7f13a010eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:48.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:48.031+0000 7f13a67d4700 1 --2- 172.21.15.40:0/3032068139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13a006b620 0x7f13a010eb90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1394009fd0 tx=0x7f1394003680 comp rx=0 tx=0).stop 2024-06-06T03:35:48.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:48.031+0000 7f13a67d4700 1 -- 172.21.15.40:0/3032068139 shutdown_connections 2024-06-06T03:35:48.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:48.031+0000 7f13a67d4700 1 --2- 172.21.15.40:0/3032068139 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f13880a2cb0 0x7f13880a5170 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:48.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:48.031+0000 7f13a67d4700 1 --2- 172.21.15.40:0/3032068139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13a01085b0 0x7f13a01a3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:48.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:48.031+0000 7f13a67d4700 1 --2- 172.21.15.40:0/3032068139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13a006ac60 0x7f13a010e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:48.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:48.031+0000 7f13a67d4700 1 --2- 172.21.15.40:0/3032068139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13a006b620 0x7f13a010eb90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:48.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:48.031+0000 7f13a67d4700 1 -- 172.21.15.40:0/3032068139 >> 172.21.15.40:0/3032068139 conn(0x7f13a00772a0 msgr2=0x7f13a00fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:48.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:48.031+0000 7f13a67d4700 1 -- 172.21.15.40:0/3032068139 shutdown_connections 2024-06-06T03:35:48.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:48.031+0000 7f13a67d4700 1 -- 172.21.15.40:0/3032068139 wait complete. 2024-06-06T03:35:48.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:47 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-06-06T03:35:48.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:47 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-06-06T03:35:48.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:47 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-06-06T03:35:48.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:47 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-06-06T03:35:48.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:47 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-06-06T03:35:48.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:47 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-06-06T03:35:48.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:47 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-06-06T03:35:48.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:47 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-06-06T03:35:48.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:47 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-06-06T03:35:48.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:47 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-06-06T03:35:48.724 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-06T03:35:48.978 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:49.251 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:48 smithi040 ceph-mon[32220]: pgmap v318: 97 pgs: 97 active+clean; 605 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.3 KiB/s wr, 17 op/s 2024-06-06T03:35:49.251 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-06-06T03:35:49.251 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-06-06T03:35:49.251 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-06-06T03:35:49.251 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-06-06T03:35:49.251 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:48 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-06-06T03:35:49.251 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:48 smithi040 ceph-mon[32220]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T03:35:49.251 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:48 smithi040 ceph-mon[32220]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:48 smithi131 ceph-mon[36687]: pgmap v318: 97 pgs: 97 active+clean; 605 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.3 KiB/s wr, 17 op/s 2024-06-06T03:35:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-06-06T03:35:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-06-06T03:35:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-06-06T03:35:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-06-06T03:35:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:48 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-06-06T03:35:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:48 smithi131 ceph-mon[36687]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T03:35:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:48 smithi131 ceph-mon[36687]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:49.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:48 smithi104 ceph-mon[36569]: pgmap v318: 97 pgs: 97 active+clean; 605 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.3 KiB/s wr, 17 op/s 2024-06-06T03:35:49.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-06-06T03:35:49.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-06-06T03:35:49.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-06-06T03:35:49.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-06-06T03:35:49.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:48 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-06-06T03:35:49.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:48 smithi104 ceph-mon[36569]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T03:35:49.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:48 smithi104 ceph-mon[36569]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:50.242 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:49 smithi040 ceph-mon[32220]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T03:35:50.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:49 smithi131 ceph-mon[36687]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T03:35:50.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:49 smithi104 ceph-mon[36569]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T03:35:50.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.792+0000 7f9bf7194700 1 -- 172.21.15.40:0/30384161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf0101610 msgr2=0x7f9bf01019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:50.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.792+0000 7f9bf7194700 1 --2- 172.21.15.40:0/30384161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf0101610 0x7f9bf01019f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9be40099c0 tx=0x7f9be4009cd0 comp rx=0 tx=0).stop 2024-06-06T03:35:50.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.792+0000 7f9bf7194700 1 -- 172.21.15.40:0/30384161 shutdown_connections 2024-06-06T03:35:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.792+0000 7f9bf7194700 1 --2- 172.21.15.40:0/30384161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bf01066f0 0x7f9bf010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.792+0000 7f9bf7194700 1 --2- 172.21.15.40:0/30384161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf0101610 0x7f9bf01019f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.792+0000 7f9bf7194700 1 --2- 172.21.15.40:0/30384161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0101fc0 0x7f9bf0106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.792+0000 7f9bf7194700 1 -- 172.21.15.40:0/30384161 >> 172.21.15.40:0/30384161 conn(0x7f9bf0078430 msgr2=0x7f9bf0078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.792+0000 7f9bf7194700 1 -- 172.21.15.40:0/30384161 shutdown_connections 2024-06-06T03:35:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.792+0000 7f9bf7194700 1 -- 172.21.15.40:0/30384161 wait complete. 2024-06-06T03:35:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.792+0000 7f9bf7194700 1 Processor -- start 2024-06-06T03:35:50.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf7194700 1 -- start start 2024-06-06T03:35:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf7194700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0101610 0x7f9bf01030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf7194700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bf0101fc0 0x7f9bf01035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf7194700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf01066f0 0x7f9bf0103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf7194700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf01116f0 con 0x7f9bf01066f0 2024-06-06T03:35:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf7194700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf0111570 con 0x7f9bf0101fc0 2024-06-06T03:35:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf7194700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf0111870 con 0x7f9bf0101610 2024-06-06T03:35:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf5731700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf01066f0 0x7f9bf0103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf5731700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf01066f0 0x7f9bf0103b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35286/0 (socket says 172.21.15.40:35286) 2024-06-06T03:35:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf5731700 1 -- 172.21.15.40:0/4164317782 learned_addr learned my addr 172.21.15.40:0/4164317782 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf4f30700 1 --2- 172.21.15.40:0/4164317782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0101610 0x7f9bf01030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9beffff700 1 --2- 172.21.15.40:0/4164317782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bf0101fc0 0x7f9bf01035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf5731700 1 -- 172.21.15.40:0/4164317782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0101610 msgr2=0x7f9bf01030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf5731700 1 --2- 172.21.15.40:0/4164317782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0101610 0x7f9bf01030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf5731700 1 -- 172.21.15.40:0/4164317782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bf0101fc0 msgr2=0x7f9bf01035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf5731700 1 --2- 172.21.15.40:0/4164317782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bf0101fc0 0x7f9bf01035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf5731700 1 -- 172.21.15.40:0/4164317782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf01a71b0 con 0x7f9bf01066f0 2024-06-06T03:35:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.793+0000 7f9bf5731700 1 --2- 172.21.15.40:0/4164317782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf01066f0 0x7f9bf0103b30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9be8007ee0 tx=0x7f9be800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.794+0000 7f9bedffb700 1 -- 172.21.15.40:0/4164317782 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8004bb0 con 0x7f9bf01066f0 2024-06-06T03:35:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.794+0000 7f9bedffb700 1 -- 172.21.15.40:0/4164317782 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be8004d50 con 0x7f9bf01066f0 2024-06-06T03:35:50.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.794+0000 7f9bf7194700 1 -- 172.21.15.40:0/4164317782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf01a74a0 con 0x7f9bf01066f0 2024-06-06T03:35:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.794+0000 7f9bedffb700 1 -- 172.21.15.40:0/4164317782 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8005960 con 0x7f9bf01066f0 2024-06-06T03:35:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.794+0000 7f9bf7194700 1 -- 172.21.15.40:0/4164317782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf01a7990 con 0x7f9bf01066f0 2024-06-06T03:35:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.795+0000 7f9bf7194700 1 -- 172.21.15.40:0/4164317782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf0069180 con 0x7f9bf01066f0 2024-06-06T03:35:50.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.796+0000 7f9bedffb700 1 -- 172.21.15.40:0/4164317782 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9be801a020 con 0x7f9bf01066f0 2024-06-06T03:35:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.797+0000 7f9bedffb700 1 --2- 172.21.15.40:0/4164317782 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9be00a2cb0 0x7f9be00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.797+0000 7f9bf4f30700 1 --2- 172.21.15.40:0/4164317782 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9be00a2cb0 0x7f9be00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.797+0000 7f9bedffb700 1 -- 172.21.15.40:0/4164317782 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f9be80ce880 con 0x7f9bf01066f0 2024-06-06T03:35:50.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.797+0000 7f9bf4f30700 1 --2- 172.21.15.40:0/4164317782 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9be00a2cb0 0x7f9be00a5170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9be4000c00 tx=0x7f9be4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:50.803+0000 7f9bedffb700 1 -- 172.21.15.40:0/4164317782 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9be8010070 con 0x7f9bf01066f0 2024-06-06T03:35:51.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:51.046+0000 7f9bf7194700 1 -- 172.21.15.40:0/4164317782 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9bf006abc0 con 0x7f9be00a2cb0 2024-06-06T03:35:51.047 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:50 smithi040 ceph-mon[32220]: pgmap v321: 97 pgs: 97 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-06-06T03:35:51.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:51.073+0000 7f9bedffb700 1 -- 172.21.15.40:0/4164317782 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9bf006abc0 con 0x7f9be00a2cb0 2024-06-06T03:35:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:51.076+0000 7f9bf7194700 1 -- 172.21.15.40:0/4164317782 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9be00a2cb0 msgr2=0x7f9be00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:51.076+0000 7f9bf7194700 1 --2- 172.21.15.40:0/4164317782 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9be00a2cb0 0x7f9be00a5170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9be4000c00 tx=0x7f9be4005fb0 comp rx=0 tx=0).stop 2024-06-06T03:35:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:51.076+0000 7f9bf7194700 1 -- 172.21.15.40:0/4164317782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf01066f0 msgr2=0x7f9bf0103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:51.076+0000 7f9bf7194700 1 --2- 172.21.15.40:0/4164317782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf01066f0 0x7f9bf0103b30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9be8007ee0 tx=0x7f9be800d3b0 comp rx=0 tx=0).stop 2024-06-06T03:35:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:51.076+0000 7f9bf7194700 1 -- 172.21.15.40:0/4164317782 shutdown_connections 2024-06-06T03:35:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:51.076+0000 7f9bf7194700 1 --2- 172.21.15.40:0/4164317782 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9be00a2cb0 0x7f9be00a5170 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:51.076+0000 7f9bf7194700 1 --2- 172.21.15.40:0/4164317782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf0101610 0x7f9bf01030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:51.076+0000 7f9bf7194700 1 --2- 172.21.15.40:0/4164317782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf01066f0 0x7f9bf0103b30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:51.076+0000 7f9bf7194700 1 --2- 172.21.15.40:0/4164317782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bf0101fc0 0x7f9bf01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:51.076+0000 7f9bf7194700 1 -- 172.21.15.40:0/4164317782 >> 172.21.15.40:0/4164317782 conn(0x7f9bf0078430 msgr2=0x7f9bf0105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:51.076+0000 7f9bf7194700 1 -- 172.21.15.40:0/4164317782 shutdown_connections 2024-06-06T03:35:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:51.076+0000 7f9bf7194700 1 -- 172.21.15.40:0/4164317782 wait complete. 2024-06-06T03:35:51.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:50 smithi131 ceph-mon[36687]: pgmap v321: 97 pgs: 97 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-06-06T03:35:51.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:50 smithi104 ceph-mon[36569]: pgmap v321: 97 pgs: 97 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-06-06T03:35:51.962 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-06T03:35:52.221 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:52.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:52 smithi131 ceph-mon[36687]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:52.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:51 smithi040 ceph-mon[32220]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:52.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:52 smithi104 ceph-mon[36569]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:53.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:53 smithi131 ceph-mon[36687]: pgmap v322: 97 pgs: 97 active+clean; 625 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-06-06T03:35:53.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:53 smithi040 ceph-mon[32220]: pgmap v322: 97 pgs: 97 active+clean; 625 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-06-06T03:35:53.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:53 smithi104 ceph-mon[36569]: pgmap v322: 97 pgs: 97 active+clean; 625 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-06-06T03:35:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.943+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3485842121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6c0102030 msgr2=0x7fd6c010e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.943+0000 7fd6c7dbd700 1 --2- 172.21.15.40:0/3485842121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6c0102030 0x7fd6c010e270 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009990 tx=0x7fd6b8009ca0 comp rx=0 tx=0).stop 2024-06-06T03:35:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.944+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3485842121 shutdown_connections 2024-06-06T03:35:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.944+0000 7fd6c7dbd700 1 --2- 172.21.15.40:0/3485842121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6c0102030 0x7fd6c010e270 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.944+0000 7fd6c7dbd700 1 --2- 172.21.15.40:0/3485842121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c01063b0 0x7fd6c0106790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.944+0000 7fd6c7dbd700 1 --2- 172.21.15.40:0/3485842121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6c0101670 0x7fd6c0101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.944+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3485842121 >> 172.21.15.40:0/3485842121 conn(0x7fd6c0078250 msgr2=0x7fd6c0078660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.944+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3485842121 shutdown_connections 2024-06-06T03:35:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.944+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3485842121 wait complete. 2024-06-06T03:35:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.944+0000 7fd6c7dbd700 1 Processor -- start 2024-06-06T03:35:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.944+0000 7fd6c7dbd700 1 -- start start 2024-06-06T03:35:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.944+0000 7fd6c7dbd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6c0101670 0x7fd6c0199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c7dbd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6c0102030 0x7fd6c019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c7dbd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c01063b0 0x7fd6c01a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c7dbd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01117f0 con 0x7fd6c0101670 2024-06-06T03:35:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c7dbd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c0111670 con 0x7fd6c01063b0 2024-06-06T03:35:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c7dbd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c0111970 con 0x7fd6c0102030 2024-06-06T03:35:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c5b59700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6c0101670 0x7fd6c0199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c5b59700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6c0101670 0x7fd6c0199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35304/0 (socket says 172.21.15.40:35304) 2024-06-06T03:35:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c5b59700 1 -- 172.21.15.40:0/3586267094 learned_addr learned my addr 172.21.15.40:0/3586267094 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c5358700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6c0102030 0x7fd6c019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c635a700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c01063b0 0x7fd6c01a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c5b59700 1 -- 172.21.15.40:0/3586267094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6c0102030 msgr2=0x7fd6c019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c5b59700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6c0102030 0x7fd6c019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c5b59700 1 -- 172.21.15.40:0/3586267094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c01063b0 msgr2=0x7fd6c01a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c5b59700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c01063b0 0x7fd6c01a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c5b59700 1 -- 172.21.15.40:0/3586267094 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c01a52b0 con 0x7fd6c0101670 2024-06-06T03:35:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c5358700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6c0102030 0x7fd6c019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:35:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c635a700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c01063b0 0x7fd6c01a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:35:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c5b59700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6c0101670 0x7fd6c0199cf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00ada0 tx=0x7fd6bc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6b6ffd700 1 -- 172.21.15.40:0/3586267094 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc017020 con 0x7fd6c0101670 2024-06-06T03:35:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3586267094 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c0197ae0 con 0x7fd6c0101670 2024-06-06T03:35:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6b6ffd700 1 -- 172.21.15.40:0/3586267094 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6bc004bb0 con 0x7fd6c0101670 2024-06-06T03:35:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.946+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3586267094 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c0198040 con 0x7fd6c0101670 2024-06-06T03:35:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.945+0000 7fd6b6ffd700 1 -- 172.21.15.40:0/3586267094 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0055d0 con 0x7fd6c0101670 2024-06-06T03:35:53.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.947+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3586267094 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c01098f0 con 0x7fd6c0101670 2024-06-06T03:35:53.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.948+0000 7fd6b6ffd700 1 -- 172.21.15.40:0/3586267094 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd6bc0057b0 con 0x7fd6c0101670 2024-06-06T03:35:53.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.949+0000 7fd6b6ffd700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd6ac0a2cb0 0x7fd6ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:53.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.949+0000 7fd6c5358700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd6ac0a2cb0 0x7fd6ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:53.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.949+0000 7fd6b6ffd700 1 -- 172.21.15.40:0/3586267094 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fd6bc0ce8e0 con 0x7fd6c0101670 2024-06-06T03:35:53.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.950+0000 7fd6c5358700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd6ac0a2cb0 0x7fd6ac0a5170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0005fd0 tx=0x7fd6b0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:53.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:53.954+0000 7fd6b6ffd700 1 -- 172.21.15.40:0/3586267094 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd6bc021020 con 0x7fd6c0101670 2024-06-06T03:35:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:54.195+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3586267094 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fd6c0069180 con 0x7fd6ac0a2cb0 2024-06-06T03:35:54.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:54.200+0000 7fd6b6ffd700 1 -- 172.21.15.40:0/3586267094 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd6c0069180 con 0x7fd6ac0a2cb0 2024-06-06T03:35:54.200 INFO:teuthology.orchestra.run.smithi040.stdout:/volumes/qa/sv_1/5cc6513e-156c-4d04-8f86-bd2c9b139dd9 2024-06-06T03:35:54.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:54.203+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3586267094 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd6ac0a2cb0 msgr2=0x7fd6ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:54.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:54.203+0000 7fd6c7dbd700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd6ac0a2cb0 0x7fd6ac0a5170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0005fd0 tx=0x7fd6b0005e30 comp rx=0 tx=0).stop 2024-06-06T03:35:54.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:54.203+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3586267094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6c0101670 msgr2=0x7fd6c0199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:54.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:54.203+0000 7fd6c7dbd700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6c0101670 0x7fd6c0199cf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00ada0 tx=0x7fd6bc00c330 comp rx=0 tx=0).stop 2024-06-06T03:35:54.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:54.203+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3586267094 shutdown_connections 2024-06-06T03:35:54.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:54.203+0000 7fd6c7dbd700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd6ac0a2cb0 0x7fd6ac0a5170 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:54.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:54.203+0000 7fd6c7dbd700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6c01063b0 0x7fd6c01a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:54.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:54.203+0000 7fd6c7dbd700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6c0101670 0x7fd6c0199cf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:54.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:54.203+0000 7fd6c7dbd700 1 --2- 172.21.15.40:0/3586267094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6c0102030 0x7fd6c019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:54.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:54.203+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3586267094 >> 172.21.15.40:0/3586267094 conn(0x7fd6c0078250 msgr2=0x7fd6c00ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:54.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:54.203+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3586267094 shutdown_connections 2024-06-06T03:35:54.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:54.203+0000 7fd6c7dbd700 1 -- 172.21.15.40:0/3586267094 wait complete. 2024-06-06T03:35:55.161 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:35:55.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:55 smithi131 ceph-mon[36687]: pgmap v323: 97 pgs: 97 active+clean; 625 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s; 13 B/s, 1 objects/s recovering 2024-06-06T03:35:55.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:55 smithi131 ceph-mon[36687]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:55.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:55 smithi040 ceph-mon[32220]: pgmap v323: 97 pgs: 97 active+clean; 625 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s; 13 B/s, 1 objects/s recovering 2024-06-06T03:35:55.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:55 smithi040 ceph-mon[32220]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:55.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:55 smithi104 ceph-mon[36569]: pgmap v323: 97 pgs: 97 active+clean; 625 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s; 13 B/s, 1 objects/s recovering 2024-06-06T03:35:55.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:55 smithi104 ceph-mon[36569]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:35:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:57.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:57 smithi040 ceph-mon[32220]: pgmap v324: 97 pgs: 97 active+clean; 625 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2024-06-06T03:35:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.194+0000 7f29256ab700 1 -- 172.21.15.40:0/726265146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f292006ac60 msgr2=0x7f292006b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.194+0000 7f29256ab700 1 --2- 172.21.15.40:0/726265146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f292006ac60 0x7f292006b0e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f29080099c0 tx=0x7f2908009cd0 comp rx=0 tx=0).stop 2024-06-06T03:35:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.194+0000 7f29256ab700 1 -- 172.21.15.40:0/726265146 shutdown_connections 2024-06-06T03:35:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.194+0000 7f29256ab700 1 --2- 172.21.15.40:0/726265146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f292006b620 0x7f292010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.194+0000 7f29256ab700 1 --2- 172.21.15.40:0/726265146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2920108580 0x7f2920108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.194+0000 7f29256ab700 1 --2- 172.21.15.40:0/726265146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f292006ac60 0x7f292006b0e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.194+0000 7f29256ab700 1 -- 172.21.15.40:0/726265146 >> 172.21.15.40:0/726265146 conn(0x7f2920078430 msgr2=0x7f2920078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.194+0000 7f29256ab700 1 -- 172.21.15.40:0/726265146 shutdown_connections 2024-06-06T03:35:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.194+0000 7f29256ab700 1 -- 172.21.15.40:0/726265146 wait complete. 2024-06-06T03:35:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.195+0000 7f29256ab700 1 Processor -- start 2024-06-06T03:35:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.195+0000 7f29256ab700 1 -- start start 2024-06-06T03:35:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.195+0000 7f29256ab700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f292006ac60 0x7f292019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.195+0000 7f29256ab700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f292006b620 0x7f292019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.195+0000 7f29256ab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2920108580 0x7f2920198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.195+0000 7f29256ab700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2920112830 con 0x7f2920108580 2024-06-06T03:35:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.195+0000 7f29256ab700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f29201126b0 con 0x7f292006b620 2024-06-06T03:35:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.195+0000 7f29256ab700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f29201129b0 con 0x7f292006ac60 2024-06-06T03:35:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f291f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2920108580 0x7f2920198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f291f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2920108580 0x7f2920198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35336/0 (socket says 172.21.15.40:35336) 2024-06-06T03:35:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f291f7fe700 1 -- 172.21.15.40:0/3907311414 learned_addr learned my addr 172.21.15.40:0/3907311414 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:35:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f291effd700 1 --2- 172.21.15.40:0/3907311414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f292006ac60 0x7f292019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f291e7fc700 1 --2- 172.21.15.40:0/3907311414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f292006b620 0x7f292019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f291f7fe700 1 -- 172.21.15.40:0/3907311414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f292006ac60 msgr2=0x7f292019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f291f7fe700 1 --2- 172.21.15.40:0/3907311414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f292006ac60 0x7f292019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f291f7fe700 1 -- 172.21.15.40:0/3907311414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f292006b620 msgr2=0x7f292019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f291f7fe700 1 --2- 172.21.15.40:0/3907311414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f292006b620 0x7f292019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f291f7fe700 1 -- 172.21.15.40:0/3907311414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2920199170 con 0x7f2920108580 2024-06-06T03:35:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f291f7fe700 1 --2- 172.21.15.40:0/3907311414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2920108580 0x7f2920198a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f291800b550 tx=0x7f291800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f2917fff700 1 -- 172.21.15.40:0/3907311414 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918015020 con 0x7f2920108580 2024-06-06T03:35:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f2917fff700 1 -- 172.21.15.40:0/3907311414 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2918011d10 con 0x7f2920108580 2024-06-06T03:35:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f2917fff700 1 -- 172.21.15.40:0/3907311414 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291800f2e0 con 0x7f2920108580 2024-06-06T03:35:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f29256ab700 1 -- 172.21.15.40:0/3907311414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2920199460 con 0x7f2920108580 2024-06-06T03:35:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.196+0000 7f29256ab700 1 -- 172.21.15.40:0/3907311414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29201a5720 con 0x7f2920108580 2024-06-06T03:35:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.197+0000 7f29256ab700 1 -- 172.21.15.40:0/3907311414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2920069180 con 0x7f2920108580 2024-06-06T03:35:57.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.199+0000 7f2917fff700 1 -- 172.21.15.40:0/3907311414 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2918018020 con 0x7f2920108580 2024-06-06T03:35:57.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.199+0000 7f2917fff700 1 --2- 172.21.15.40:0/3907311414 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f290c0a2d30 0x7f290c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:35:57.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.199+0000 7f291effd700 1 --2- 172.21.15.40:0/3907311414 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f290c0a2d30 0x7f290c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:35:57.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.199+0000 7f2917fff700 1 -- 172.21.15.40:0/3907311414 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f29180ce680 con 0x7f2920108580 2024-06-06T03:35:57.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.200+0000 7f291effd700 1 --2- 172.21.15.40:0/3907311414 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f290c0a2d30 0x7f290c0a51f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2910009550 tx=0x7f29100091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:35:57.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.204+0000 7f2917fff700 1 -- 172.21.15.40:0/3907311414 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f291801f020 con 0x7f2920108580 2024-06-06T03:35:57.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:57 smithi131 ceph-mon[36687]: pgmap v324: 97 pgs: 97 active+clean; 625 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2024-06-06T03:35:57.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:57 smithi104 ceph-mon[36569]: pgmap v324: 97 pgs: 97 active+clean; 625 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2024-06-06T03:35:57.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.456+0000 7f29256ab700 1 -- 172.21.15.40:0/3907311414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f292002dca0 con 0x7f2920108580 2024-06-06T03:35:57.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.457+0000 7f2917fff700 1 -- 172.21.15.40:0/3907311414 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11933 (secure 0 0 0) 0x7f291809b6b0 con 0x7f2920108580 2024-06-06T03:35:57.457 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:35:57.457 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:35:38.053505+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24398,24403,24425,24443,24470]},"id":1}]} 2024-06-06T03:35:57.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.460+0000 7f29256ab700 1 -- 172.21.15.40:0/3907311414 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f290c0a2d30 msgr2=0x7f290c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:57.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.460+0000 7f29256ab700 1 --2- 172.21.15.40:0/3907311414 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f290c0a2d30 0x7f290c0a51f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2910009550 tx=0x7f29100091b0 comp rx=0 tx=0).stop 2024-06-06T03:35:57.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.460+0000 7f29256ab700 1 -- 172.21.15.40:0/3907311414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2920108580 msgr2=0x7f2920198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:35:57.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.460+0000 7f29256ab700 1 --2- 172.21.15.40:0/3907311414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2920108580 0x7f2920198a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f291800b550 tx=0x7f291800b860 comp rx=0 tx=0).stop 2024-06-06T03:35:57.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.460+0000 7f29256ab700 1 -- 172.21.15.40:0/3907311414 shutdown_connections 2024-06-06T03:35:57.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.460+0000 7f29256ab700 1 --2- 172.21.15.40:0/3907311414 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f290c0a2d30 0x7f290c0a51f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:57.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.460+0000 7f29256ab700 1 --2- 172.21.15.40:0/3907311414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f292006ac60 0x7f292019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:57.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.460+0000 7f29256ab700 1 --2- 172.21.15.40:0/3907311414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2920108580 0x7f2920198a10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:57.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.460+0000 7f29256ab700 1 --2- 172.21.15.40:0/3907311414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f292006b620 0x7f292019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:35:57.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.460+0000 7f29256ab700 1 -- 172.21.15.40:0/3907311414 >> 172.21.15.40:0/3907311414 conn(0x7f2920078430 msgr2=0x7f29200ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:35:57.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.460+0000 7f29256ab700 1 -- 172.21.15.40:0/3907311414 shutdown_connections 2024-06-06T03:35:57.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:35:57.460+0000 7f29256ab700 1 -- 172.21.15.40:0/3907311414 wait complete. 2024-06-06T03:35:57.461 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 28 2024-06-06T03:35:58.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:58 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3907311414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:35:58.133 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T03:35:58.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:58 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3907311414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:35:58.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:58 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3907311414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:35:58.383 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:35:59.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:35:59 smithi040 ceph-mon[32220]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 12 op/s; 11 B/s, 1 objects/s recovering 2024-06-06T03:35:59.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:35:59 smithi131 ceph-mon[36687]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 12 op/s; 11 B/s, 1 objects/s recovering 2024-06-06T03:35:59.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:35:59 smithi104 ceph-mon[36569]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 12 op/s; 11 B/s, 1 objects/s recovering 2024-06-06T03:36:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.065+0000 7efe2ba22700 1 -- 172.21.15.40:0/1430697463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe24107400 msgr2=0x7efe241077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.065+0000 7efe2ba22700 1 --2- 172.21.15.40:0/1430697463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe24107400 0x7efe241077e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efe20009990 tx=0x7efe20009ca0 comp rx=0 tx=0).stop 2024-06-06T03:36:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.065+0000 7efe2ba22700 1 -- 172.21.15.40:0/1430697463 shutdown_connections 2024-06-06T03:36:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.065+0000 7efe2ba22700 1 --2- 172.21.15.40:0/1430697463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe24101d50 0x7efe2410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.065+0000 7efe2ba22700 1 --2- 172.21.15.40:0/1430697463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe24107400 0x7efe241077e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.065+0000 7efe2ba22700 1 --2- 172.21.15.40:0/1430697463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe24101390 0x7efe24101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.065+0000 7efe2ba22700 1 -- 172.21.15.40:0/1430697463 >> 172.21.15.40:0/1430697463 conn(0x7efe24078340 msgr2=0x7efe24078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.065+0000 7efe2ba22700 1 -- 172.21.15.40:0/1430697463 shutdown_connections 2024-06-06T03:36:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.065+0000 7efe2ba22700 1 -- 172.21.15.40:0/1430697463 wait complete. 2024-06-06T03:36:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.066+0000 7efe2ba22700 1 Processor -- start 2024-06-06T03:36:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.066+0000 7efe2ba22700 1 -- start start 2024-06-06T03:36:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.066+0000 7efe2ba22700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe24101390 0x7efe2419d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.066+0000 7efe2ba22700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe24101d50 0x7efe2419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.066+0000 7efe2ba22700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe24107400 0x7efe24197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.066+0000 7efe2ba22700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efe241116f0 con 0x7efe24101d50 2024-06-06T03:36:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.066+0000 7efe2ba22700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efe24111570 con 0x7efe24107400 2024-06-06T03:36:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.066+0000 7efe2ba22700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efe24111870 con 0x7efe24101390 2024-06-06T03:36:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.066+0000 7efe28fbd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe24101d50 0x7efe2419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.066+0000 7efe28fbd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe24101d50 0x7efe2419dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57730/0 (socket says 172.21.15.40:57730) 2024-06-06T03:36:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.066+0000 7efe28fbd700 1 -- 172.21.15.40:0/2061798412 learned_addr learned my addr 172.21.15.40:0/2061798412 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.067+0000 7efe29fbf700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe24107400 0x7efe24197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.067+0000 7efe297be700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe24101390 0x7efe2419d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.067+0000 7efe28fbd700 1 -- 172.21.15.40:0/2061798412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe24101390 msgr2=0x7efe2419d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.067+0000 7efe28fbd700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe24101390 0x7efe2419d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.067+0000 7efe28fbd700 1 -- 172.21.15.40:0/2061798412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe24107400 msgr2=0x7efe24197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.067+0000 7efe28fbd700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe24107400 0x7efe24197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.067+0000 7efe28fbd700 1 -- 172.21.15.40:0/2061798412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe24198170 con 0x7efe24101d50 2024-06-06T03:36:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.067+0000 7efe297be700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe24101390 0x7efe2419d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:36:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.067+0000 7efe28fbd700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe24101d50 0x7efe2419dd40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efe1400d9e0 tx=0x7efe1400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.067+0000 7efe1a7fc700 1 -- 172.21.15.40:0/2061798412 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1401b070 con 0x7efe24101d50 2024-06-06T03:36:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.067+0000 7efe1a7fc700 1 -- 172.21.15.40:0/2061798412 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe1400ecd0 con 0x7efe24101d50 2024-06-06T03:36:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.067+0000 7efe2ba22700 1 -- 172.21.15.40:0/2061798412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe24198460 con 0x7efe24101d50 2024-06-06T03:36:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.067+0000 7efe1a7fc700 1 -- 172.21.15.40:0/2061798412 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14014760 con 0x7efe24101d50 2024-06-06T03:36:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.067+0000 7efe2ba22700 1 -- 172.21.15.40:0/2061798412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe2406a170 con 0x7efe24101d50 2024-06-06T03:36:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.068+0000 7efe2ba22700 1 -- 172.21.15.40:0/2061798412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe24069180 con 0x7efe24101d50 2024-06-06T03:36:00.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.069+0000 7efe1a7fc700 1 -- 172.21.15.40:0/2061798412 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efe1401e070 con 0x7efe24101d50 2024-06-06T03:36:00.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.070+0000 7efe1a7fc700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7efe100a2cb0 0x7efe100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:00.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.070+0000 7efe297be700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7efe100a2cb0 0x7efe100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:00.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.070+0000 7efe1a7fc700 1 -- 172.21.15.40:0/2061798412 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7efe140d06d0 con 0x7efe24101d50 2024-06-06T03:36:00.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.070+0000 7efe297be700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7efe100a2cb0 0x7efe100a5170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efe20000c00 tx=0x7efe20005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.075+0000 7efe1a7fc700 1 -- 172.21.15.40:0/2061798412 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efe1409d690 con 0x7efe24101d50 2024-06-06T03:36:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.280+0000 7efe2ba22700 1 -- 172.21.15.40:0/2061798412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe2406ac00 con 0x7efe24101d50 2024-06-06T03:36:00.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.283+0000 7efe1a7fc700 1 -- 172.21.15.40:0/2061798412 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22534 (secure 0 0 0) 0x7efe1409cf90 con 0x7efe24101d50 2024-06-06T03:36:00.283 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:36:00.285 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"fsid":"77c25d68-23b4-11ef-bc9b-c7b262605968","created":"2024-06-06T03:26:53.165489+0000","modified":"2024-06-06T03:35:48.980461+0000","last_up_change":"2024-06-06T03:34:07.114448+0000","last_in_change":"2024-06-06T03:33:46.152948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T03:30:14.102785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T03:35:08.521025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T03:35:12.369035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63c0a320-a2f2-4b84-87e7-68501d335285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6803","nonce":3453526793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6805","nonce":3453526793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6809","nonce":3453526793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6807","nonce":3453526793}]},"public_addr":"172.21.15.40:6803/3453526793","cluster_addr":"172.21.15.40:6805/3453526793","heartbeat_back_addr":"172.21.15.40:6809/3453526793","heartbeat_front_addr":"172.21.15.40:6807/3453526793","state":["exists","up"]},{"osd":1,"uuid":"70a31534-06a1-4ffa-a54d-577d45bccf98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6801","nonce":405854252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6803","nonce":405854252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6807","nonce":405854252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6805","nonce":405854252}]},"public_addr":"172.21.15.104:6801/405854252","cluster_addr":"172.21.15.104:6803/405854252","heartbeat_back_addr":"172.21.15.104:6807/405854252","heartbeat_front_addr":"172.21.15.104:6805/405854252","state":["exists","up"]},{"osd":2,"uuid":"f5a89f43-e533-45f0-a777-bd3e9bfa8072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6801","nonce":3919221141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6803","nonce":3919221141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6807","nonce":3919221141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6805","nonce":3919221141}]},"public_addr":"172.21.15.131:6801/3919221141","cluster_addr":"172.21.15.131:6803/3919221141","heartbeat_back_addr":"172.21.15.131:6807/3919221141","heartbeat_front_addr":"172.21.15.131:6805/3919221141","state":["exists","up"]},{"osd":3,"uuid":"4df19390-6378-4346-a2be-29bce1dd2380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6811","nonce":2008541522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6813","nonce":2008541522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6817","nonce":2008541522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6815","nonce":2008541522}]},"public_addr":"172.21.15.40:6811/2008541522","cluster_addr":"172.21.15.40:6813/2008541522","heartbeat_back_addr":"172.21.15.40:6817/2008541522","heartbeat_front_addr":"172.21.15.40:6815/2008541522","state":["exists","up"]},{"osd":4,"uuid":"508de408-1cb1-4423-8199-db6bde9034f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6809","nonce":1153394847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6811","nonce":1153394847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6815","nonce":1153394847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6813","nonce":1153394847}]},"public_addr":"172.21.15.104:6809/1153394847","cluster_addr":"172.21.15.104:6811/1153394847","heartbeat_back_addr":"172.21.15.104:6815/1153394847","heartbeat_front_addr":"172.21.15.104:6813/1153394847","state":["exists","up"]},{"osd":5,"uuid":"a8c0cec9-a8ad-4a5d-800b-8b891ca6d836","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6809","nonce":1051933303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6811","nonce":1051933303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6815","nonce":1051933303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6813","nonce":1051933303}]},"public_addr":"172.21.15.131:6809/1051933303","cluster_addr":"172.21.15.131:6811/1051933303","heartbeat_back_addr":"172.21.15.131:6815/1051933303","heartbeat_front_addr":"172.21.15.131:6813/1051933303","state":["exists","up"]},{"osd":6,"uuid":"5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6819","nonce":2670922011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6821","nonce":2670922011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6825","nonce":2670922011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6823","nonce":2670922011}]},"public_addr":"172.21.15.40:6819/2670922011","cluster_addr":"172.21.15.40:6821/2670922011","heartbeat_back_addr":"172.21.15.40:6825/2670922011","heartbeat_front_addr":"172.21.15.40:6823/2670922011","state":["exists","up"]},{"osd":7,"uuid":"bfc536bb-6ae4-459e-afd9-8197c7420bd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6817","nonce":2353571051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6819","nonce":2353571051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6823","nonce":2353571051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6821","nonce":2353571051}]},"public_addr":"172.21.15.104:6817/2353571051","cluster_addr":"172.21.15.104:6819/2353571051","heartbeat_back_addr":"172.21.15.104:6823/2353571051","heartbeat_front_addr":"172.21.15.104:6821/2353571051","state":["exists","up"]},{"osd":8,"uuid":"1fe214f2-5ea3-4fc5-83bf-d9bccafe1529","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6817","nonce":1364533794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6819","nonce":1364533794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6823","nonce":1364533794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6821","nonce":1364533794}]},"public_addr":"172.21.15.131:6817/1364533794","cluster_addr":"172.21.15.131:6819/1364533794","heartbeat_back_addr":"172.21.15.131:6823/1364533794","heartbeat_front_addr":"172.21.15.131:6821/1364533794","state":["exists","up"]},{"osd":9,"uuid":"4b2b3555-6870-4ead-be50-b635799265a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6827","nonce":2604133190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6829","nonce":2604133190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6833","nonce":2604133190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6831","nonce":2604133190}]},"public_addr":"172.21.15.40:6827/2604133190","cluster_addr":"172.21.15.40:6829/2604133190","heartbeat_back_addr":"172.21.15.40:6833/2604133190","heartbeat_front_addr":"172.21.15.40:6831/2604133190","state":["exists","up"]},{"osd":10,"uuid":"70049e24-94ce-458e-97e3-dbcf9de05444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6825","nonce":2231164832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6827","nonce":2231164832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6831","nonce":2231164832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6829","nonce":2231164832}]},"public_addr":"172.21.15.104:6825/2231164832","cluster_addr":"172.21.15.104:6827/2231164832","heartbeat_back_addr":"172.21.15.104:6831/2231164832","heartbeat_front_addr":"172.21.15.104:6829/2231164832","state":["exists","up"]},{"osd":11,"uuid":"80447548-1fbe-4892-ac13-eb4bb6acf10d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6825","nonce":4128185670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6827","nonce":4128185670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6831","nonce":4128185670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6829","nonce":4128185670}]},"public_addr":"172.21.15.131:6825/4128185670","cluster_addr":"172.21.15.131:6827/4128185670","heartbeat_back_addr":"172.21.15.131:6831/4128185670","heartbeat_front_addr":"172.21.15.131:6829/4128185670","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:23.289784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:47.271677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:10.866792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:36.781924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:01.995848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:26.904606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:52.997438+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:19.221567+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:44.543977+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:11.948545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:38.402943+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:34:04.712200+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/23942578":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2234980855":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6801/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6800/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2301695357":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/1676880727":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6801/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6800/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/1597665640":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/513375601":"2024-06-07T03:27:19.799499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T03:36:00.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.286+0000 7efe2ba22700 1 -- 172.21.15.40:0/2061798412 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7efe100a2cb0 msgr2=0x7efe100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:00.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.286+0000 7efe2ba22700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7efe100a2cb0 0x7efe100a5170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efe20000c00 tx=0x7efe20005fb0 comp rx=0 tx=0).stop 2024-06-06T03:36:00.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.286+0000 7efe2ba22700 1 -- 172.21.15.40:0/2061798412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe24101d50 msgr2=0x7efe2419dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.286+0000 7efe2ba22700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe24101d50 0x7efe2419dd40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efe1400d9e0 tx=0x7efe1400dda0 comp rx=0 tx=0).stop 2024-06-06T03:36:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.286+0000 7efe2ba22700 1 -- 172.21.15.40:0/2061798412 shutdown_connections 2024-06-06T03:36:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.286+0000 7efe2ba22700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7efe100a2cb0 0x7efe100a5170 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.286+0000 7efe2ba22700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe24101390 0x7efe2419d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.286+0000 7efe2ba22700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe24107400 0x7efe24197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.286+0000 7efe2ba22700 1 --2- 172.21.15.40:0/2061798412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe24101d50 0x7efe2419dd40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.286+0000 7efe2ba22700 1 -- 172.21.15.40:0/2061798412 >> 172.21.15.40:0/2061798412 conn(0x7efe24078340 msgr2=0x7efe24100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.286+0000 7efe2ba22700 1 -- 172.21.15.40:0/2061798412 shutdown_connections 2024-06-06T03:36:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:00.286+0000 7efe2ba22700 1 -- 172.21.15.40:0/2061798412 wait complete. 2024-06-06T03:36:00.944 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-06T03:36:00.944 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T03:36:01.195 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:01.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:01 smithi131 ceph-mon[36687]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.9 KiB/s wr, 12 op/s; 10 B/s, 0 objects/s recovering 2024-06-06T03:36:01.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:01 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2061798412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:36:01.318 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:01 smithi040 ceph-mon[32220]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.9 KiB/s wr, 12 op/s; 10 B/s, 0 objects/s recovering 2024-06-06T03:36:01.318 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:01 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2061798412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:36:01.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:01 smithi104 ceph-mon[36569]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.9 KiB/s wr, 12 op/s; 10 B/s, 0 objects/s recovering 2024-06-06T03:36:01.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:01 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2061798412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:36:02.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.649+0000 7f0e570d8700 1 -- 172.21.15.40:0/442782501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e501027a0 msgr2=0x7f0e50106020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.649+0000 7f0e570d8700 1 --2- 172.21.15.40:0/442782501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e501027a0 0x7f0e50106020 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000b0d0 tx=0x7f0e4000b3e0 comp rx=0 tx=0).stop 2024-06-06T03:36:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.649+0000 7f0e570d8700 1 -- 172.21.15.40:0/442782501 shutdown_connections 2024-06-06T03:36:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.649+0000 7f0e570d8700 1 --2- 172.21.15.40:0/442782501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e50106750 0x7f0e5010e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.649+0000 7f0e570d8700 1 --2- 172.21.15.40:0/442782501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e50101df0 0x7f0e501021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.649+0000 7f0e570d8700 1 --2- 172.21.15.40:0/442782501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e501027a0 0x7f0e50106020 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.649+0000 7f0e570d8700 1 -- 172.21.15.40:0/442782501 >> 172.21.15.40:0/442782501 conn(0x7f0e50077a00 msgr2=0x7f0e500fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.649+0000 7f0e570d8700 1 -- 172.21.15.40:0/442782501 shutdown_connections 2024-06-06T03:36:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.649+0000 7f0e570d8700 1 -- 172.21.15.40:0/442782501 wait complete. 2024-06-06T03:36:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e570d8700 1 Processor -- start 2024-06-06T03:36:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e570d8700 1 -- start start 2024-06-06T03:36:02.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e570d8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e50101df0 0x7f0e500fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:02.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e570d8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e501027a0 0x7f0e500fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:02.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e570d8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e50106750 0x7f0e500fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:02.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e570d8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0e501116b0 con 0x7f0e50101df0 2024-06-06T03:36:02.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e570d8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0e50111530 con 0x7f0e501027a0 2024-06-06T03:36:02.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e570d8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0e50111830 con 0x7f0e50106750 2024-06-06T03:36:02.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e54e74700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e50101df0 0x7f0e500fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:02.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e54e74700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e50101df0 0x7f0e500fce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57744/0 (socket says 172.21.15.40:57744) 2024-06-06T03:36:02.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e54e74700 1 -- 172.21.15.40:0/2949525265 learned_addr learned my addr 172.21.15.40:0/2949525265 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:02.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e55675700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e50106750 0x7f0e500fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:02.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e54e74700 1 -- 172.21.15.40:0/2949525265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e50106750 msgr2=0x7f0e500fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e54e74700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e50106750 0x7f0e500fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e54e74700 1 -- 172.21.15.40:0/2949525265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e501027a0 msgr2=0x7f0e500fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e4ffff700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e501027a0 0x7f0e500fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e54e74700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e501027a0 0x7f0e500fd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e54e74700 1 -- 172.21.15.40:0/2949525265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e501a71b0 con 0x7f0e50101df0 2024-06-06T03:36:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.650+0000 7f0e55675700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e50106750 0x7f0e500fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:36:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.651+0000 7f0e54e74700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e50101df0 0x7f0e500fce70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800d950 tx=0x7f0e3800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.651+0000 7f0e4dffb700 1 -- 172.21.15.40:0/2949525265 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38016070 con 0x7f0e50101df0 2024-06-06T03:36:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.651+0000 7f0e4dffb700 1 -- 172.21.15.40:0/2949525265 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e38009b40 con 0x7f0e50101df0 2024-06-06T03:36:02.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.651+0000 7f0e570d8700 1 -- 172.21.15.40:0/2949525265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e501a74a0 con 0x7f0e50101df0 2024-06-06T03:36:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.651+0000 7f0e4dffb700 1 -- 172.21.15.40:0/2949525265 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3800f620 con 0x7f0e50101df0 2024-06-06T03:36:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.651+0000 7f0e570d8700 1 -- 172.21.15.40:0/2949525265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e501a79e0 con 0x7f0e50101df0 2024-06-06T03:36:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.652+0000 7f0e570d8700 1 -- 172.21.15.40:0/2949525265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e50069180 con 0x7f0e50101df0 2024-06-06T03:36:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.653+0000 7f0e4dffb700 1 -- 172.21.15.40:0/2949525265 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0e38019070 con 0x7f0e50101df0 2024-06-06T03:36:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.654+0000 7f0e4dffb700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0e3c0a2cb0 0x7f0e3c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.654+0000 7f0e4dffb700 1 -- 172.21.15.40:0/2949525265 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f0e380cf0d0 con 0x7f0e50101df0 2024-06-06T03:36:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.654+0000 7f0e4ffff700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0e3c0a2cb0 0x7f0e3c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.654+0000 7f0e4ffff700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0e3c0a2cb0 0x7f0e3c0a5170 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e50104080 tx=0x7f0e4000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:02.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.657+0000 7f0e4dffb700 1 -- 172.21.15.40:0/2949525265 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0e3809bff0 con 0x7f0e50101df0 2024-06-06T03:36:02.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.860+0000 7f0e570d8700 1 -- 172.21.15.40:0/2949525265 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e500628d0 con 0x7f0e3c0a2cb0 2024-06-06T03:36:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.868+0000 7f0e4dffb700 1 -- 172.21.15.40:0/2949525265 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258940 (secure 0 0 0) 0x7f0e500628d0 con 0x7f0e3c0a2cb0 2024-06-06T03:36:02.871 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:36:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.870+0000 7f0e570d8700 1 -- 172.21.15.40:0/2949525265 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0e3c0a2cb0 msgr2=0x7f0e3c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.870+0000 7f0e570d8700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0e3c0a2cb0 0x7f0e3c0a5170 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e50104080 tx=0x7f0e4000b040 comp rx=0 tx=0).stop 2024-06-06T03:36:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.870+0000 7f0e570d8700 1 -- 172.21.15.40:0/2949525265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e50101df0 msgr2=0x7f0e500fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.870+0000 7f0e570d8700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e50101df0 0x7f0e500fce70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800d950 tx=0x7f0e3800dd10 comp rx=0 tx=0).stop 2024-06-06T03:36:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.871+0000 7f0e570d8700 1 -- 172.21.15.40:0/2949525265 shutdown_connections 2024-06-06T03:36:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.871+0000 7f0e570d8700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0e3c0a2cb0 0x7f0e3c0a5170 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.871+0000 7f0e570d8700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e50106750 0x7f0e500fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.871+0000 7f0e570d8700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e50101df0 0x7f0e500fce70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.871+0000 7f0e570d8700 1 --2- 172.21.15.40:0/2949525265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e501027a0 0x7f0e500fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.871+0000 7f0e570d8700 1 -- 172.21.15.40:0/2949525265 >> 172.21.15.40:0/2949525265 conn(0x7f0e50077a00 msgr2=0x7f0e50100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.871+0000 7f0e570d8700 1 -- 172.21.15.40:0/2949525265 shutdown_connections 2024-06-06T03:36:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:02.871+0000 7f0e570d8700 1 -- 172.21.15.40:0/2949525265 wait complete. 2024-06-06T03:36:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-06-06T03:36:03.239 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:03 smithi040 ceph-mon[32220]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.7 KiB/s wr, 13 op/s; 9 B/s, 0 objects/s recovering 2024-06-06T03:36:03.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:03 smithi131 ceph-mon[36687]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.7 KiB/s wr, 13 op/s; 9 B/s, 0 objects/s recovering 2024-06-06T03:36:03.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:03 smithi104 ceph-mon[36569]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.7 KiB/s wr, 13 op/s; 9 B/s, 0 objects/s recovering 2024-06-06T03:36:03.587 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-06-06T03:36:01.690552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655881,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":432,"num_read_kb":342,"num_write":425,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365476,"kb_used_data":12140,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576348,"statfs":{"total":1151940427776,"available":1151566180352,"internally_reserved":0,"allocated":12431360,"data_stored":7714865,"data_compressed":107001,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35865,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":133,"num_read_kb":106,"num_write":24,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002859"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987328+0000","last_change":"2024-06-06T03:35:13.854938+0000","last_active":"2024-06-06T03:35:48.987328+0000","last_peered":"2024-06-06T03:35:48.987328+0000","last_clean":"2024-06-06T03:35:48.987328+0000","last_became_active":"2024-06-06T03:35:13.853574+0000","last_became_peered":"2024-06-06T03:35:13.853574+0000","last_unstale":"2024-06-06T03:35:48.987328+0000","last_undegraded":"2024-06-06T03:35:48.987328+0000","last_fullsized":"2024-06-06T03:35:48.987328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:21:09.256002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990557+0000","last_change":"2024-06-06T03:35:13.858299+0000","last_active":"2024-06-06T03:35:48.990557+0000","last_peered":"2024-06-06T03:35:48.990557+0000","last_clean":"2024-06-06T03:35:48.990557+0000","last_became_active":"2024-06-06T03:35:13.857993+0000","last_became_peered":"2024-06-06T03:35:13.857993+0000","last_unstale":"2024-06-06T03:35:48.990557+0000","last_undegraded":"2024-06-06T03:35:48.990557+0000","last_fullsized":"2024-06-06T03:35:48.990557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.811056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990502+0000","last_change":"2024-06-06T03:35:13.852783+0000","last_active":"2024-06-06T03:35:48.990502+0000","last_peered":"2024-06-06T03:35:48.990502+0000","last_clean":"2024-06-06T03:35:48.990502+0000","last_became_active":"2024-06-06T03:35:13.852468+0000","last_became_peered":"2024-06-06T03:35:13.852468+0000","last_unstale":"2024-06-06T03:35:48.990502+0000","last_undegraded":"2024-06-06T03:35:48.990502+0000","last_fullsized":"2024-06-06T03:35:48.990502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:04:59.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.004002+0000","last_change":"2024-06-06T03:35:13.845223+0000","last_active":"2024-06-06T03:35:49.004002+0000","last_peered":"2024-06-06T03:35:49.004002+0000","last_clean":"2024-06-06T03:35:49.004002+0000","last_became_active":"2024-06-06T03:35:13.844854+0000","last_became_peered":"2024-06-06T03:35:13.844854+0000","last_unstale":"2024-06-06T03:35:49.004002+0000","last_undegraded":"2024-06-06T03:35:49.004002+0000","last_fullsized":"2024-06-06T03:35:49.004002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:35:20.058188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991571+0000","last_change":"2024-06-06T03:35:13.853703+0000","last_active":"2024-06-06T03:35:48.991571+0000","last_peered":"2024-06-06T03:35:48.991571+0000","last_clean":"2024-06-06T03:35:48.991571+0000","last_became_active":"2024-06-06T03:35:13.853358+0000","last_became_peered":"2024-06-06T03:35:13.853358+0000","last_unstale":"2024-06-06T03:35:48.991571+0000","last_undegraded":"2024-06-06T03:35:48.991571+0000","last_fullsized":"2024-06-06T03:35:48.991571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:23:39.092306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988483+0000","last_change":"2024-06-06T03:35:13.859414+0000","last_active":"2024-06-06T03:35:48.988483+0000","last_peered":"2024-06-06T03:35:48.988483+0000","last_clean":"2024-06-06T03:35:48.988483+0000","last_became_active":"2024-06-06T03:35:13.859038+0000","last_became_peered":"2024-06-06T03:35:13.859038+0000","last_unstale":"2024-06-06T03:35:48.988483+0000","last_undegraded":"2024-06-06T03:35:48.988483+0000","last_fullsized":"2024-06-06T03:35:48.988483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:14:31.780064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985258+0000","last_change":"2024-06-06T03:35:13.859245+0000","last_active":"2024-06-06T03:35:48.985258+0000","last_peered":"2024-06-06T03:35:48.985258+0000","last_clean":"2024-06-06T03:35:48.985258+0000","last_became_active":"2024-06-06T03:35:13.858857+0000","last_became_peered":"2024-06-06T03:35:13.858857+0000","last_unstale":"2024-06-06T03:35:48.985258+0000","last_undegraded":"2024-06-06T03:35:48.985258+0000","last_fullsized":"2024-06-06T03:35:48.985258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:03:23.240007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990694+0000","last_change":"2024-06-06T03:35:13.851249+0000","last_active":"2024-06-06T03:35:48.990694+0000","last_peered":"2024-06-06T03:35:48.990694+0000","last_clean":"2024-06-06T03:35:48.990694+0000","last_became_active":"2024-06-06T03:35:13.850777+0000","last_became_peered":"2024-06-06T03:35:13.850777+0000","last_unstale":"2024-06-06T03:35:48.990694+0000","last_undegraded":"2024-06-06T03:35:48.990694+0000","last_fullsized":"2024-06-06T03:35:48.990694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:53:48.463672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985036+0000","last_change":"2024-06-06T03:35:13.852950+0000","last_active":"2024-06-06T03:35:48.985036+0000","last_peered":"2024-06-06T03:35:48.985036+0000","last_clean":"2024-06-06T03:35:48.985036+0000","last_became_active":"2024-06-06T03:35:13.852160+0000","last_became_peered":"2024-06-06T03:35:13.852160+0000","last_unstale":"2024-06-06T03:35:48.985036+0000","last_undegraded":"2024-06-06T03:35:48.985036+0000","last_fullsized":"2024-06-06T03:35:48.985036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:00.498440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988970+0000","last_change":"2024-06-06T03:35:13.856081+0000","last_active":"2024-06-06T03:35:48.988970+0000","last_peered":"2024-06-06T03:35:48.988970+0000","last_clean":"2024-06-06T03:35:48.988970+0000","last_became_active":"2024-06-06T03:35:13.855718+0000","last_became_peered":"2024-06-06T03:35:13.855718+0000","last_unstale":"2024-06-06T03:35:48.988970+0000","last_undegraded":"2024-06-06T03:35:48.988970+0000","last_fullsized":"2024-06-06T03:35:48.988970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:33:38.171996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985594+0000","last_change":"2024-06-06T03:35:13.861850+0000","last_active":"2024-06-06T03:35:48.985594+0000","last_peered":"2024-06-06T03:35:48.985594+0000","last_clean":"2024-06-06T03:35:48.985594+0000","last_became_active":"2024-06-06T03:35:13.861513+0000","last_became_peered":"2024-06-06T03:35:13.861513+0000","last_unstale":"2024-06-06T03:35:48.985594+0000","last_undegraded":"2024-06-06T03:35:48.985594+0000","last_fullsized":"2024-06-06T03:35:48.985594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:32:45.938302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.003896+0000","last_change":"2024-06-06T03:35:13.847742+0000","last_active":"2024-06-06T03:35:49.003896+0000","last_peered":"2024-06-06T03:35:49.003896+0000","last_clean":"2024-06-06T03:35:49.003896+0000","last_became_active":"2024-06-06T03:35:13.847409+0000","last_became_peered":"2024-06-06T03:35:13.847409+0000","last_unstale":"2024-06-06T03:35:49.003896+0000","last_undegraded":"2024-06-06T03:35:49.003896+0000","last_fullsized":"2024-06-06T03:35:49.003896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:20:31.353919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983746+0000","last_change":"2024-06-06T03:35:13.848785+0000","last_active":"2024-06-06T03:35:48.983746+0000","last_peered":"2024-06-06T03:35:48.983746+0000","last_clean":"2024-06-06T03:35:48.983746+0000","last_became_active":"2024-06-06T03:35:13.848048+0000","last_became_peered":"2024-06-06T03:35:13.848048+0000","last_unstale":"2024-06-06T03:35:48.983746+0000","last_undegraded":"2024-06-06T03:35:48.983746+0000","last_fullsized":"2024-06-06T03:35:48.983746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:43.657761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990820+0000","last_change":"2024-06-06T03:35:13.856083+0000","last_active":"2024-06-06T03:35:48.990820+0000","last_peered":"2024-06-06T03:35:48.990820+0000","last_clean":"2024-06-06T03:35:48.990820+0000","last_became_active":"2024-06-06T03:35:13.854390+0000","last_became_peered":"2024-06-06T03:35:13.854390+0000","last_unstale":"2024-06-06T03:35:48.990820+0000","last_undegraded":"2024-06-06T03:35:48.990820+0000","last_fullsized":"2024-06-06T03:35:48.990820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:51:37.163814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990153+0000","last_change":"2024-06-06T03:35:13.852611+0000","last_active":"2024-06-06T03:35:48.990153+0000","last_peered":"2024-06-06T03:35:48.990153+0000","last_clean":"2024-06-06T03:35:48.990153+0000","last_became_active":"2024-06-06T03:35:13.852244+0000","last_became_peered":"2024-06-06T03:35:13.852244+0000","last_unstale":"2024-06-06T03:35:48.990153+0000","last_undegraded":"2024-06-06T03:35:48.990153+0000","last_fullsized":"2024-06-06T03:35:48.990153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:59:53.513150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984532+0000","last_change":"2024-06-06T03:35:13.857840+0000","last_active":"2024-06-06T03:35:48.984532+0000","last_peered":"2024-06-06T03:35:48.984532+0000","last_clean":"2024-06-06T03:35:48.984532+0000","last_became_active":"2024-06-06T03:35:13.857137+0000","last_became_peered":"2024-06-06T03:35:13.857137+0000","last_unstale":"2024-06-06T03:35:48.984532+0000","last_undegraded":"2024-06-06T03:35:48.984532+0000","last_fullsized":"2024-06-06T03:35:48.984532+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:42:07.924735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987452+0000","last_change":"2024-06-06T03:35:13.857033+0000","last_active":"2024-06-06T03:35:48.987452+0000","last_peered":"2024-06-06T03:35:48.987452+0000","last_clean":"2024-06-06T03:35:48.987452+0000","last_became_active":"2024-06-06T03:35:13.856519+0000","last_became_peered":"2024-06-06T03:35:13.856519+0000","last_unstale":"2024-06-06T03:35:48.987452+0000","last_undegraded":"2024-06-06T03:35:48.987452+0000","last_fullsized":"2024-06-06T03:35:48.987452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:05:25.668344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.986761+0000","last_change":"2024-06-06T03:35:13.851356+0000","last_active":"2024-06-06T03:35:48.986761+0000","last_peered":"2024-06-06T03:35:48.986761+0000","last_clean":"2024-06-06T03:35:48.986761+0000","last_became_active":"2024-06-06T03:35:13.851030+0000","last_became_peered":"2024-06-06T03:35:13.851030+0000","last_unstale":"2024-06-06T03:35:48.986761+0000","last_undegraded":"2024-06-06T03:35:48.986761+0000","last_fullsized":"2024-06-06T03:35:48.986761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:49:38.833022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991734+0000","last_change":"2024-06-06T03:35:13.858613+0000","last_active":"2024-06-06T03:35:48.991734+0000","last_peered":"2024-06-06T03:35:48.991734+0000","last_clean":"2024-06-06T03:35:48.991734+0000","last_became_active":"2024-06-06T03:35:13.858182+0000","last_became_peered":"2024-06-06T03:35:13.858182+0000","last_unstale":"2024-06-06T03:35:48.991734+0000","last_undegraded":"2024-06-06T03:35:48.991734+0000","last_fullsized":"2024-06-06T03:35:48.991734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:23:08.216431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:57.991918+0000","last_change":"2024-06-06T03:35:09.821896+0000","last_active":"2024-06-06T03:35:57.991918+0000","last_peered":"2024-06-06T03:35:57.991918+0000","last_clean":"2024-06-06T03:35:57.991918+0000","last_became_active":"2024-06-06T03:35:09.821567+0000","last_became_peered":"2024-06-06T03:35:09.821567+0000","last_unstale":"2024-06-06T03:35:57.991918+0000","last_undegraded":"2024-06-06T03:35:57.991918+0000","last_fullsized":"2024-06-06T03:35:57.991918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:04:51.918393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991350+0000","last_change":"2024-06-06T03:35:13.857357+0000","last_active":"2024-06-06T03:35:48.991350+0000","last_peered":"2024-06-06T03:35:48.991350+0000","last_clean":"2024-06-06T03:35:48.991350+0000","last_became_active":"2024-06-06T03:35:13.857016+0000","last_became_peered":"2024-06-06T03:35:13.857016+0000","last_unstale":"2024-06-06T03:35:48.991350+0000","last_undegraded":"2024-06-06T03:35:48.991350+0000","last_fullsized":"2024-06-06T03:35:48.991350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:42.566425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991461+0000","last_change":"2024-06-06T03:35:10.277316+0000","last_active":"2024-06-06T03:35:48.991461+0000","last_peered":"2024-06-06T03:35:48.991461+0000","last_clean":"2024-06-06T03:35:48.991461+0000","last_became_active":"2024-06-06T03:35:10.276986+0000","last_became_peered":"2024-06-06T03:35:10.276986+0000","last_unstale":"2024-06-06T03:35:48.991461+0000","last_undegraded":"2024-06-06T03:35:48.991461+0000","last_fullsized":"2024-06-06T03:35:48.991461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:17:29.034224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990383+0000","last_change":"2024-06-06T03:35:13.863069+0000","last_active":"2024-06-06T03:35:48.990383+0000","last_peered":"2024-06-06T03:35:48.990383+0000","last_clean":"2024-06-06T03:35:48.990383+0000","last_became_active":"2024-06-06T03:35:13.862647+0000","last_became_peered":"2024-06-06T03:35:13.862647+0000","last_unstale":"2024-06-06T03:35:48.990383+0000","last_undegraded":"2024-06-06T03:35:48.990383+0000","last_fullsized":"2024-06-06T03:35:48.990383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:48.252606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992306+0000","last_change":"2024-06-06T03:35:09.824977+0000","last_active":"2024-06-06T03:35:48.992306+0000","last_peered":"2024-06-06T03:35:48.992306+0000","last_clean":"2024-06-06T03:35:48.992306+0000","last_became_active":"2024-06-06T03:35:09.824643+0000","last_became_peered":"2024-06-06T03:35:09.824643+0000","last_unstale":"2024-06-06T03:35:48.992306+0000","last_undegraded":"2024-06-06T03:35:48.992306+0000","last_fullsized":"2024-06-06T03:35:48.992306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:35:37.208023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984591+0000","last_change":"2024-06-06T03:35:13.860572+0000","last_active":"2024-06-06T03:35:48.984591+0000","last_peered":"2024-06-06T03:35:48.984591+0000","last_clean":"2024-06-06T03:35:48.984591+0000","last_became_active":"2024-06-06T03:35:13.858398+0000","last_became_peered":"2024-06-06T03:35:13.858398+0000","last_unstale":"2024-06-06T03:35:48.984591+0000","last_undegraded":"2024-06-06T03:35:48.984591+0000","last_fullsized":"2024-06-06T03:35:48.984591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:02:12.683096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991795+0000","last_change":"2024-06-06T03:35:09.827016+0000","last_active":"2024-06-06T03:35:48.991795+0000","last_peered":"2024-06-06T03:35:48.991795+0000","last_clean":"2024-06-06T03:35:48.991795+0000","last_became_active":"2024-06-06T03:35:09.826576+0000","last_became_peered":"2024-06-06T03:35:09.826576+0000","last_unstale":"2024-06-06T03:35:48.991795+0000","last_undegraded":"2024-06-06T03:35:48.991795+0000","last_fullsized":"2024-06-06T03:35:48.991795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:24.120549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984420+0000","last_change":"2024-06-06T03:35:13.853988+0000","last_active":"2024-06-06T03:35:48.984420+0000","last_peered":"2024-06-06T03:35:48.984420+0000","last_clean":"2024-06-06T03:35:48.984420+0000","last_became_active":"2024-06-06T03:35:13.852343+0000","last_became_peered":"2024-06-06T03:35:13.852343+0000","last_unstale":"2024-06-06T03:35:48.984420+0000","last_undegraded":"2024-06-06T03:35:48.984420+0000","last_fullsized":"2024-06-06T03:35:48.984420+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:39:03.054045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:56.976623+0000","last_change":"2024-06-06T03:35:09.829109+0000","last_active":"2024-06-06T03:35:56.976623+0000","last_peered":"2024-06-06T03:35:56.976623+0000","last_clean":"2024-06-06T03:35:56.976623+0000","last_became_active":"2024-06-06T03:35:09.828734+0000","last_became_peered":"2024-06-06T03:35:09.828734+0000","last_unstale":"2024-06-06T03:35:56.976623+0000","last_undegraded":"2024-06-06T03:35:56.976623+0000","last_fullsized":"2024-06-06T03:35:56.976623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:50.475600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992478+0000","last_change":"2024-06-06T03:35:13.859198+0000","last_active":"2024-06-06T03:35:48.992478+0000","last_peered":"2024-06-06T03:35:48.992478+0000","last_clean":"2024-06-06T03:35:48.992478+0000","last_became_active":"2024-06-06T03:35:13.858863+0000","last_became_peered":"2024-06-06T03:35:13.858863+0000","last_unstale":"2024-06-06T03:35:48.992478+0000","last_undegraded":"2024-06-06T03:35:48.992478+0000","last_fullsized":"2024-06-06T03:35:48.992478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:57.153174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988385+0000","last_change":"2024-06-06T03:35:09.824352+0000","last_active":"2024-06-06T03:35:48.988385+0000","last_peered":"2024-06-06T03:35:48.988385+0000","last_clean":"2024-06-06T03:35:48.988385+0000","last_became_active":"2024-06-06T03:35:09.823998+0000","last_became_peered":"2024-06-06T03:35:09.823998+0000","last_unstale":"2024-06-06T03:35:48.988385+0000","last_undegraded":"2024-06-06T03:35:48.988385+0000","last_fullsized":"2024-06-06T03:35:48.988385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:47:24.646067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984826+0000","last_change":"2024-06-06T03:35:13.836724+0000","last_active":"2024-06-06T03:35:48.984826+0000","last_peered":"2024-06-06T03:35:48.984826+0000","last_clean":"2024-06-06T03:35:48.984826+0000","last_became_active":"2024-06-06T03:35:13.836180+0000","last_became_peered":"2024-06-06T03:35:13.836180+0000","last_unstale":"2024-06-06T03:35:48.984826+0000","last_undegraded":"2024-06-06T03:35:48.984826+0000","last_fullsized":"2024-06-06T03:35:48.984826+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:08.868462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983408+0000","last_change":"2024-06-06T03:35:09.823047+0000","last_active":"2024-06-06T03:35:48.983408+0000","last_peered":"2024-06-06T03:35:48.983408+0000","last_clean":"2024-06-06T03:35:48.983408+0000","last_became_active":"2024-06-06T03:35:09.822744+0000","last_became_peered":"2024-06-06T03:35:09.822744+0000","last_unstale":"2024-06-06T03:35:48.983408+0000","last_undegraded":"2024-06-06T03:35:48.983408+0000","last_fullsized":"2024-06-06T03:35:48.983408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:20:20.165682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.989848+0000","last_change":"2024-06-06T03:35:13.857222+0000","last_active":"2024-06-06T03:35:48.989848+0000","last_peered":"2024-06-06T03:35:48.989848+0000","last_clean":"2024-06-06T03:35:48.989848+0000","last_became_active":"2024-06-06T03:35:13.856873+0000","last_became_peered":"2024-06-06T03:35:13.856873+0000","last_unstale":"2024-06-06T03:35:48.989848+0000","last_undegraded":"2024-06-06T03:35:48.989848+0000","last_fullsized":"2024-06-06T03:35:48.989848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:32:16.646275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984083+0000","last_change":"2024-06-06T03:35:09.826036+0000","last_active":"2024-06-06T03:35:48.984083+0000","last_peered":"2024-06-06T03:35:48.984083+0000","last_clean":"2024-06-06T03:35:48.984083+0000","last_became_active":"2024-06-06T03:35:09.825690+0000","last_became_peered":"2024-06-06T03:35:09.825690+0000","last_unstale":"2024-06-06T03:35:48.984083+0000","last_undegraded":"2024-06-06T03:35:48.984083+0000","last_fullsized":"2024-06-06T03:35:48.984083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:03:35.858828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.986086+0000","last_change":"2024-06-06T03:35:13.849016+0000","last_active":"2024-06-06T03:35:48.986086+0000","last_peered":"2024-06-06T03:35:48.986086+0000","last_clean":"2024-06-06T03:35:48.986086+0000","last_became_active":"2024-06-06T03:35:13.848709+0000","last_became_peered":"2024-06-06T03:35:13.848709+0000","last_unstale":"2024-06-06T03:35:48.986086+0000","last_undegraded":"2024-06-06T03:35:48.986086+0000","last_fullsized":"2024-06-06T03:35:48.986086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:42.778612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":84,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:36:00.981680+0000","last_change":"2024-06-06T03:35:09.827072+0000","last_active":"2024-06-06T03:36:00.981680+0000","last_peered":"2024-06-06T03:36:00.981680+0000","last_clean":"2024-06-06T03:36:00.981680+0000","last_became_active":"2024-06-06T03:35:09.826734+0000","last_became_peered":"2024-06-06T03:35:09.826734+0000","last_unstale":"2024-06-06T03:36:00.981680+0000","last_undegraded":"2024-06-06T03:36:00.981680+0000","last_fullsized":"2024-06-06T03:36:00.981680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:07:13.388398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985601+0000","last_change":"2024-06-06T03:35:13.856883+0000","last_active":"2024-06-06T03:35:48.985601+0000","last_peered":"2024-06-06T03:35:48.985601+0000","last_clean":"2024-06-06T03:35:48.985601+0000","last_became_active":"2024-06-06T03:35:13.856521+0000","last_became_peered":"2024-06-06T03:35:13.856521+0000","last_unstale":"2024-06-06T03:35:48.985601+0000","last_undegraded":"2024-06-06T03:35:48.985601+0000","last_fullsized":"2024-06-06T03:35:48.985601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:19.466218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985292+0000","last_change":"2024-06-06T03:35:09.829153+0000","last_active":"2024-06-06T03:35:48.985292+0000","last_peered":"2024-06-06T03:35:48.985292+0000","last_clean":"2024-06-06T03:35:48.985292+0000","last_became_active":"2024-06-06T03:35:09.828797+0000","last_became_peered":"2024-06-06T03:35:09.828797+0000","last_unstale":"2024-06-06T03:35:48.985292+0000","last_undegraded":"2024-06-06T03:35:48.985292+0000","last_fullsized":"2024-06-06T03:35:48.985292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:06:02.986396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990268+0000","last_change":"2024-06-06T03:35:13.857094+0000","last_active":"2024-06-06T03:35:48.990268+0000","last_peered":"2024-06-06T03:35:48.990268+0000","last_clean":"2024-06-06T03:35:48.990268+0000","last_became_active":"2024-06-06T03:35:13.856778+0000","last_became_peered":"2024-06-06T03:35:13.856778+0000","last_unstale":"2024-06-06T03:35:48.990268+0000","last_undegraded":"2024-06-06T03:35:48.990268+0000","last_fullsized":"2024-06-06T03:35:48.990268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:50:30.097762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984506+0000","last_change":"2024-06-06T03:35:10.277802+0000","last_active":"2024-06-06T03:35:48.984506+0000","last_peered":"2024-06-06T03:35:48.984506+0000","last_clean":"2024-06-06T03:35:48.984506+0000","last_became_active":"2024-06-06T03:35:10.277417+0000","last_became_peered":"2024-06-06T03:35:10.277417+0000","last_unstale":"2024-06-06T03:35:48.984506+0000","last_undegraded":"2024-06-06T03:35:48.984506+0000","last_fullsized":"2024-06-06T03:35:48.984506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:44:35.178184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984699+0000","last_change":"2024-06-06T03:35:13.858160+0000","last_active":"2024-06-06T03:35:48.984699+0000","last_peered":"2024-06-06T03:35:48.984699+0000","last_clean":"2024-06-06T03:35:48.984699+0000","last_became_active":"2024-06-06T03:35:13.853149+0000","last_became_peered":"2024-06-06T03:35:13.853149+0000","last_unstale":"2024-06-06T03:35:48.984699+0000","last_undegraded":"2024-06-06T03:35:48.984699+0000","last_fullsized":"2024-06-06T03:35:48.984699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:10:06.697274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'30","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:36:00.983512+0000","last_change":"2024-06-06T03:35:09.829443+0000","last_active":"2024-06-06T03:36:00.983512+0000","last_peered":"2024-06-06T03:36:00.983512+0000","last_clean":"2024-06-06T03:36:00.983512+0000","last_became_active":"2024-06-06T03:35:09.828093+0000","last_became_peered":"2024-06-06T03:35:09.828093+0000","last_unstale":"2024-06-06T03:36:00.983512+0000","last_undegraded":"2024-06-06T03:36:00.983512+0000","last_fullsized":"2024-06-06T03:36:00.983512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:05:19.395177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58077,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":62,"num_read_kb":87,"num_write":32,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991968+0000","last_change":"2024-06-06T03:35:13.850869+0000","last_active":"2024-06-06T03:35:48.991968+0000","last_peered":"2024-06-06T03:35:48.991968+0000","last_clean":"2024-06-06T03:35:48.991968+0000","last_became_active":"2024-06-06T03:35:13.849489+0000","last_became_peered":"2024-06-06T03:35:13.849489+0000","last_unstale":"2024-06-06T03:35:48.991968+0000","last_undegraded":"2024-06-06T03:35:48.991968+0000","last_fullsized":"2024-06-06T03:35:48.991968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:27:45.052033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991491+0000","last_change":"2024-06-06T03:35:09.823117+0000","last_active":"2024-06-06T03:35:48.991491+0000","last_peered":"2024-06-06T03:35:48.991491+0000","last_clean":"2024-06-06T03:35:48.991491+0000","last_became_active":"2024-06-06T03:35:09.822624+0000","last_became_peered":"2024-06-06T03:35:09.822624+0000","last_unstale":"2024-06-06T03:35:48.991491+0000","last_undegraded":"2024-06-06T03:35:48.991491+0000","last_fullsized":"2024-06-06T03:35:48.991491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:59.171343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983437+0000","last_change":"2024-06-06T03:35:13.861043+0000","last_active":"2024-06-06T03:35:48.983437+0000","last_peered":"2024-06-06T03:35:48.983437+0000","last_clean":"2024-06-06T03:35:48.983437+0000","last_became_active":"2024-06-06T03:35:13.860718+0000","last_became_peered":"2024-06-06T03:35:13.860718+0000","last_unstale":"2024-06-06T03:35:48.983437+0000","last_undegraded":"2024-06-06T03:35:48.983437+0000","last_fullsized":"2024-06-06T03:35:48.983437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:34:04.595780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984355+0000","last_change":"2024-06-06T03:35:09.825505+0000","last_active":"2024-06-06T03:35:48.984355+0000","last_peered":"2024-06-06T03:35:48.984355+0000","last_clean":"2024-06-06T03:35:48.984355+0000","last_became_active":"2024-06-06T03:35:09.825127+0000","last_became_peered":"2024-06-06T03:35:09.825127+0000","last_unstale":"2024-06-06T03:35:48.984355+0000","last_undegraded":"2024-06-06T03:35:48.984355+0000","last_fullsized":"2024-06-06T03:35:48.984355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:07:27.664589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987834+0000","last_change":"2024-06-06T03:35:48.987834+0000","last_active":"2024-06-06T03:35:48.987834+0000","last_peered":"2024-06-06T03:35:48.987834+0000","last_clean":"2024-06-06T03:35:48.987834+0000","last_became_active":"2024-06-06T03:35:48.987515+0000","last_became_peered":"2024-06-06T03:35:48.987515+0000","last_unstale":"2024-06-06T03:35:48.987834+0000","last_undegraded":"2024-06-06T03:35:48.987834+0000","last_fullsized":"2024-06-06T03:35:48.987834+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:05:19.342289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:56.976008+0000","last_change":"2024-06-06T03:35:09.826192+0000","last_active":"2024-06-06T03:35:56.976008+0000","last_peered":"2024-06-06T03:35:56.976008+0000","last_clean":"2024-06-06T03:35:56.976008+0000","last_became_active":"2024-06-06T03:35:09.825481+0000","last_became_peered":"2024-06-06T03:35:09.825481+0000","last_unstale":"2024-06-06T03:35:56.976008+0000","last_undegraded":"2024-06-06T03:35:56.976008+0000","last_fullsized":"2024-06-06T03:35:56.976008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:18:30.772991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985035+0000","last_change":"2024-06-06T03:35:13.853113+0000","last_active":"2024-06-06T03:35:48.985035+0000","last_peered":"2024-06-06T03:35:48.985035+0000","last_clean":"2024-06-06T03:35:48.985035+0000","last_became_active":"2024-06-06T03:35:13.852751+0000","last_became_peered":"2024-06-06T03:35:13.852751+0000","last_unstale":"2024-06-06T03:35:48.985035+0000","last_undegraded":"2024-06-06T03:35:48.985035+0000","last_fullsized":"2024-06-06T03:35:48.985035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:53.536685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.006465+0000","last_change":"2024-06-06T03:35:49.005958+0000","last_active":"2024-06-06T03:35:49.006465+0000","last_peered":"2024-06-06T03:35:49.006465+0000","last_clean":"2024-06-06T03:35:49.006465+0000","last_became_active":"2024-06-06T03:35:48.993266+0000","last_became_peered":"2024-06-06T03:35:48.993266+0000","last_unstale":"2024-06-06T03:35:49.006465+0000","last_undegraded":"2024-06-06T03:35:49.006465+0000","last_fullsized":"2024-06-06T03:35:49.006465+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:47.782824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984158+0000","last_change":"2024-06-06T03:35:13.853614+0000","last_active":"2024-06-06T03:35:48.984158+0000","last_peered":"2024-06-06T03:35:48.984158+0000","last_clean":"2024-06-06T03:35:48.984158+0000","last_became_active":"2024-06-06T03:35:13.852327+0000","last_became_peered":"2024-06-06T03:35:13.852327+0000","last_unstale":"2024-06-06T03:35:48.984158+0000","last_undegraded":"2024-06-06T03:35:48.984158+0000","last_fullsized":"2024-06-06T03:35:48.984158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:02:32.463028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991648+0000","last_change":"2024-06-06T03:35:09.826220+0000","last_active":"2024-06-06T03:35:48.991648+0000","last_peered":"2024-06-06T03:35:48.991648+0000","last_clean":"2024-06-06T03:35:48.991648+0000","last_became_active":"2024-06-06T03:35:09.825887+0000","last_became_peered":"2024-06-06T03:35:09.825887+0000","last_unstale":"2024-06-06T03:35:48.991648+0000","last_undegraded":"2024-06-06T03:35:48.991648+0000","last_fullsized":"2024-06-06T03:35:48.991648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:23:33.704822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987180+0000","last_change":"2024-06-06T03:35:13.852455+0000","last_active":"2024-06-06T03:35:48.987180+0000","last_peered":"2024-06-06T03:35:48.987180+0000","last_clean":"2024-06-06T03:35:48.987180+0000","last_became_active":"2024-06-06T03:35:13.852134+0000","last_became_peered":"2024-06-06T03:35:13.852134+0000","last_unstale":"2024-06-06T03:35:48.987180+0000","last_undegraded":"2024-06-06T03:35:48.987180+0000","last_fullsized":"2024-06-06T03:35:48.987180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:00:16.418479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.986584+0000","last_change":"2024-06-06T03:35:10.277767+0000","last_active":"2024-06-06T03:35:48.986584+0000","last_peered":"2024-06-06T03:35:48.986584+0000","last_clean":"2024-06-06T03:35:48.986584+0000","last_became_active":"2024-06-06T03:35:10.277381+0000","last_became_peered":"2024-06-06T03:35:10.277381+0000","last_unstale":"2024-06-06T03:35:48.986584+0000","last_undegraded":"2024-06-06T03:35:48.986584+0000","last_fullsized":"2024-06-06T03:35:48.986584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:12:13.425284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988580+0000","last_change":"2024-06-06T03:35:13.859594+0000","last_active":"2024-06-06T03:35:48.988580+0000","last_peered":"2024-06-06T03:35:48.988580+0000","last_clean":"2024-06-06T03:35:48.988580+0000","last_became_active":"2024-06-06T03:35:13.859240+0000","last_became_peered":"2024-06-06T03:35:13.859240+0000","last_unstale":"2024-06-06T03:35:48.988580+0000","last_undegraded":"2024-06-06T03:35:48.988580+0000","last_fullsized":"2024-06-06T03:35:48.988580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:39:34.557581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:58.993547+0000","last_change":"2024-06-06T03:35:09.826204+0000","last_active":"2024-06-06T03:35:58.993547+0000","last_peered":"2024-06-06T03:35:58.993547+0000","last_clean":"2024-06-06T03:35:58.993547+0000","last_became_active":"2024-06-06T03:35:09.825815+0000","last_became_peered":"2024-06-06T03:35:09.825815+0000","last_unstale":"2024-06-06T03:35:58.993547+0000","last_undegraded":"2024-06-06T03:35:58.993547+0000","last_fullsized":"2024-06-06T03:35:58.993547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:02:04.169227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992786+0000","last_change":"2024-06-06T03:35:48.992786+0000","last_active":"2024-06-06T03:35:48.992786+0000","last_peered":"2024-06-06T03:35:48.992786+0000","last_clean":"2024-06-06T03:35:48.992786+0000","last_became_active":"2024-06-06T03:35:48.992456+0000","last_became_peered":"2024-06-06T03:35:48.992456+0000","last_unstale":"2024-06-06T03:35:48.992786+0000","last_undegraded":"2024-06-06T03:35:48.992786+0000","last_fullsized":"2024-06-06T03:35:48.992786+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:06:58.563975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.005564+0000","last_change":"2024-06-06T03:35:49.005119+0000","last_active":"2024-06-06T03:35:49.005564+0000","last_peered":"2024-06-06T03:35:49.005564+0000","last_clean":"2024-06-06T03:35:49.005564+0000","last_became_active":"2024-06-06T03:35:49.004788+0000","last_became_peered":"2024-06-06T03:35:49.004788+0000","last_unstale":"2024-06-06T03:35:49.005564+0000","last_undegraded":"2024-06-06T03:35:49.005564+0000","last_fullsized":"2024-06-06T03:35:49.005564+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:20:06.961454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:58.978408+0000","last_change":"2024-06-06T03:35:10.278559+0000","last_active":"2024-06-06T03:35:58.978408+0000","last_peered":"2024-06-06T03:35:58.978408+0000","last_clean":"2024-06-06T03:35:58.978408+0000","last_became_active":"2024-06-06T03:35:10.278104+0000","last_became_peered":"2024-06-06T03:35:10.278104+0000","last_unstale":"2024-06-06T03:35:58.978408+0000","last_undegraded":"2024-06-06T03:35:58.978408+0000","last_fullsized":"2024-06-06T03:35:58.978408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:23:47.131731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990973+0000","last_change":"2024-06-06T03:35:13.862303+0000","last_active":"2024-06-06T03:35:48.990973+0000","last_peered":"2024-06-06T03:35:48.990973+0000","last_clean":"2024-06-06T03:35:48.990973+0000","last_became_active":"2024-06-06T03:35:13.861978+0000","last_became_peered":"2024-06-06T03:35:13.861978+0000","last_unstale":"2024-06-06T03:35:48.990973+0000","last_undegraded":"2024-06-06T03:35:48.990973+0000","last_fullsized":"2024-06-06T03:35:48.990973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:24:00.824646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.004144+0000","last_change":"2024-06-06T03:35:09.825365+0000","last_active":"2024-06-06T03:35:49.004144+0000","last_peered":"2024-06-06T03:35:49.004144+0000","last_clean":"2024-06-06T03:35:49.004144+0000","last_became_active":"2024-06-06T03:35:09.824948+0000","last_became_peered":"2024-06-06T03:35:09.824948+0000","last_unstale":"2024-06-06T03:35:49.004144+0000","last_undegraded":"2024-06-06T03:35:49.004144+0000","last_fullsized":"2024-06-06T03:35:49.004144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:23:48.189292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":358,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985268+0000","last_change":"2024-06-06T03:33:44.124626+0000","last_active":"2024-06-06T03:35:48.985268+0000","last_peered":"2024-06-06T03:35:48.985268+0000","last_clean":"2024-06-06T03:35:48.985268+0000","last_became_active":"2024-06-06T03:33:44.124046+0000","last_became_peered":"2024-06-06T03:33:44.124046+0000","last_unstale":"2024-06-06T03:35:48.985268+0000","last_undegraded":"2024-06-06T03:35:48.985268+0000","last_fullsized":"2024-06-06T03:35:48.985268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:30:14.586719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:30:14.586719+0000","last_clean_scrub_stamp":"2024-06-06T03:30:14.586719+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:17.604180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983301+0000","last_change":"2024-06-06T03:35:13.852044+0000","last_active":"2024-06-06T03:35:48.983301+0000","last_peered":"2024-06-06T03:35:48.983301+0000","last_clean":"2024-06-06T03:35:48.983301+0000","last_became_active":"2024-06-06T03:35:13.851715+0000","last_became_peered":"2024-06-06T03:35:13.851715+0000","last_unstale":"2024-06-06T03:35:48.983301+0000","last_undegraded":"2024-06-06T03:35:48.983301+0000","last_fullsized":"2024-06-06T03:35:48.983301+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:19:23.778367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985449+0000","last_change":"2024-06-06T03:35:09.828463+0000","last_active":"2024-06-06T03:35:48.985449+0000","last_peered":"2024-06-06T03:35:48.985449+0000","last_clean":"2024-06-06T03:35:48.985449+0000","last_became_active":"2024-06-06T03:35:09.828121+0000","last_became_peered":"2024-06-06T03:35:09.828121+0000","last_unstale":"2024-06-06T03:35:48.985449+0000","last_undegraded":"2024-06-06T03:35:48.985449+0000","last_fullsized":"2024-06-06T03:35:48.985449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:54:25.095741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992104+0000","last_change":"2024-06-06T03:35:10.277901+0000","last_active":"2024-06-06T03:35:48.992104+0000","last_peered":"2024-06-06T03:35:48.992104+0000","last_clean":"2024-06-06T03:35:48.992104+0000","last_became_active":"2024-06-06T03:35:10.277447+0000","last_became_peered":"2024-06-06T03:35:10.277447+0000","last_unstale":"2024-06-06T03:35:48.992104+0000","last_undegraded":"2024-06-06T03:35:48.992104+0000","last_fullsized":"2024-06-06T03:35:48.992104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:22:50.550287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983583+0000","last_change":"2024-06-06T03:35:13.864323+0000","last_active":"2024-06-06T03:35:48.983583+0000","last_peered":"2024-06-06T03:35:48.983583+0000","last_clean":"2024-06-06T03:35:48.983583+0000","last_became_active":"2024-06-06T03:35:13.863101+0000","last_became_peered":"2024-06-06T03:35:13.863101+0000","last_unstale":"2024-06-06T03:35:48.983583+0000","last_undegraded":"2024-06-06T03:35:48.983583+0000","last_fullsized":"2024-06-06T03:35:48.983583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:32.399325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'14","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:54.869403+0000","last_change":"2024-06-06T03:35:09.822280+0000","last_active":"2024-06-06T03:35:54.869403+0000","last_peered":"2024-06-06T03:35:54.869403+0000","last_clean":"2024-06-06T03:35:54.869403+0000","last_became_active":"2024-06-06T03:35:09.821922+0000","last_became_peered":"2024-06-06T03:35:09.821922+0000","last_unstale":"2024-06-06T03:35:54.869403+0000","last_undegraded":"2024-06-06T03:35:54.869403+0000","last_fullsized":"2024-06-06T03:35:54.869403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:00:03.328747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991350+0000","last_change":"2024-06-06T03:35:13.853538+0000","last_active":"2024-06-06T03:35:48.991350+0000","last_peered":"2024-06-06T03:35:48.991350+0000","last_clean":"2024-06-06T03:35:48.991350+0000","last_became_active":"2024-06-06T03:35:13.853200+0000","last_became_peered":"2024-06-06T03:35:13.853200+0000","last_unstale":"2024-06-06T03:35:48.991350+0000","last_undegraded":"2024-06-06T03:35:48.991350+0000","last_fullsized":"2024-06-06T03:35:48.991350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:13:06.574272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.004336+0000","last_change":"2024-06-06T03:35:10.761675+0000","last_active":"2024-06-06T03:35:49.004336+0000","last_peered":"2024-06-06T03:35:49.004336+0000","last_clean":"2024-06-06T03:35:49.004336+0000","last_became_active":"2024-06-06T03:35:10.761287+0000","last_became_peered":"2024-06-06T03:35:10.761287+0000","last_unstale":"2024-06-06T03:35:49.004336+0000","last_undegraded":"2024-06-06T03:35:49.004336+0000","last_fullsized":"2024-06-06T03:35:49.004336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:58:18.524273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990632+0000","last_change":"2024-06-06T03:35:13.847680+0000","last_active":"2024-06-06T03:35:48.990632+0000","last_peered":"2024-06-06T03:35:48.990632+0000","last_clean":"2024-06-06T03:35:48.990632+0000","last_became_active":"2024-06-06T03:35:13.847359+0000","last_became_peered":"2024-06-06T03:35:13.847359+0000","last_unstale":"2024-06-06T03:35:48.990632+0000","last_undegraded":"2024-06-06T03:35:48.990632+0000","last_fullsized":"2024-06-06T03:35:48.990632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:29:18.041078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984544+0000","last_change":"2024-06-06T03:35:10.277491+0000","last_active":"2024-06-06T03:35:48.984544+0000","last_peered":"2024-06-06T03:35:48.984544+0000","last_clean":"2024-06-06T03:35:48.984544+0000","last_became_active":"2024-06-06T03:35:10.277135+0000","last_became_peered":"2024-06-06T03:35:10.277135+0000","last_unstale":"2024-06-06T03:35:48.984544+0000","last_undegraded":"2024-06-06T03:35:48.984544+0000","last_fullsized":"2024-06-06T03:35:48.984544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:55.198789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991400+0000","last_change":"2024-06-06T03:35:13.846013+0000","last_active":"2024-06-06T03:35:48.991400+0000","last_peered":"2024-06-06T03:35:48.991400+0000","last_clean":"2024-06-06T03:35:48.991400+0000","last_became_active":"2024-06-06T03:35:13.845650+0000","last_became_peered":"2024-06-06T03:35:13.845650+0000","last_unstale":"2024-06-06T03:35:48.991400+0000","last_undegraded":"2024-06-06T03:35:48.991400+0000","last_fullsized":"2024-06-06T03:35:48.991400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:51.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990583+0000","last_change":"2024-06-06T03:35:09.826380+0000","last_active":"2024-06-06T03:35:48.990583+0000","last_peered":"2024-06-06T03:35:48.990583+0000","last_clean":"2024-06-06T03:35:48.990583+0000","last_became_active":"2024-06-06T03:35:09.825813+0000","last_became_peered":"2024-06-06T03:35:09.825813+0000","last_unstale":"2024-06-06T03:35:48.990583+0000","last_undegraded":"2024-06-06T03:35:48.990583+0000","last_fullsized":"2024-06-06T03:35:48.990583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:24.792398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987048+0000","last_change":"2024-06-06T03:35:13.855051+0000","last_active":"2024-06-06T03:35:48.987048+0000","last_peered":"2024-06-06T03:35:48.987048+0000","last_clean":"2024-06-06T03:35:48.987048+0000","last_became_active":"2024-06-06T03:35:13.853781+0000","last_became_peered":"2024-06-06T03:35:13.853781+0000","last_unstale":"2024-06-06T03:35:48.987048+0000","last_undegraded":"2024-06-06T03:35:48.987048+0000","last_fullsized":"2024-06-06T03:35:48.987048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:17:56.174924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.009988+0000","last_change":"2024-06-06T03:35:49.009583+0000","last_active":"2024-06-06T03:35:49.009988+0000","last_peered":"2024-06-06T03:35:49.009988+0000","last_clean":"2024-06-06T03:35:49.009988+0000","last_became_active":"2024-06-06T03:35:48.994275+0000","last_became_peered":"2024-06-06T03:35:48.994275+0000","last_unstale":"2024-06-06T03:35:49.009988+0000","last_undegraded":"2024-06-06T03:35:49.009988+0000","last_fullsized":"2024-06-06T03:35:49.009988+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:24:54.985854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991216+0000","last_change":"2024-06-06T03:35:13.855503+0000","last_active":"2024-06-06T03:35:48.991216+0000","last_peered":"2024-06-06T03:35:48.991216+0000","last_clean":"2024-06-06T03:35:48.991216+0000","last_became_active":"2024-06-06T03:35:13.855069+0000","last_became_peered":"2024-06-06T03:35:13.855069+0000","last_unstale":"2024-06-06T03:35:48.991216+0000","last_undegraded":"2024-06-06T03:35:48.991216+0000","last_fullsized":"2024-06-06T03:35:48.991216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:43.778258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991692+0000","last_change":"2024-06-06T03:35:09.827116+0000","last_active":"2024-06-06T03:35:48.991692+0000","last_peered":"2024-06-06T03:35:48.991692+0000","last_clean":"2024-06-06T03:35:48.991692+0000","last_became_active":"2024-06-06T03:35:09.826756+0000","last_became_peered":"2024-06-06T03:35:09.826756+0000","last_unstale":"2024-06-06T03:35:48.991692+0000","last_undegraded":"2024-06-06T03:35:48.991692+0000","last_fullsized":"2024-06-06T03:35:48.991692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:31.501411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985532+0000","last_change":"2024-06-06T03:35:13.861189+0000","last_active":"2024-06-06T03:35:48.985532+0000","last_peered":"2024-06-06T03:35:48.985532+0000","last_clean":"2024-06-06T03:35:48.985532+0000","last_became_active":"2024-06-06T03:35:13.860751+0000","last_became_peered":"2024-06-06T03:35:13.860751+0000","last_unstale":"2024-06-06T03:35:48.985532+0000","last_undegraded":"2024-06-06T03:35:48.985532+0000","last_fullsized":"2024-06-06T03:35:48.985532+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:16:34.036420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:56.976460+0000","last_change":"2024-06-06T03:35:10.761606+0000","last_active":"2024-06-06T03:35:56.976460+0000","last_peered":"2024-06-06T03:35:56.976460+0000","last_clean":"2024-06-06T03:35:56.976460+0000","last_became_active":"2024-06-06T03:35:10.761104+0000","last_became_peered":"2024-06-06T03:35:10.761104+0000","last_unstale":"2024-06-06T03:35:56.976460+0000","last_undegraded":"2024-06-06T03:35:56.976460+0000","last_fullsized":"2024-06-06T03:35:56.976460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:51:39.867088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985751+0000","last_change":"2024-06-06T03:35:13.856047+0000","last_active":"2024-06-06T03:35:48.985751+0000","last_peered":"2024-06-06T03:35:48.985751+0000","last_clean":"2024-06-06T03:35:48.985751+0000","last_became_active":"2024-06-06T03:35:13.855718+0000","last_became_peered":"2024-06-06T03:35:13.855718+0000","last_unstale":"2024-06-06T03:35:48.985751+0000","last_undegraded":"2024-06-06T03:35:48.985751+0000","last_fullsized":"2024-06-06T03:35:48.985751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:18:40.926872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984381+0000","last_change":"2024-06-06T03:35:09.827283+0000","last_active":"2024-06-06T03:35:48.984381+0000","last_peered":"2024-06-06T03:35:48.984381+0000","last_clean":"2024-06-06T03:35:48.984381+0000","last_became_active":"2024-06-06T03:35:09.826595+0000","last_became_peered":"2024-06-06T03:35:09.826595+0000","last_unstale":"2024-06-06T03:35:48.984381+0000","last_undegraded":"2024-06-06T03:35:48.984381+0000","last_fullsized":"2024-06-06T03:35:48.984381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:34:16.398847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985446+0000","last_change":"2024-06-06T03:35:13.851900+0000","last_active":"2024-06-06T03:35:48.985446+0000","last_peered":"2024-06-06T03:35:48.985446+0000","last_clean":"2024-06-06T03:35:48.985446+0000","last_became_active":"2024-06-06T03:35:13.851477+0000","last_became_peered":"2024-06-06T03:35:13.851477+0000","last_unstale":"2024-06-06T03:35:48.985446+0000","last_undegraded":"2024-06-06T03:35:48.985446+0000","last_fullsized":"2024-06-06T03:35:48.985446+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:31:54.001226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983666+0000","last_change":"2024-06-06T03:35:09.824675+0000","last_active":"2024-06-06T03:35:48.983666+0000","last_peered":"2024-06-06T03:35:48.983666+0000","last_clean":"2024-06-06T03:35:48.983666+0000","last_became_active":"2024-06-06T03:35:09.824347+0000","last_became_peered":"2024-06-06T03:35:09.824347+0000","last_unstale":"2024-06-06T03:35:48.983666+0000","last_undegraded":"2024-06-06T03:35:48.983666+0000","last_fullsized":"2024-06-06T03:35:48.983666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:01:43.080552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985522+0000","last_change":"2024-06-06T03:35:13.854706+0000","last_active":"2024-06-06T03:35:48.985522+0000","last_peered":"2024-06-06T03:35:48.985522+0000","last_clean":"2024-06-06T03:35:48.985522+0000","last_became_active":"2024-06-06T03:35:13.854392+0000","last_became_peered":"2024-06-06T03:35:13.854392+0000","last_unstale":"2024-06-06T03:35:48.985522+0000","last_undegraded":"2024-06-06T03:35:48.985522+0000","last_fullsized":"2024-06-06T03:35:48.985522+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:58:10.102367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987223+0000","last_change":"2024-06-06T03:35:13.853594+0000","last_active":"2024-06-06T03:35:48.987223+0000","last_peered":"2024-06-06T03:35:48.987223+0000","last_clean":"2024-06-06T03:35:48.987223+0000","last_became_active":"2024-06-06T03:35:13.852962+0000","last_became_peered":"2024-06-06T03:35:13.852962+0000","last_unstale":"2024-06-06T03:35:48.987223+0000","last_undegraded":"2024-06-06T03:35:48.987223+0000","last_fullsized":"2024-06-06T03:35:48.987223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:02:44.233868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991198+0000","last_change":"2024-06-06T03:35:13.848790+0000","last_active":"2024-06-06T03:35:48.991198+0000","last_peered":"2024-06-06T03:35:48.991198+0000","last_clean":"2024-06-06T03:35:48.991198+0000","last_became_active":"2024-06-06T03:35:13.848442+0000","last_became_peered":"2024-06-06T03:35:13.848442+0000","last_unstale":"2024-06-06T03:35:48.991198+0000","last_undegraded":"2024-06-06T03:35:48.991198+0000","last_fullsized":"2024-06-06T03:35:48.991198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:51.382210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991331+0000","last_change":"2024-06-06T03:35:13.866999+0000","last_active":"2024-06-06T03:35:48.991331+0000","last_peered":"2024-06-06T03:35:48.991331+0000","last_clean":"2024-06-06T03:35:48.991331+0000","last_became_active":"2024-06-06T03:35:13.866664+0000","last_became_peered":"2024-06-06T03:35:13.866664+0000","last_unstale":"2024-06-06T03:35:48.991331+0000","last_undegraded":"2024-06-06T03:35:48.991331+0000","last_fullsized":"2024-06-06T03:35:48.991331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:31:14.166122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990482+0000","last_change":"2024-06-06T03:35:13.849062+0000","last_active":"2024-06-06T03:35:48.990482+0000","last_peered":"2024-06-06T03:35:48.990482+0000","last_clean":"2024-06-06T03:35:48.990482+0000","last_became_active":"2024-06-06T03:35:13.848750+0000","last_became_peered":"2024-06-06T03:35:13.848750+0000","last_unstale":"2024-06-06T03:35:48.990482+0000","last_undegraded":"2024-06-06T03:35:48.990482+0000","last_fullsized":"2024-06-06T03:35:48.990482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:55:58.084908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983429+0000","last_change":"2024-06-06T03:35:13.857601+0000","last_active":"2024-06-06T03:35:48.983429+0000","last_peered":"2024-06-06T03:35:48.983429+0000","last_clean":"2024-06-06T03:35:48.983429+0000","last_became_active":"2024-06-06T03:35:13.857270+0000","last_became_peered":"2024-06-06T03:35:13.857270+0000","last_unstale":"2024-06-06T03:35:48.983429+0000","last_undegraded":"2024-06-06T03:35:48.983429+0000","last_fullsized":"2024-06-06T03:35:48.983429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:56:29.970210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987336+0000","last_change":"2024-06-06T03:35:13.846699+0000","last_active":"2024-06-06T03:35:48.987336+0000","last_peered":"2024-06-06T03:35:48.987336+0000","last_clean":"2024-06-06T03:35:48.987336+0000","last_became_active":"2024-06-06T03:35:13.846355+0000","last_became_peered":"2024-06-06T03:35:13.846355+0000","last_unstale":"2024-06-06T03:35:48.987336+0000","last_undegraded":"2024-06-06T03:35:48.987336+0000","last_fullsized":"2024-06-06T03:35:48.987336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:50:22.537339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991821+0000","last_change":"2024-06-06T03:35:13.854013+0000","last_active":"2024-06-06T03:35:48.991821+0000","last_peered":"2024-06-06T03:35:48.991821+0000","last_clean":"2024-06-06T03:35:48.991821+0000","last_became_active":"2024-06-06T03:35:13.853739+0000","last_became_peered":"2024-06-06T03:35:13.853739+0000","last_unstale":"2024-06-06T03:35:48.991821+0000","last_undegraded":"2024-06-06T03:35:48.991821+0000","last_fullsized":"2024-06-06T03:35:48.991821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:27:12.330943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991256+0000","last_change":"2024-06-06T03:35:13.854251+0000","last_active":"2024-06-06T03:35:48.991256+0000","last_peered":"2024-06-06T03:35:48.991256+0000","last_clean":"2024-06-06T03:35:48.991256+0000","last_became_active":"2024-06-06T03:35:13.853915+0000","last_became_peered":"2024-06-06T03:35:13.853915+0000","last_unstale":"2024-06-06T03:35:48.991256+0000","last_undegraded":"2024-06-06T03:35:48.991256+0000","last_fullsized":"2024-06-06T03:35:48.991256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:08.813316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984146+0000","last_change":"2024-06-06T03:35:13.849998+0000","last_active":"2024-06-06T03:35:48.984146+0000","last_peered":"2024-06-06T03:35:48.984146+0000","last_clean":"2024-06-06T03:35:48.984146+0000","last_became_active":"2024-06-06T03:35:13.848789+0000","last_became_peered":"2024-06-06T03:35:13.848789+0000","last_unstale":"2024-06-06T03:35:48.984146+0000","last_undegraded":"2024-06-06T03:35:48.984146+0000","last_fullsized":"2024-06-06T03:35:48.984146+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:53:39.043356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990461+0000","last_change":"2024-06-06T03:35:13.858020+0000","last_active":"2024-06-06T03:35:48.990461+0000","last_peered":"2024-06-06T03:35:48.990461+0000","last_clean":"2024-06-06T03:35:48.990461+0000","last_became_active":"2024-06-06T03:35:13.857723+0000","last_became_peered":"2024-06-06T03:35:13.857723+0000","last_unstale":"2024-06-06T03:35:48.990461+0000","last_undegraded":"2024-06-06T03:35:48.990461+0000","last_fullsized":"2024-06-06T03:35:48.990461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:08:08.651213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984730+0000","last_change":"2024-06-06T03:35:13.867546+0000","last_active":"2024-06-06T03:35:48.984730+0000","last_peered":"2024-06-06T03:35:48.984730+0000","last_clean":"2024-06-06T03:35:48.984730+0000","last_became_active":"2024-06-06T03:35:13.867182+0000","last_became_peered":"2024-06-06T03:35:13.867182+0000","last_unstale":"2024-06-06T03:35:48.984730+0000","last_undegraded":"2024-06-06T03:35:48.984730+0000","last_fullsized":"2024-06-06T03:35:48.984730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:25:55.831943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983770+0000","last_change":"2024-06-06T03:35:13.849080+0000","last_active":"2024-06-06T03:35:48.983770+0000","last_peered":"2024-06-06T03:35:48.983770+0000","last_clean":"2024-06-06T03:35:48.983770+0000","last_became_active":"2024-06-06T03:35:13.848476+0000","last_became_peered":"2024-06-06T03:35:13.848476+0000","last_unstale":"2024-06-06T03:35:48.983770+0000","last_undegraded":"2024-06-06T03:35:48.983770+0000","last_fullsized":"2024-06-06T03:35:48.983770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:48:37.306528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987065+0000","last_change":"2024-06-06T03:35:13.848442+0000","last_active":"2024-06-06T03:35:48.987065+0000","last_peered":"2024-06-06T03:35:48.987065+0000","last_clean":"2024-06-06T03:35:48.987065+0000","last_became_active":"2024-06-06T03:35:13.848074+0000","last_became_peered":"2024-06-06T03:35:13.848074+0000","last_unstale":"2024-06-06T03:35:48.987065+0000","last_undegraded":"2024-06-06T03:35:48.987065+0000","last_fullsized":"2024-06-06T03:35:48.987065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:07:00.197671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280416,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65279,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":296,"num_read_kb":223,"num_write":242,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":194699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383705,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":480334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645343,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1129328,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808867,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070505,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":480418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1073437,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495667,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":480259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":539249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":481208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":538331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":479694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":479730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":1072725,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34027,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34027,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34027,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":59813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T03:36:03.592 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T03:36:03.845 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:04.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:04 smithi131 ceph-mon[36687]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:36:04.312 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:04 smithi040 ceph-mon[32220]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:36:04.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:04 smithi104 ceph-mon[36569]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:36:05.232 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:05 smithi040 ceph-mon[32220]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 10 op/s; 9 B/s, 0 objects/s recovering 2024-06-06T03:36:05.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:05 smithi131 ceph-mon[36687]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 10 op/s; 9 B/s, 0 objects/s recovering 2024-06-06T03:36:05.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:05 smithi104 ceph-mon[36569]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 10 op/s; 9 B/s, 0 objects/s recovering 2024-06-06T03:36:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.428+0000 7f82fcc48700 1 -- 172.21.15.40:0/70024433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f8102f10 msgr2=0x7f82f810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.428+0000 7f82fcc48700 1 --2- 172.21.15.40:0/70024433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f8102f10 0x7f82f810f330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f82f000b0a0 tx=0x7f82f000b3b0 comp rx=0 tx=0).stop 2024-06-06T03:36:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.428+0000 7f82fcc48700 1 -- 172.21.15.40:0/70024433 shutdown_connections 2024-06-06T03:36:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.428+0000 7f82fcc48700 1 --2- 172.21.15.40:0/70024433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82f81084f0 0x7f82f81088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.428+0000 7f82fcc48700 1 --2- 172.21.15.40:0/70024433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f8102f10 0x7f82f810f330 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.428+0000 7f82fcc48700 1 --2- 172.21.15.40:0/70024433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82f8102550 0x7f82f81029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.428+0000 7f82fcc48700 1 -- 172.21.15.40:0/70024433 >> 172.21.15.40:0/70024433 conn(0x7f82f8079650 msgr2=0x7f82f8079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.428+0000 7f82fcc48700 1 -- 172.21.15.40:0/70024433 shutdown_connections 2024-06-06T03:36:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.428+0000 7f82fcc48700 1 -- 172.21.15.40:0/70024433 wait complete. 2024-06-06T03:36:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82fcc48700 1 Processor -- start 2024-06-06T03:36:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82fcc48700 1 -- start start 2024-06-06T03:36:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82fcc48700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82f8102550 0x7f82f8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82fcc48700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f8102f10 0x7f82f8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82fcc48700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82f81084f0 0x7f82f8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82fcc48700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f82f8112770 con 0x7f82f8102f10 2024-06-06T03:36:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82fcc48700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f82f81125f0 con 0x7f82f8102550 2024-06-06T03:36:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82fcc48700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f82f81128f0 con 0x7f82f81084f0 2024-06-06T03:36:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82f5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f8102f10 0x7f82f8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82f5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f8102f10 0x7f82f8074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57756/0 (socket says 172.21.15.40:57756) 2024-06-06T03:36:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82f5d9b700 1 -- 172.21.15.40:0/1604769169 learned_addr learned my addr 172.21.15.40:0/1604769169 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82f6d9d700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82f81084f0 0x7f82f8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82f659c700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82f8102550 0x7f82f8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82f5d9b700 1 -- 172.21.15.40:0/1604769169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82f81084f0 msgr2=0x7f82f8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82f5d9b700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82f81084f0 0x7f82f8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82f5d9b700 1 -- 172.21.15.40:0/1604769169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82f8102550 msgr2=0x7f82f8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82f5d9b700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82f8102550 0x7f82f8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82f5d9b700 1 -- 172.21.15.40:0/1604769169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f81a71d0 con 0x7f82f8102f10 2024-06-06T03:36:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82f6d9d700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82f81084f0 0x7f82f8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:36:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.429+0000 7f82f659c700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82f8102550 0x7f82f8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:36:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.430+0000 7f82f5d9b700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f8102f10 0x7f82f8074cd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00d980 tx=0x7f82ec00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.430+0000 7f82e77fe700 1 -- 172.21.15.40:0/1604769169 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec016070 con 0x7f82f8102f10 2024-06-06T03:36:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.430+0000 7f82e77fe700 1 -- 172.21.15.40:0/1604769169 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ec004500 con 0x7f82f8102f10 2024-06-06T03:36:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.430+0000 7f82e77fe700 1 -- 172.21.15.40:0/1604769169 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec004d00 con 0x7f82f8102f10 2024-06-06T03:36:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.430+0000 7f82fcc48700 1 -- 172.21.15.40:0/1604769169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f81a74c0 con 0x7f82f8102f10 2024-06-06T03:36:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.430+0000 7f82fcc48700 1 -- 172.21.15.40:0/1604769169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f81a7a50 con 0x7f82f8102f10 2024-06-06T03:36:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.431+0000 7f82fcc48700 1 -- 172.21.15.40:0/1604769169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f8069180 con 0x7f82f8102f10 2024-06-06T03:36:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.432+0000 7f82e77fe700 1 -- 172.21.15.40:0/1604769169 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f82ec00bbf0 con 0x7f82f8102f10 2024-06-06T03:36:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.433+0000 7f82e77fe700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f82e00a2d30 0x7f82e00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.433+0000 7f82f659c700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f82e00a2d30 0x7f82e00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.433+0000 7f82e77fe700 1 -- 172.21.15.40:0/1604769169 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f82ec0cef80 con 0x7f82f8102f10 2024-06-06T03:36:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.433+0000 7f82f659c700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f82e00a2d30 0x7f82e00a51f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f82e8005e10 tx=0x7f82e8005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.438+0000 7f82e77fe700 1 -- 172.21.15.40:0/1604769169 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f82ec01f020 con 0x7f82f8102f10 2024-06-06T03:36:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.639+0000 7f82fcc48700 1 -- 172.21.15.40:0/1604769169 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82f80628d0 con 0x7f82e00a2d30 2024-06-06T03:36:05.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.647+0000 7f82e77fe700 1 -- 172.21.15.40:0/1604769169 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258940 (secure 0 0 0) 0x7f82f80628d0 con 0x7f82e00a2d30 2024-06-06T03:36:05.651 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:36:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.650+0000 7f82fcc48700 1 -- 172.21.15.40:0/1604769169 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f82e00a2d30 msgr2=0x7f82e00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.650+0000 7f82fcc48700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f82e00a2d30 0x7f82e00a51f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f82e8005e10 tx=0x7f82e8005d80 comp rx=0 tx=0).stop 2024-06-06T03:36:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.650+0000 7f82fcc48700 1 -- 172.21.15.40:0/1604769169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f8102f10 msgr2=0x7f82f8074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.650+0000 7f82fcc48700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f8102f10 0x7f82f8074cd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00d980 tx=0x7f82ec00dd40 comp rx=0 tx=0).stop 2024-06-06T03:36:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.650+0000 7f82fcc48700 1 -- 172.21.15.40:0/1604769169 shutdown_connections 2024-06-06T03:36:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.650+0000 7f82fcc48700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f82e00a2d30 0x7f82e00a51f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.650+0000 7f82fcc48700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82f81084f0 0x7f82f8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.650+0000 7f82fcc48700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82f8102550 0x7f82f8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.650+0000 7f82fcc48700 1 --2- 172.21.15.40:0/1604769169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f8102f10 0x7f82f8074cd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.650+0000 7f82fcc48700 1 -- 172.21.15.40:0/1604769169 >> 172.21.15.40:0/1604769169 conn(0x7f82f8079650 msgr2=0x7f82f81005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.650+0000 7f82fcc48700 1 -- 172.21.15.40:0/1604769169 shutdown_connections 2024-06-06T03:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:05.650+0000 7f82fcc48700 1 -- 172.21.15.40:0/1604769169 wait complete. 2024-06-06T03:36:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-06-06T03:36:06.320 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-06-06T03:36:03.691001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655881,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":442,"num_read_kb":342,"num_write":425,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365480,"kb_used_data":12144,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576344,"statfs":{"total":1151940427776,"available":1151566176256,"internally_reserved":0,"allocated":12435456,"data_stored":7716003,"data_compressed":107001,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15994,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":112,"num_read_kb":80,"num_write":18,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002877"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987328+0000","last_change":"2024-06-06T03:35:13.854938+0000","last_active":"2024-06-06T03:35:48.987328+0000","last_peered":"2024-06-06T03:35:48.987328+0000","last_clean":"2024-06-06T03:35:48.987328+0000","last_became_active":"2024-06-06T03:35:13.853574+0000","last_became_peered":"2024-06-06T03:35:13.853574+0000","last_unstale":"2024-06-06T03:35:48.987328+0000","last_undegraded":"2024-06-06T03:35:48.987328+0000","last_fullsized":"2024-06-06T03:35:48.987328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:21:09.256002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990557+0000","last_change":"2024-06-06T03:35:13.858299+0000","last_active":"2024-06-06T03:35:48.990557+0000","last_peered":"2024-06-06T03:35:48.990557+0000","last_clean":"2024-06-06T03:35:48.990557+0000","last_became_active":"2024-06-06T03:35:13.857993+0000","last_became_peered":"2024-06-06T03:35:13.857993+0000","last_unstale":"2024-06-06T03:35:48.990557+0000","last_undegraded":"2024-06-06T03:35:48.990557+0000","last_fullsized":"2024-06-06T03:35:48.990557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.811056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990502+0000","last_change":"2024-06-06T03:35:13.852783+0000","last_active":"2024-06-06T03:35:48.990502+0000","last_peered":"2024-06-06T03:35:48.990502+0000","last_clean":"2024-06-06T03:35:48.990502+0000","last_became_active":"2024-06-06T03:35:13.852468+0000","last_became_peered":"2024-06-06T03:35:13.852468+0000","last_unstale":"2024-06-06T03:35:48.990502+0000","last_undegraded":"2024-06-06T03:35:48.990502+0000","last_fullsized":"2024-06-06T03:35:48.990502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:04:59.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.004002+0000","last_change":"2024-06-06T03:35:13.845223+0000","last_active":"2024-06-06T03:35:49.004002+0000","last_peered":"2024-06-06T03:35:49.004002+0000","last_clean":"2024-06-06T03:35:49.004002+0000","last_became_active":"2024-06-06T03:35:13.844854+0000","last_became_peered":"2024-06-06T03:35:13.844854+0000","last_unstale":"2024-06-06T03:35:49.004002+0000","last_undegraded":"2024-06-06T03:35:49.004002+0000","last_fullsized":"2024-06-06T03:35:49.004002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:35:20.058188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991571+0000","last_change":"2024-06-06T03:35:13.853703+0000","last_active":"2024-06-06T03:35:48.991571+0000","last_peered":"2024-06-06T03:35:48.991571+0000","last_clean":"2024-06-06T03:35:48.991571+0000","last_became_active":"2024-06-06T03:35:13.853358+0000","last_became_peered":"2024-06-06T03:35:13.853358+0000","last_unstale":"2024-06-06T03:35:48.991571+0000","last_undegraded":"2024-06-06T03:35:48.991571+0000","last_fullsized":"2024-06-06T03:35:48.991571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:23:39.092306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988483+0000","last_change":"2024-06-06T03:35:13.859414+0000","last_active":"2024-06-06T03:35:48.988483+0000","last_peered":"2024-06-06T03:35:48.988483+0000","last_clean":"2024-06-06T03:35:48.988483+0000","last_became_active":"2024-06-06T03:35:13.859038+0000","last_became_peered":"2024-06-06T03:35:13.859038+0000","last_unstale":"2024-06-06T03:35:48.988483+0000","last_undegraded":"2024-06-06T03:35:48.988483+0000","last_fullsized":"2024-06-06T03:35:48.988483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:14:31.780064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985258+0000","last_change":"2024-06-06T03:35:13.859245+0000","last_active":"2024-06-06T03:35:48.985258+0000","last_peered":"2024-06-06T03:35:48.985258+0000","last_clean":"2024-06-06T03:35:48.985258+0000","last_became_active":"2024-06-06T03:35:13.858857+0000","last_became_peered":"2024-06-06T03:35:13.858857+0000","last_unstale":"2024-06-06T03:35:48.985258+0000","last_undegraded":"2024-06-06T03:35:48.985258+0000","last_fullsized":"2024-06-06T03:35:48.985258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:03:23.240007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990694+0000","last_change":"2024-06-06T03:35:13.851249+0000","last_active":"2024-06-06T03:35:48.990694+0000","last_peered":"2024-06-06T03:35:48.990694+0000","last_clean":"2024-06-06T03:35:48.990694+0000","last_became_active":"2024-06-06T03:35:13.850777+0000","last_became_peered":"2024-06-06T03:35:13.850777+0000","last_unstale":"2024-06-06T03:35:48.990694+0000","last_undegraded":"2024-06-06T03:35:48.990694+0000","last_fullsized":"2024-06-06T03:35:48.990694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:53:48.463672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985036+0000","last_change":"2024-06-06T03:35:13.852950+0000","last_active":"2024-06-06T03:35:48.985036+0000","last_peered":"2024-06-06T03:35:48.985036+0000","last_clean":"2024-06-06T03:35:48.985036+0000","last_became_active":"2024-06-06T03:35:13.852160+0000","last_became_peered":"2024-06-06T03:35:13.852160+0000","last_unstale":"2024-06-06T03:35:48.985036+0000","last_undegraded":"2024-06-06T03:35:48.985036+0000","last_fullsized":"2024-06-06T03:35:48.985036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:00.498440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988970+0000","last_change":"2024-06-06T03:35:13.856081+0000","last_active":"2024-06-06T03:35:48.988970+0000","last_peered":"2024-06-06T03:35:48.988970+0000","last_clean":"2024-06-06T03:35:48.988970+0000","last_became_active":"2024-06-06T03:35:13.855718+0000","last_became_peered":"2024-06-06T03:35:13.855718+0000","last_unstale":"2024-06-06T03:35:48.988970+0000","last_undegraded":"2024-06-06T03:35:48.988970+0000","last_fullsized":"2024-06-06T03:35:48.988970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:33:38.171996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985594+0000","last_change":"2024-06-06T03:35:13.861850+0000","last_active":"2024-06-06T03:35:48.985594+0000","last_peered":"2024-06-06T03:35:48.985594+0000","last_clean":"2024-06-06T03:35:48.985594+0000","last_became_active":"2024-06-06T03:35:13.861513+0000","last_became_peered":"2024-06-06T03:35:13.861513+0000","last_unstale":"2024-06-06T03:35:48.985594+0000","last_undegraded":"2024-06-06T03:35:48.985594+0000","last_fullsized":"2024-06-06T03:35:48.985594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:32:45.938302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.003896+0000","last_change":"2024-06-06T03:35:13.847742+0000","last_active":"2024-06-06T03:35:49.003896+0000","last_peered":"2024-06-06T03:35:49.003896+0000","last_clean":"2024-06-06T03:35:49.003896+0000","last_became_active":"2024-06-06T03:35:13.847409+0000","last_became_peered":"2024-06-06T03:35:13.847409+0000","last_unstale":"2024-06-06T03:35:49.003896+0000","last_undegraded":"2024-06-06T03:35:49.003896+0000","last_fullsized":"2024-06-06T03:35:49.003896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:20:31.353919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983746+0000","last_change":"2024-06-06T03:35:13.848785+0000","last_active":"2024-06-06T03:35:48.983746+0000","last_peered":"2024-06-06T03:35:48.983746+0000","last_clean":"2024-06-06T03:35:48.983746+0000","last_became_active":"2024-06-06T03:35:13.848048+0000","last_became_peered":"2024-06-06T03:35:13.848048+0000","last_unstale":"2024-06-06T03:35:48.983746+0000","last_undegraded":"2024-06-06T03:35:48.983746+0000","last_fullsized":"2024-06-06T03:35:48.983746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:43.657761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990820+0000","last_change":"2024-06-06T03:35:13.856083+0000","last_active":"2024-06-06T03:35:48.990820+0000","last_peered":"2024-06-06T03:35:48.990820+0000","last_clean":"2024-06-06T03:35:48.990820+0000","last_became_active":"2024-06-06T03:35:13.854390+0000","last_became_peered":"2024-06-06T03:35:13.854390+0000","last_unstale":"2024-06-06T03:35:48.990820+0000","last_undegraded":"2024-06-06T03:35:48.990820+0000","last_fullsized":"2024-06-06T03:35:48.990820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:51:37.163814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990153+0000","last_change":"2024-06-06T03:35:13.852611+0000","last_active":"2024-06-06T03:35:48.990153+0000","last_peered":"2024-06-06T03:35:48.990153+0000","last_clean":"2024-06-06T03:35:48.990153+0000","last_became_active":"2024-06-06T03:35:13.852244+0000","last_became_peered":"2024-06-06T03:35:13.852244+0000","last_unstale":"2024-06-06T03:35:48.990153+0000","last_undegraded":"2024-06-06T03:35:48.990153+0000","last_fullsized":"2024-06-06T03:35:48.990153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:59:53.513150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984532+0000","last_change":"2024-06-06T03:35:13.857840+0000","last_active":"2024-06-06T03:35:48.984532+0000","last_peered":"2024-06-06T03:35:48.984532+0000","last_clean":"2024-06-06T03:35:48.984532+0000","last_became_active":"2024-06-06T03:35:13.857137+0000","last_became_peered":"2024-06-06T03:35:13.857137+0000","last_unstale":"2024-06-06T03:35:48.984532+0000","last_undegraded":"2024-06-06T03:35:48.984532+0000","last_fullsized":"2024-06-06T03:35:48.984532+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:42:07.924735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987452+0000","last_change":"2024-06-06T03:35:13.857033+0000","last_active":"2024-06-06T03:35:48.987452+0000","last_peered":"2024-06-06T03:35:48.987452+0000","last_clean":"2024-06-06T03:35:48.987452+0000","last_became_active":"2024-06-06T03:35:13.856519+0000","last_became_peered":"2024-06-06T03:35:13.856519+0000","last_unstale":"2024-06-06T03:35:48.987452+0000","last_undegraded":"2024-06-06T03:35:48.987452+0000","last_fullsized":"2024-06-06T03:35:48.987452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:05:25.668344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.986761+0000","last_change":"2024-06-06T03:35:13.851356+0000","last_active":"2024-06-06T03:35:48.986761+0000","last_peered":"2024-06-06T03:35:48.986761+0000","last_clean":"2024-06-06T03:35:48.986761+0000","last_became_active":"2024-06-06T03:35:13.851030+0000","last_became_peered":"2024-06-06T03:35:13.851030+0000","last_unstale":"2024-06-06T03:35:48.986761+0000","last_undegraded":"2024-06-06T03:35:48.986761+0000","last_fullsized":"2024-06-06T03:35:48.986761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:49:38.833022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991734+0000","last_change":"2024-06-06T03:35:13.858613+0000","last_active":"2024-06-06T03:35:48.991734+0000","last_peered":"2024-06-06T03:35:48.991734+0000","last_clean":"2024-06-06T03:35:48.991734+0000","last_became_active":"2024-06-06T03:35:13.858182+0000","last_became_peered":"2024-06-06T03:35:13.858182+0000","last_unstale":"2024-06-06T03:35:48.991734+0000","last_undegraded":"2024-06-06T03:35:48.991734+0000","last_fullsized":"2024-06-06T03:35:48.991734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:23:08.216431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:57.991918+0000","last_change":"2024-06-06T03:35:09.821896+0000","last_active":"2024-06-06T03:35:57.991918+0000","last_peered":"2024-06-06T03:35:57.991918+0000","last_clean":"2024-06-06T03:35:57.991918+0000","last_became_active":"2024-06-06T03:35:09.821567+0000","last_became_peered":"2024-06-06T03:35:09.821567+0000","last_unstale":"2024-06-06T03:35:57.991918+0000","last_undegraded":"2024-06-06T03:35:57.991918+0000","last_fullsized":"2024-06-06T03:35:57.991918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:04:51.918393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991350+0000","last_change":"2024-06-06T03:35:13.857357+0000","last_active":"2024-06-06T03:35:48.991350+0000","last_peered":"2024-06-06T03:35:48.991350+0000","last_clean":"2024-06-06T03:35:48.991350+0000","last_became_active":"2024-06-06T03:35:13.857016+0000","last_became_peered":"2024-06-06T03:35:13.857016+0000","last_unstale":"2024-06-06T03:35:48.991350+0000","last_undegraded":"2024-06-06T03:35:48.991350+0000","last_fullsized":"2024-06-06T03:35:48.991350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:42.566425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991461+0000","last_change":"2024-06-06T03:35:10.277316+0000","last_active":"2024-06-06T03:35:48.991461+0000","last_peered":"2024-06-06T03:35:48.991461+0000","last_clean":"2024-06-06T03:35:48.991461+0000","last_became_active":"2024-06-06T03:35:10.276986+0000","last_became_peered":"2024-06-06T03:35:10.276986+0000","last_unstale":"2024-06-06T03:35:48.991461+0000","last_undegraded":"2024-06-06T03:35:48.991461+0000","last_fullsized":"2024-06-06T03:35:48.991461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:17:29.034224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990383+0000","last_change":"2024-06-06T03:35:13.863069+0000","last_active":"2024-06-06T03:35:48.990383+0000","last_peered":"2024-06-06T03:35:48.990383+0000","last_clean":"2024-06-06T03:35:48.990383+0000","last_became_active":"2024-06-06T03:35:13.862647+0000","last_became_peered":"2024-06-06T03:35:13.862647+0000","last_unstale":"2024-06-06T03:35:48.990383+0000","last_undegraded":"2024-06-06T03:35:48.990383+0000","last_fullsized":"2024-06-06T03:35:48.990383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:48.252606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992306+0000","last_change":"2024-06-06T03:35:09.824977+0000","last_active":"2024-06-06T03:35:48.992306+0000","last_peered":"2024-06-06T03:35:48.992306+0000","last_clean":"2024-06-06T03:35:48.992306+0000","last_became_active":"2024-06-06T03:35:09.824643+0000","last_became_peered":"2024-06-06T03:35:09.824643+0000","last_unstale":"2024-06-06T03:35:48.992306+0000","last_undegraded":"2024-06-06T03:35:48.992306+0000","last_fullsized":"2024-06-06T03:35:48.992306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:35:37.208023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984591+0000","last_change":"2024-06-06T03:35:13.860572+0000","last_active":"2024-06-06T03:35:48.984591+0000","last_peered":"2024-06-06T03:35:48.984591+0000","last_clean":"2024-06-06T03:35:48.984591+0000","last_became_active":"2024-06-06T03:35:13.858398+0000","last_became_peered":"2024-06-06T03:35:13.858398+0000","last_unstale":"2024-06-06T03:35:48.984591+0000","last_undegraded":"2024-06-06T03:35:48.984591+0000","last_fullsized":"2024-06-06T03:35:48.984591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:02:12.683096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991795+0000","last_change":"2024-06-06T03:35:09.827016+0000","last_active":"2024-06-06T03:35:48.991795+0000","last_peered":"2024-06-06T03:35:48.991795+0000","last_clean":"2024-06-06T03:35:48.991795+0000","last_became_active":"2024-06-06T03:35:09.826576+0000","last_became_peered":"2024-06-06T03:35:09.826576+0000","last_unstale":"2024-06-06T03:35:48.991795+0000","last_undegraded":"2024-06-06T03:35:48.991795+0000","last_fullsized":"2024-06-06T03:35:48.991795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:24.120549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984420+0000","last_change":"2024-06-06T03:35:13.853988+0000","last_active":"2024-06-06T03:35:48.984420+0000","last_peered":"2024-06-06T03:35:48.984420+0000","last_clean":"2024-06-06T03:35:48.984420+0000","last_became_active":"2024-06-06T03:35:13.852343+0000","last_became_peered":"2024-06-06T03:35:13.852343+0000","last_unstale":"2024-06-06T03:35:48.984420+0000","last_undegraded":"2024-06-06T03:35:48.984420+0000","last_fullsized":"2024-06-06T03:35:48.984420+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:39:03.054045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:56.976623+0000","last_change":"2024-06-06T03:35:09.829109+0000","last_active":"2024-06-06T03:35:56.976623+0000","last_peered":"2024-06-06T03:35:56.976623+0000","last_clean":"2024-06-06T03:35:56.976623+0000","last_became_active":"2024-06-06T03:35:09.828734+0000","last_became_peered":"2024-06-06T03:35:09.828734+0000","last_unstale":"2024-06-06T03:35:56.976623+0000","last_undegraded":"2024-06-06T03:35:56.976623+0000","last_fullsized":"2024-06-06T03:35:56.976623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:50.475600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992478+0000","last_change":"2024-06-06T03:35:13.859198+0000","last_active":"2024-06-06T03:35:48.992478+0000","last_peered":"2024-06-06T03:35:48.992478+0000","last_clean":"2024-06-06T03:35:48.992478+0000","last_became_active":"2024-06-06T03:35:13.858863+0000","last_became_peered":"2024-06-06T03:35:13.858863+0000","last_unstale":"2024-06-06T03:35:48.992478+0000","last_undegraded":"2024-06-06T03:35:48.992478+0000","last_fullsized":"2024-06-06T03:35:48.992478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:57.153174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988385+0000","last_change":"2024-06-06T03:35:09.824352+0000","last_active":"2024-06-06T03:35:48.988385+0000","last_peered":"2024-06-06T03:35:48.988385+0000","last_clean":"2024-06-06T03:35:48.988385+0000","last_became_active":"2024-06-06T03:35:09.823998+0000","last_became_peered":"2024-06-06T03:35:09.823998+0000","last_unstale":"2024-06-06T03:35:48.988385+0000","last_undegraded":"2024-06-06T03:35:48.988385+0000","last_fullsized":"2024-06-06T03:35:48.988385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:47:24.646067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984826+0000","last_change":"2024-06-06T03:35:13.836724+0000","last_active":"2024-06-06T03:35:48.984826+0000","last_peered":"2024-06-06T03:35:48.984826+0000","last_clean":"2024-06-06T03:35:48.984826+0000","last_became_active":"2024-06-06T03:35:13.836180+0000","last_became_peered":"2024-06-06T03:35:13.836180+0000","last_unstale":"2024-06-06T03:35:48.984826+0000","last_undegraded":"2024-06-06T03:35:48.984826+0000","last_fullsized":"2024-06-06T03:35:48.984826+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:08.868462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983408+0000","last_change":"2024-06-06T03:35:09.823047+0000","last_active":"2024-06-06T03:35:48.983408+0000","last_peered":"2024-06-06T03:35:48.983408+0000","last_clean":"2024-06-06T03:35:48.983408+0000","last_became_active":"2024-06-06T03:35:09.822744+0000","last_became_peered":"2024-06-06T03:35:09.822744+0000","last_unstale":"2024-06-06T03:35:48.983408+0000","last_undegraded":"2024-06-06T03:35:48.983408+0000","last_fullsized":"2024-06-06T03:35:48.983408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:20:20.165682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.989848+0000","last_change":"2024-06-06T03:35:13.857222+0000","last_active":"2024-06-06T03:35:48.989848+0000","last_peered":"2024-06-06T03:35:48.989848+0000","last_clean":"2024-06-06T03:35:48.989848+0000","last_became_active":"2024-06-06T03:35:13.856873+0000","last_became_peered":"2024-06-06T03:35:13.856873+0000","last_unstale":"2024-06-06T03:35:48.989848+0000","last_undegraded":"2024-06-06T03:35:48.989848+0000","last_fullsized":"2024-06-06T03:35:48.989848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:32:16.646275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984083+0000","last_change":"2024-06-06T03:35:09.826036+0000","last_active":"2024-06-06T03:35:48.984083+0000","last_peered":"2024-06-06T03:35:48.984083+0000","last_clean":"2024-06-06T03:35:48.984083+0000","last_became_active":"2024-06-06T03:35:09.825690+0000","last_became_peered":"2024-06-06T03:35:09.825690+0000","last_unstale":"2024-06-06T03:35:48.984083+0000","last_undegraded":"2024-06-06T03:35:48.984083+0000","last_fullsized":"2024-06-06T03:35:48.984083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:03:35.858828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.986086+0000","last_change":"2024-06-06T03:35:13.849016+0000","last_active":"2024-06-06T03:35:48.986086+0000","last_peered":"2024-06-06T03:35:48.986086+0000","last_clean":"2024-06-06T03:35:48.986086+0000","last_became_active":"2024-06-06T03:35:13.848709+0000","last_became_peered":"2024-06-06T03:35:13.848709+0000","last_unstale":"2024-06-06T03:35:48.986086+0000","last_undegraded":"2024-06-06T03:35:48.986086+0000","last_fullsized":"2024-06-06T03:35:48.986086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:42.778612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":84,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:36:00.981680+0000","last_change":"2024-06-06T03:35:09.827072+0000","last_active":"2024-06-06T03:36:00.981680+0000","last_peered":"2024-06-06T03:36:00.981680+0000","last_clean":"2024-06-06T03:36:00.981680+0000","last_became_active":"2024-06-06T03:35:09.826734+0000","last_became_peered":"2024-06-06T03:35:09.826734+0000","last_unstale":"2024-06-06T03:36:00.981680+0000","last_undegraded":"2024-06-06T03:36:00.981680+0000","last_fullsized":"2024-06-06T03:36:00.981680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:07:13.388398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985601+0000","last_change":"2024-06-06T03:35:13.856883+0000","last_active":"2024-06-06T03:35:48.985601+0000","last_peered":"2024-06-06T03:35:48.985601+0000","last_clean":"2024-06-06T03:35:48.985601+0000","last_became_active":"2024-06-06T03:35:13.856521+0000","last_became_peered":"2024-06-06T03:35:13.856521+0000","last_unstale":"2024-06-06T03:35:48.985601+0000","last_undegraded":"2024-06-06T03:35:48.985601+0000","last_fullsized":"2024-06-06T03:35:48.985601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:19.466218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985292+0000","last_change":"2024-06-06T03:35:09.829153+0000","last_active":"2024-06-06T03:35:48.985292+0000","last_peered":"2024-06-06T03:35:48.985292+0000","last_clean":"2024-06-06T03:35:48.985292+0000","last_became_active":"2024-06-06T03:35:09.828797+0000","last_became_peered":"2024-06-06T03:35:09.828797+0000","last_unstale":"2024-06-06T03:35:48.985292+0000","last_undegraded":"2024-06-06T03:35:48.985292+0000","last_fullsized":"2024-06-06T03:35:48.985292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:06:02.986396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990268+0000","last_change":"2024-06-06T03:35:13.857094+0000","last_active":"2024-06-06T03:35:48.990268+0000","last_peered":"2024-06-06T03:35:48.990268+0000","last_clean":"2024-06-06T03:35:48.990268+0000","last_became_active":"2024-06-06T03:35:13.856778+0000","last_became_peered":"2024-06-06T03:35:13.856778+0000","last_unstale":"2024-06-06T03:35:48.990268+0000","last_undegraded":"2024-06-06T03:35:48.990268+0000","last_fullsized":"2024-06-06T03:35:48.990268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:50:30.097762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984506+0000","last_change":"2024-06-06T03:35:10.277802+0000","last_active":"2024-06-06T03:35:48.984506+0000","last_peered":"2024-06-06T03:35:48.984506+0000","last_clean":"2024-06-06T03:35:48.984506+0000","last_became_active":"2024-06-06T03:35:10.277417+0000","last_became_peered":"2024-06-06T03:35:10.277417+0000","last_unstale":"2024-06-06T03:35:48.984506+0000","last_undegraded":"2024-06-06T03:35:48.984506+0000","last_fullsized":"2024-06-06T03:35:48.984506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:44:35.178184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984699+0000","last_change":"2024-06-06T03:35:13.858160+0000","last_active":"2024-06-06T03:35:48.984699+0000","last_peered":"2024-06-06T03:35:48.984699+0000","last_clean":"2024-06-06T03:35:48.984699+0000","last_became_active":"2024-06-06T03:35:13.853149+0000","last_became_peered":"2024-06-06T03:35:13.853149+0000","last_unstale":"2024-06-06T03:35:48.984699+0000","last_undegraded":"2024-06-06T03:35:48.984699+0000","last_fullsized":"2024-06-06T03:35:48.984699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:10:06.697274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'30","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:36:00.983512+0000","last_change":"2024-06-06T03:35:09.829443+0000","last_active":"2024-06-06T03:36:00.983512+0000","last_peered":"2024-06-06T03:36:00.983512+0000","last_clean":"2024-06-06T03:36:00.983512+0000","last_became_active":"2024-06-06T03:35:09.828093+0000","last_became_peered":"2024-06-06T03:35:09.828093+0000","last_unstale":"2024-06-06T03:36:00.983512+0000","last_undegraded":"2024-06-06T03:36:00.983512+0000","last_fullsized":"2024-06-06T03:36:00.983512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:05:19.395177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58077,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":62,"num_read_kb":87,"num_write":32,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991968+0000","last_change":"2024-06-06T03:35:13.850869+0000","last_active":"2024-06-06T03:35:48.991968+0000","last_peered":"2024-06-06T03:35:48.991968+0000","last_clean":"2024-06-06T03:35:48.991968+0000","last_became_active":"2024-06-06T03:35:13.849489+0000","last_became_peered":"2024-06-06T03:35:13.849489+0000","last_unstale":"2024-06-06T03:35:48.991968+0000","last_undegraded":"2024-06-06T03:35:48.991968+0000","last_fullsized":"2024-06-06T03:35:48.991968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:27:45.052033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991491+0000","last_change":"2024-06-06T03:35:09.823117+0000","last_active":"2024-06-06T03:35:48.991491+0000","last_peered":"2024-06-06T03:35:48.991491+0000","last_clean":"2024-06-06T03:35:48.991491+0000","last_became_active":"2024-06-06T03:35:09.822624+0000","last_became_peered":"2024-06-06T03:35:09.822624+0000","last_unstale":"2024-06-06T03:35:48.991491+0000","last_undegraded":"2024-06-06T03:35:48.991491+0000","last_fullsized":"2024-06-06T03:35:48.991491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:59.171343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983437+0000","last_change":"2024-06-06T03:35:13.861043+0000","last_active":"2024-06-06T03:35:48.983437+0000","last_peered":"2024-06-06T03:35:48.983437+0000","last_clean":"2024-06-06T03:35:48.983437+0000","last_became_active":"2024-06-06T03:35:13.860718+0000","last_became_peered":"2024-06-06T03:35:13.860718+0000","last_unstale":"2024-06-06T03:35:48.983437+0000","last_undegraded":"2024-06-06T03:35:48.983437+0000","last_fullsized":"2024-06-06T03:35:48.983437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:34:04.595780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984355+0000","last_change":"2024-06-06T03:35:09.825505+0000","last_active":"2024-06-06T03:35:48.984355+0000","last_peered":"2024-06-06T03:35:48.984355+0000","last_clean":"2024-06-06T03:35:48.984355+0000","last_became_active":"2024-06-06T03:35:09.825127+0000","last_became_peered":"2024-06-06T03:35:09.825127+0000","last_unstale":"2024-06-06T03:35:48.984355+0000","last_undegraded":"2024-06-06T03:35:48.984355+0000","last_fullsized":"2024-06-06T03:35:48.984355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:07:27.664589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987834+0000","last_change":"2024-06-06T03:35:48.987834+0000","last_active":"2024-06-06T03:35:48.987834+0000","last_peered":"2024-06-06T03:35:48.987834+0000","last_clean":"2024-06-06T03:35:48.987834+0000","last_became_active":"2024-06-06T03:35:48.987515+0000","last_became_peered":"2024-06-06T03:35:48.987515+0000","last_unstale":"2024-06-06T03:35:48.987834+0000","last_undegraded":"2024-06-06T03:35:48.987834+0000","last_fullsized":"2024-06-06T03:35:48.987834+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:05:19.342289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:36:01.983604+0000","last_change":"2024-06-06T03:35:09.826192+0000","last_active":"2024-06-06T03:36:01.983604+0000","last_peered":"2024-06-06T03:36:01.983604+0000","last_clean":"2024-06-06T03:36:01.983604+0000","last_became_active":"2024-06-06T03:35:09.825481+0000","last_became_peered":"2024-06-06T03:35:09.825481+0000","last_unstale":"2024-06-06T03:36:01.983604+0000","last_undegraded":"2024-06-06T03:36:01.983604+0000","last_fullsized":"2024-06-06T03:36:01.983604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:18:30.772991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985035+0000","last_change":"2024-06-06T03:35:13.853113+0000","last_active":"2024-06-06T03:35:48.985035+0000","last_peered":"2024-06-06T03:35:48.985035+0000","last_clean":"2024-06-06T03:35:48.985035+0000","last_became_active":"2024-06-06T03:35:13.852751+0000","last_became_peered":"2024-06-06T03:35:13.852751+0000","last_unstale":"2024-06-06T03:35:48.985035+0000","last_undegraded":"2024-06-06T03:35:48.985035+0000","last_fullsized":"2024-06-06T03:35:48.985035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:53.536685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.006465+0000","last_change":"2024-06-06T03:35:49.005958+0000","last_active":"2024-06-06T03:35:49.006465+0000","last_peered":"2024-06-06T03:35:49.006465+0000","last_clean":"2024-06-06T03:35:49.006465+0000","last_became_active":"2024-06-06T03:35:48.993266+0000","last_became_peered":"2024-06-06T03:35:48.993266+0000","last_unstale":"2024-06-06T03:35:49.006465+0000","last_undegraded":"2024-06-06T03:35:49.006465+0000","last_fullsized":"2024-06-06T03:35:49.006465+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:47.782824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984158+0000","last_change":"2024-06-06T03:35:13.853614+0000","last_active":"2024-06-06T03:35:48.984158+0000","last_peered":"2024-06-06T03:35:48.984158+0000","last_clean":"2024-06-06T03:35:48.984158+0000","last_became_active":"2024-06-06T03:35:13.852327+0000","last_became_peered":"2024-06-06T03:35:13.852327+0000","last_unstale":"2024-06-06T03:35:48.984158+0000","last_undegraded":"2024-06-06T03:35:48.984158+0000","last_fullsized":"2024-06-06T03:35:48.984158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:02:32.463028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991648+0000","last_change":"2024-06-06T03:35:09.826220+0000","last_active":"2024-06-06T03:35:48.991648+0000","last_peered":"2024-06-06T03:35:48.991648+0000","last_clean":"2024-06-06T03:35:48.991648+0000","last_became_active":"2024-06-06T03:35:09.825887+0000","last_became_peered":"2024-06-06T03:35:09.825887+0000","last_unstale":"2024-06-06T03:35:48.991648+0000","last_undegraded":"2024-06-06T03:35:48.991648+0000","last_fullsized":"2024-06-06T03:35:48.991648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:23:33.704822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987180+0000","last_change":"2024-06-06T03:35:13.852455+0000","last_active":"2024-06-06T03:35:48.987180+0000","last_peered":"2024-06-06T03:35:48.987180+0000","last_clean":"2024-06-06T03:35:48.987180+0000","last_became_active":"2024-06-06T03:35:13.852134+0000","last_became_peered":"2024-06-06T03:35:13.852134+0000","last_unstale":"2024-06-06T03:35:48.987180+0000","last_undegraded":"2024-06-06T03:35:48.987180+0000","last_fullsized":"2024-06-06T03:35:48.987180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:00:16.418479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.986584+0000","last_change":"2024-06-06T03:35:10.277767+0000","last_active":"2024-06-06T03:35:48.986584+0000","last_peered":"2024-06-06T03:35:48.986584+0000","last_clean":"2024-06-06T03:35:48.986584+0000","last_became_active":"2024-06-06T03:35:10.277381+0000","last_became_peered":"2024-06-06T03:35:10.277381+0000","last_unstale":"2024-06-06T03:35:48.986584+0000","last_undegraded":"2024-06-06T03:35:48.986584+0000","last_fullsized":"2024-06-06T03:35:48.986584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:12:13.425284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988580+0000","last_change":"2024-06-06T03:35:13.859594+0000","last_active":"2024-06-06T03:35:48.988580+0000","last_peered":"2024-06-06T03:35:48.988580+0000","last_clean":"2024-06-06T03:35:48.988580+0000","last_became_active":"2024-06-06T03:35:13.859240+0000","last_became_peered":"2024-06-06T03:35:13.859240+0000","last_unstale":"2024-06-06T03:35:48.988580+0000","last_undegraded":"2024-06-06T03:35:48.988580+0000","last_fullsized":"2024-06-06T03:35:48.988580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:39:34.557581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:58.993547+0000","last_change":"2024-06-06T03:35:09.826204+0000","last_active":"2024-06-06T03:35:58.993547+0000","last_peered":"2024-06-06T03:35:58.993547+0000","last_clean":"2024-06-06T03:35:58.993547+0000","last_became_active":"2024-06-06T03:35:09.825815+0000","last_became_peered":"2024-06-06T03:35:09.825815+0000","last_unstale":"2024-06-06T03:35:58.993547+0000","last_undegraded":"2024-06-06T03:35:58.993547+0000","last_fullsized":"2024-06-06T03:35:58.993547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:02:04.169227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992786+0000","last_change":"2024-06-06T03:35:48.992786+0000","last_active":"2024-06-06T03:35:48.992786+0000","last_peered":"2024-06-06T03:35:48.992786+0000","last_clean":"2024-06-06T03:35:48.992786+0000","last_became_active":"2024-06-06T03:35:48.992456+0000","last_became_peered":"2024-06-06T03:35:48.992456+0000","last_unstale":"2024-06-06T03:35:48.992786+0000","last_undegraded":"2024-06-06T03:35:48.992786+0000","last_fullsized":"2024-06-06T03:35:48.992786+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:06:58.563975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.005564+0000","last_change":"2024-06-06T03:35:49.005119+0000","last_active":"2024-06-06T03:35:49.005564+0000","last_peered":"2024-06-06T03:35:49.005564+0000","last_clean":"2024-06-06T03:35:49.005564+0000","last_became_active":"2024-06-06T03:35:49.004788+0000","last_became_peered":"2024-06-06T03:35:49.004788+0000","last_unstale":"2024-06-06T03:35:49.005564+0000","last_undegraded":"2024-06-06T03:35:49.005564+0000","last_fullsized":"2024-06-06T03:35:49.005564+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:20:06.961454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:58.978408+0000","last_change":"2024-06-06T03:35:10.278559+0000","last_active":"2024-06-06T03:35:58.978408+0000","last_peered":"2024-06-06T03:35:58.978408+0000","last_clean":"2024-06-06T03:35:58.978408+0000","last_became_active":"2024-06-06T03:35:10.278104+0000","last_became_peered":"2024-06-06T03:35:10.278104+0000","last_unstale":"2024-06-06T03:35:58.978408+0000","last_undegraded":"2024-06-06T03:35:58.978408+0000","last_fullsized":"2024-06-06T03:35:58.978408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:23:47.131731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990973+0000","last_change":"2024-06-06T03:35:13.862303+0000","last_active":"2024-06-06T03:35:48.990973+0000","last_peered":"2024-06-06T03:35:48.990973+0000","last_clean":"2024-06-06T03:35:48.990973+0000","last_became_active":"2024-06-06T03:35:13.861978+0000","last_became_peered":"2024-06-06T03:35:13.861978+0000","last_unstale":"2024-06-06T03:35:48.990973+0000","last_undegraded":"2024-06-06T03:35:48.990973+0000","last_fullsized":"2024-06-06T03:35:48.990973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:24:00.824646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.004144+0000","last_change":"2024-06-06T03:35:09.825365+0000","last_active":"2024-06-06T03:35:49.004144+0000","last_peered":"2024-06-06T03:35:49.004144+0000","last_clean":"2024-06-06T03:35:49.004144+0000","last_became_active":"2024-06-06T03:35:09.824948+0000","last_became_peered":"2024-06-06T03:35:09.824948+0000","last_unstale":"2024-06-06T03:35:49.004144+0000","last_undegraded":"2024-06-06T03:35:49.004144+0000","last_fullsized":"2024-06-06T03:35:49.004144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:23:48.189292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":358,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985268+0000","last_change":"2024-06-06T03:33:44.124626+0000","last_active":"2024-06-06T03:35:48.985268+0000","last_peered":"2024-06-06T03:35:48.985268+0000","last_clean":"2024-06-06T03:35:48.985268+0000","last_became_active":"2024-06-06T03:33:44.124046+0000","last_became_peered":"2024-06-06T03:33:44.124046+0000","last_unstale":"2024-06-06T03:35:48.985268+0000","last_undegraded":"2024-06-06T03:35:48.985268+0000","last_fullsized":"2024-06-06T03:35:48.985268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:30:14.586719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:30:14.586719+0000","last_clean_scrub_stamp":"2024-06-06T03:30:14.586719+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:17.604180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983301+0000","last_change":"2024-06-06T03:35:13.852044+0000","last_active":"2024-06-06T03:35:48.983301+0000","last_peered":"2024-06-06T03:35:48.983301+0000","last_clean":"2024-06-06T03:35:48.983301+0000","last_became_active":"2024-06-06T03:35:13.851715+0000","last_became_peered":"2024-06-06T03:35:13.851715+0000","last_unstale":"2024-06-06T03:35:48.983301+0000","last_undegraded":"2024-06-06T03:35:48.983301+0000","last_fullsized":"2024-06-06T03:35:48.983301+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:19:23.778367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985449+0000","last_change":"2024-06-06T03:35:09.828463+0000","last_active":"2024-06-06T03:35:48.985449+0000","last_peered":"2024-06-06T03:35:48.985449+0000","last_clean":"2024-06-06T03:35:48.985449+0000","last_became_active":"2024-06-06T03:35:09.828121+0000","last_became_peered":"2024-06-06T03:35:09.828121+0000","last_unstale":"2024-06-06T03:35:48.985449+0000","last_undegraded":"2024-06-06T03:35:48.985449+0000","last_fullsized":"2024-06-06T03:35:48.985449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:54:25.095741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992104+0000","last_change":"2024-06-06T03:35:10.277901+0000","last_active":"2024-06-06T03:35:48.992104+0000","last_peered":"2024-06-06T03:35:48.992104+0000","last_clean":"2024-06-06T03:35:48.992104+0000","last_became_active":"2024-06-06T03:35:10.277447+0000","last_became_peered":"2024-06-06T03:35:10.277447+0000","last_unstale":"2024-06-06T03:35:48.992104+0000","last_undegraded":"2024-06-06T03:35:48.992104+0000","last_fullsized":"2024-06-06T03:35:48.992104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:22:50.550287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983583+0000","last_change":"2024-06-06T03:35:13.864323+0000","last_active":"2024-06-06T03:35:48.983583+0000","last_peered":"2024-06-06T03:35:48.983583+0000","last_clean":"2024-06-06T03:35:48.983583+0000","last_became_active":"2024-06-06T03:35:13.863101+0000","last_became_peered":"2024-06-06T03:35:13.863101+0000","last_unstale":"2024-06-06T03:35:48.983583+0000","last_undegraded":"2024-06-06T03:35:48.983583+0000","last_fullsized":"2024-06-06T03:35:48.983583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:32.399325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'14","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:54.869403+0000","last_change":"2024-06-06T03:35:09.822280+0000","last_active":"2024-06-06T03:35:54.869403+0000","last_peered":"2024-06-06T03:35:54.869403+0000","last_clean":"2024-06-06T03:35:54.869403+0000","last_became_active":"2024-06-06T03:35:09.821922+0000","last_became_peered":"2024-06-06T03:35:09.821922+0000","last_unstale":"2024-06-06T03:35:54.869403+0000","last_undegraded":"2024-06-06T03:35:54.869403+0000","last_fullsized":"2024-06-06T03:35:54.869403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:00:03.328747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991350+0000","last_change":"2024-06-06T03:35:13.853538+0000","last_active":"2024-06-06T03:35:48.991350+0000","last_peered":"2024-06-06T03:35:48.991350+0000","last_clean":"2024-06-06T03:35:48.991350+0000","last_became_active":"2024-06-06T03:35:13.853200+0000","last_became_peered":"2024-06-06T03:35:13.853200+0000","last_unstale":"2024-06-06T03:35:48.991350+0000","last_undegraded":"2024-06-06T03:35:48.991350+0000","last_fullsized":"2024-06-06T03:35:48.991350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:13:06.574272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.004336+0000","last_change":"2024-06-06T03:35:10.761675+0000","last_active":"2024-06-06T03:35:49.004336+0000","last_peered":"2024-06-06T03:35:49.004336+0000","last_clean":"2024-06-06T03:35:49.004336+0000","last_became_active":"2024-06-06T03:35:10.761287+0000","last_became_peered":"2024-06-06T03:35:10.761287+0000","last_unstale":"2024-06-06T03:35:49.004336+0000","last_undegraded":"2024-06-06T03:35:49.004336+0000","last_fullsized":"2024-06-06T03:35:49.004336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:58:18.524273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990632+0000","last_change":"2024-06-06T03:35:13.847680+0000","last_active":"2024-06-06T03:35:48.990632+0000","last_peered":"2024-06-06T03:35:48.990632+0000","last_clean":"2024-06-06T03:35:48.990632+0000","last_became_active":"2024-06-06T03:35:13.847359+0000","last_became_peered":"2024-06-06T03:35:13.847359+0000","last_unstale":"2024-06-06T03:35:48.990632+0000","last_undegraded":"2024-06-06T03:35:48.990632+0000","last_fullsized":"2024-06-06T03:35:48.990632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:29:18.041078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984544+0000","last_change":"2024-06-06T03:35:10.277491+0000","last_active":"2024-06-06T03:35:48.984544+0000","last_peered":"2024-06-06T03:35:48.984544+0000","last_clean":"2024-06-06T03:35:48.984544+0000","last_became_active":"2024-06-06T03:35:10.277135+0000","last_became_peered":"2024-06-06T03:35:10.277135+0000","last_unstale":"2024-06-06T03:35:48.984544+0000","last_undegraded":"2024-06-06T03:35:48.984544+0000","last_fullsized":"2024-06-06T03:35:48.984544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:55.198789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991400+0000","last_change":"2024-06-06T03:35:13.846013+0000","last_active":"2024-06-06T03:35:48.991400+0000","last_peered":"2024-06-06T03:35:48.991400+0000","last_clean":"2024-06-06T03:35:48.991400+0000","last_became_active":"2024-06-06T03:35:13.845650+0000","last_became_peered":"2024-06-06T03:35:13.845650+0000","last_unstale":"2024-06-06T03:35:48.991400+0000","last_undegraded":"2024-06-06T03:35:48.991400+0000","last_fullsized":"2024-06-06T03:35:48.991400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:51.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990583+0000","last_change":"2024-06-06T03:35:09.826380+0000","last_active":"2024-06-06T03:35:48.990583+0000","last_peered":"2024-06-06T03:35:48.990583+0000","last_clean":"2024-06-06T03:35:48.990583+0000","last_became_active":"2024-06-06T03:35:09.825813+0000","last_became_peered":"2024-06-06T03:35:09.825813+0000","last_unstale":"2024-06-06T03:35:48.990583+0000","last_undegraded":"2024-06-06T03:35:48.990583+0000","last_fullsized":"2024-06-06T03:35:48.990583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:24.792398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987048+0000","last_change":"2024-06-06T03:35:13.855051+0000","last_active":"2024-06-06T03:35:48.987048+0000","last_peered":"2024-06-06T03:35:48.987048+0000","last_clean":"2024-06-06T03:35:48.987048+0000","last_became_active":"2024-06-06T03:35:13.853781+0000","last_became_peered":"2024-06-06T03:35:13.853781+0000","last_unstale":"2024-06-06T03:35:48.987048+0000","last_undegraded":"2024-06-06T03:35:48.987048+0000","last_fullsized":"2024-06-06T03:35:48.987048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:17:56.174924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.009988+0000","last_change":"2024-06-06T03:35:49.009583+0000","last_active":"2024-06-06T03:35:49.009988+0000","last_peered":"2024-06-06T03:35:49.009988+0000","last_clean":"2024-06-06T03:35:49.009988+0000","last_became_active":"2024-06-06T03:35:48.994275+0000","last_became_peered":"2024-06-06T03:35:48.994275+0000","last_unstale":"2024-06-06T03:35:49.009988+0000","last_undegraded":"2024-06-06T03:35:49.009988+0000","last_fullsized":"2024-06-06T03:35:49.009988+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:24:54.985854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991216+0000","last_change":"2024-06-06T03:35:13.855503+0000","last_active":"2024-06-06T03:35:48.991216+0000","last_peered":"2024-06-06T03:35:48.991216+0000","last_clean":"2024-06-06T03:35:48.991216+0000","last_became_active":"2024-06-06T03:35:13.855069+0000","last_became_peered":"2024-06-06T03:35:13.855069+0000","last_unstale":"2024-06-06T03:35:48.991216+0000","last_undegraded":"2024-06-06T03:35:48.991216+0000","last_fullsized":"2024-06-06T03:35:48.991216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:43.778258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991692+0000","last_change":"2024-06-06T03:35:09.827116+0000","last_active":"2024-06-06T03:35:48.991692+0000","last_peered":"2024-06-06T03:35:48.991692+0000","last_clean":"2024-06-06T03:35:48.991692+0000","last_became_active":"2024-06-06T03:35:09.826756+0000","last_became_peered":"2024-06-06T03:35:09.826756+0000","last_unstale":"2024-06-06T03:35:48.991692+0000","last_undegraded":"2024-06-06T03:35:48.991692+0000","last_fullsized":"2024-06-06T03:35:48.991692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:31.501411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985532+0000","last_change":"2024-06-06T03:35:13.861189+0000","last_active":"2024-06-06T03:35:48.985532+0000","last_peered":"2024-06-06T03:35:48.985532+0000","last_clean":"2024-06-06T03:35:48.985532+0000","last_became_active":"2024-06-06T03:35:13.860751+0000","last_became_peered":"2024-06-06T03:35:13.860751+0000","last_unstale":"2024-06-06T03:35:48.985532+0000","last_undegraded":"2024-06-06T03:35:48.985532+0000","last_fullsized":"2024-06-06T03:35:48.985532+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:16:34.036420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:36:01.984086+0000","last_change":"2024-06-06T03:35:10.761606+0000","last_active":"2024-06-06T03:36:01.984086+0000","last_peered":"2024-06-06T03:36:01.984086+0000","last_clean":"2024-06-06T03:36:01.984086+0000","last_became_active":"2024-06-06T03:35:10.761104+0000","last_became_peered":"2024-06-06T03:35:10.761104+0000","last_unstale":"2024-06-06T03:36:01.984086+0000","last_undegraded":"2024-06-06T03:36:01.984086+0000","last_fullsized":"2024-06-06T03:36:01.984086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:51:39.867088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985751+0000","last_change":"2024-06-06T03:35:13.856047+0000","last_active":"2024-06-06T03:35:48.985751+0000","last_peered":"2024-06-06T03:35:48.985751+0000","last_clean":"2024-06-06T03:35:48.985751+0000","last_became_active":"2024-06-06T03:35:13.855718+0000","last_became_peered":"2024-06-06T03:35:13.855718+0000","last_unstale":"2024-06-06T03:35:48.985751+0000","last_undegraded":"2024-06-06T03:35:48.985751+0000","last_fullsized":"2024-06-06T03:35:48.985751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:18:40.926872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984381+0000","last_change":"2024-06-06T03:35:09.827283+0000","last_active":"2024-06-06T03:35:48.984381+0000","last_peered":"2024-06-06T03:35:48.984381+0000","last_clean":"2024-06-06T03:35:48.984381+0000","last_became_active":"2024-06-06T03:35:09.826595+0000","last_became_peered":"2024-06-06T03:35:09.826595+0000","last_unstale":"2024-06-06T03:35:48.984381+0000","last_undegraded":"2024-06-06T03:35:48.984381+0000","last_fullsized":"2024-06-06T03:35:48.984381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:34:16.398847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985446+0000","last_change":"2024-06-06T03:35:13.851900+0000","last_active":"2024-06-06T03:35:48.985446+0000","last_peered":"2024-06-06T03:35:48.985446+0000","last_clean":"2024-06-06T03:35:48.985446+0000","last_became_active":"2024-06-06T03:35:13.851477+0000","last_became_peered":"2024-06-06T03:35:13.851477+0000","last_unstale":"2024-06-06T03:35:48.985446+0000","last_undegraded":"2024-06-06T03:35:48.985446+0000","last_fullsized":"2024-06-06T03:35:48.985446+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:31:54.001226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983666+0000","last_change":"2024-06-06T03:35:09.824675+0000","last_active":"2024-06-06T03:35:48.983666+0000","last_peered":"2024-06-06T03:35:48.983666+0000","last_clean":"2024-06-06T03:35:48.983666+0000","last_became_active":"2024-06-06T03:35:09.824347+0000","last_became_peered":"2024-06-06T03:35:09.824347+0000","last_unstale":"2024-06-06T03:35:48.983666+0000","last_undegraded":"2024-06-06T03:35:48.983666+0000","last_fullsized":"2024-06-06T03:35:48.983666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:01:43.080552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985522+0000","last_change":"2024-06-06T03:35:13.854706+0000","last_active":"2024-06-06T03:35:48.985522+0000","last_peered":"2024-06-06T03:35:48.985522+0000","last_clean":"2024-06-06T03:35:48.985522+0000","last_became_active":"2024-06-06T03:35:13.854392+0000","last_became_peered":"2024-06-06T03:35:13.854392+0000","last_unstale":"2024-06-06T03:35:48.985522+0000","last_undegraded":"2024-06-06T03:35:48.985522+0000","last_fullsized":"2024-06-06T03:35:48.985522+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:58:10.102367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987223+0000","last_change":"2024-06-06T03:35:13.853594+0000","last_active":"2024-06-06T03:35:48.987223+0000","last_peered":"2024-06-06T03:35:48.987223+0000","last_clean":"2024-06-06T03:35:48.987223+0000","last_became_active":"2024-06-06T03:35:13.852962+0000","last_became_peered":"2024-06-06T03:35:13.852962+0000","last_unstale":"2024-06-06T03:35:48.987223+0000","last_undegraded":"2024-06-06T03:35:48.987223+0000","last_fullsized":"2024-06-06T03:35:48.987223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:02:44.233868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991198+0000","last_change":"2024-06-06T03:35:13.848790+0000","last_active":"2024-06-06T03:35:48.991198+0000","last_peered":"2024-06-06T03:35:48.991198+0000","last_clean":"2024-06-06T03:35:48.991198+0000","last_became_active":"2024-06-06T03:35:13.848442+0000","last_became_peered":"2024-06-06T03:35:13.848442+0000","last_unstale":"2024-06-06T03:35:48.991198+0000","last_undegraded":"2024-06-06T03:35:48.991198+0000","last_fullsized":"2024-06-06T03:35:48.991198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:51.382210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991331+0000","last_change":"2024-06-06T03:35:13.866999+0000","last_active":"2024-06-06T03:35:48.991331+0000","last_peered":"2024-06-06T03:35:48.991331+0000","last_clean":"2024-06-06T03:35:48.991331+0000","last_became_active":"2024-06-06T03:35:13.866664+0000","last_became_peered":"2024-06-06T03:35:13.866664+0000","last_unstale":"2024-06-06T03:35:48.991331+0000","last_undegraded":"2024-06-06T03:35:48.991331+0000","last_fullsized":"2024-06-06T03:35:48.991331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:31:14.166122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990482+0000","last_change":"2024-06-06T03:35:13.849062+0000","last_active":"2024-06-06T03:35:48.990482+0000","last_peered":"2024-06-06T03:35:48.990482+0000","last_clean":"2024-06-06T03:35:48.990482+0000","last_became_active":"2024-06-06T03:35:13.848750+0000","last_became_peered":"2024-06-06T03:35:13.848750+0000","last_unstale":"2024-06-06T03:35:48.990482+0000","last_undegraded":"2024-06-06T03:35:48.990482+0000","last_fullsized":"2024-06-06T03:35:48.990482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:55:58.084908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983429+0000","last_change":"2024-06-06T03:35:13.857601+0000","last_active":"2024-06-06T03:35:48.983429+0000","last_peered":"2024-06-06T03:35:48.983429+0000","last_clean":"2024-06-06T03:35:48.983429+0000","last_became_active":"2024-06-06T03:35:13.857270+0000","last_became_peered":"2024-06-06T03:35:13.857270+0000","last_unstale":"2024-06-06T03:35:48.983429+0000","last_undegraded":"2024-06-06T03:35:48.983429+0000","last_fullsized":"2024-06-06T03:35:48.983429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:56:29.970210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987336+0000","last_change":"2024-06-06T03:35:13.846699+0000","last_active":"2024-06-06T03:35:48.987336+0000","last_peered":"2024-06-06T03:35:48.987336+0000","last_clean":"2024-06-06T03:35:48.987336+0000","last_became_active":"2024-06-06T03:35:13.846355+0000","last_became_peered":"2024-06-06T03:35:13.846355+0000","last_unstale":"2024-06-06T03:35:48.987336+0000","last_undegraded":"2024-06-06T03:35:48.987336+0000","last_fullsized":"2024-06-06T03:35:48.987336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:50:22.537339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991821+0000","last_change":"2024-06-06T03:35:13.854013+0000","last_active":"2024-06-06T03:35:48.991821+0000","last_peered":"2024-06-06T03:35:48.991821+0000","last_clean":"2024-06-06T03:35:48.991821+0000","last_became_active":"2024-06-06T03:35:13.853739+0000","last_became_peered":"2024-06-06T03:35:13.853739+0000","last_unstale":"2024-06-06T03:35:48.991821+0000","last_undegraded":"2024-06-06T03:35:48.991821+0000","last_fullsized":"2024-06-06T03:35:48.991821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:27:12.330943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991256+0000","last_change":"2024-06-06T03:35:13.854251+0000","last_active":"2024-06-06T03:35:48.991256+0000","last_peered":"2024-06-06T03:35:48.991256+0000","last_clean":"2024-06-06T03:35:48.991256+0000","last_became_active":"2024-06-06T03:35:13.853915+0000","last_became_peered":"2024-06-06T03:35:13.853915+0000","last_unstale":"2024-06-06T03:35:48.991256+0000","last_undegraded":"2024-06-06T03:35:48.991256+0000","last_fullsized":"2024-06-06T03:35:48.991256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:08.813316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984146+0000","last_change":"2024-06-06T03:35:13.849998+0000","last_active":"2024-06-06T03:35:48.984146+0000","last_peered":"2024-06-06T03:35:48.984146+0000","last_clean":"2024-06-06T03:35:48.984146+0000","last_became_active":"2024-06-06T03:35:13.848789+0000","last_became_peered":"2024-06-06T03:35:13.848789+0000","last_unstale":"2024-06-06T03:35:48.984146+0000","last_undegraded":"2024-06-06T03:35:48.984146+0000","last_fullsized":"2024-06-06T03:35:48.984146+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:53:39.043356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990461+0000","last_change":"2024-06-06T03:35:13.858020+0000","last_active":"2024-06-06T03:35:48.990461+0000","last_peered":"2024-06-06T03:35:48.990461+0000","last_clean":"2024-06-06T03:35:48.990461+0000","last_became_active":"2024-06-06T03:35:13.857723+0000","last_became_peered":"2024-06-06T03:35:13.857723+0000","last_unstale":"2024-06-06T03:35:48.990461+0000","last_undegraded":"2024-06-06T03:35:48.990461+0000","last_fullsized":"2024-06-06T03:35:48.990461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:08:08.651213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984730+0000","last_change":"2024-06-06T03:35:13.867546+0000","last_active":"2024-06-06T03:35:48.984730+0000","last_peered":"2024-06-06T03:35:48.984730+0000","last_clean":"2024-06-06T03:35:48.984730+0000","last_became_active":"2024-06-06T03:35:13.867182+0000","last_became_peered":"2024-06-06T03:35:13.867182+0000","last_unstale":"2024-06-06T03:35:48.984730+0000","last_undegraded":"2024-06-06T03:35:48.984730+0000","last_fullsized":"2024-06-06T03:35:48.984730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:25:55.831943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983770+0000","last_change":"2024-06-06T03:35:13.849080+0000","last_active":"2024-06-06T03:35:48.983770+0000","last_peered":"2024-06-06T03:35:48.983770+0000","last_clean":"2024-06-06T03:35:48.983770+0000","last_became_active":"2024-06-06T03:35:13.848476+0000","last_became_peered":"2024-06-06T03:35:13.848476+0000","last_unstale":"2024-06-06T03:35:48.983770+0000","last_undegraded":"2024-06-06T03:35:48.983770+0000","last_fullsized":"2024-06-06T03:35:48.983770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:48:37.306528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987065+0000","last_change":"2024-06-06T03:35:13.848442+0000","last_active":"2024-06-06T03:35:48.987065+0000","last_peered":"2024-06-06T03:35:48.987065+0000","last_clean":"2024-06-06T03:35:48.987065+0000","last_became_active":"2024-06-06T03:35:13.848074+0000","last_became_peered":"2024-06-06T03:35:13.848074+0000","last_unstale":"2024-06-06T03:35:48.987065+0000","last_undegraded":"2024-06-06T03:35:48.987065+0000","last_fullsized":"2024-06-06T03:35:48.987065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:07:00.197671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280416,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65279,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":306,"num_read_kb":223,"num_write":242,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":195837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383706,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":480334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645343,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1129328,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808867,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070505,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":480418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1073437,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495667,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":480259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":539249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":481208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":539469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":479694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":479730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":1072725,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34027,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34027,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34027,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":59813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T03:36:06.324 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-06T03:36:06.324 INFO:tasks.cephadm:Setting up client nodes... 2024-06-06T03:36:06.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-06T03:36:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:07.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:07 smithi131 ceph-mon[36687]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:36:07.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:07 smithi131 ceph-mon[36687]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-06-06T03:36:07.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:07 smithi040 ceph-mon[32220]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:36:07.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:07 smithi040 ceph-mon[32220]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-06-06T03:36:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:07 smithi104 ceph-mon[36569]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:36:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:07 smithi104 ceph-mon[36569]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-06-06T03:36:08.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.387+0000 7fb99899a700 1 -- 172.21.15.40:0/2322266124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9941013f0 msgr2=0x7fb994101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:08.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.387+0000 7fb99899a700 1 --2- 172.21.15.40:0/2322266124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9941013f0 0x7fb994101810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb9840099c0 tx=0x7fb984009cd0 comp rx=0 tx=0).stop 2024-06-06T03:36:08.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.387+0000 7fb99899a700 1 -- 172.21.15.40:0/2322266124 shutdown_connections 2024-06-06T03:36:08.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.387+0000 7fb99899a700 1 --2- 172.21.15.40:0/2322266124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb994102f90 0x7fb994109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:08.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.387+0000 7fb99899a700 1 --2- 172.21.15.40:0/2322266124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9941013f0 0x7fb994101810 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:08.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.387+0000 7fb99899a700 1 --2- 172.21.15.40:0/2322266124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9941025f0 0x7fb994102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:08.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.387+0000 7fb99899a700 1 -- 172.21.15.40:0/2322266124 >> 172.21.15.40:0/2322266124 conn(0x7fb9940fc990 msgr2=0x7fb9940fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:08.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.387+0000 7fb99899a700 1 -- 172.21.15.40:0/2322266124 shutdown_connections 2024-06-06T03:36:08.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.387+0000 7fb99899a700 1 -- 172.21.15.40:0/2322266124 wait complete. 2024-06-06T03:36:08.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.388+0000 7fb99899a700 1 Processor -- start 2024-06-06T03:36:08.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.388+0000 7fb99899a700 1 -- start start 2024-06-06T03:36:08.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.388+0000 7fb99899a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9941013f0 0x7fb994198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:08.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.388+0000 7fb99899a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9941025f0 0x7fb994198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:08.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.388+0000 7fb99899a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb994102f90 0x7fb99419fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:08.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.388+0000 7fb99899a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb99410d0b0 con 0x7fb9941013f0 2024-06-06T03:36:08.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.388+0000 7fb99899a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb99410cf30 con 0x7fb994102f90 2024-06-06T03:36:08.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.388+0000 7fb99899a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb99410d230 con 0x7fb9941025f0 2024-06-06T03:36:08.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb99259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9941013f0 0x7fb994198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb99259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9941013f0 0x7fb994198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57780/0 (socket says 172.21.15.40:57780) 2024-06-06T03:36:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb99259c700 1 -- 172.21.15.40:0/1247290759 learned_addr learned my addr 172.21.15.40:0/1247290759 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb992d9d700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb994102f90 0x7fb99419fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb991d9b700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9941025f0 0x7fb994198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb99259c700 1 -- 172.21.15.40:0/1247290759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9941025f0 msgr2=0x7fb994198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb99259c700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9941025f0 0x7fb994198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb99259c700 1 -- 172.21.15.40:0/1247290759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb994102f90 msgr2=0x7fb99419fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb99259c700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb994102f90 0x7fb99419fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb99259c700 1 -- 172.21.15.40:0/1247290759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9941a04c0 con 0x7fb9941013f0 2024-06-06T03:36:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb991d9b700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9941025f0 0x7fb994198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:36:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb992d9d700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb994102f90 0x7fb99419fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:36:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb99259c700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9941013f0 0x7fb994198860 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb98400bff0 tx=0x7fb984003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb9837fe700 1 -- 172.21.15.40:0/1247290759 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984027070 con 0x7fb9941013f0 2024-06-06T03:36:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb9837fe700 1 -- 172.21.15.40:0/1247290759 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb984022470 con 0x7fb9941013f0 2024-06-06T03:36:08.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb99899a700 1 -- 172.21.15.40:0/1247290759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9941a0750 con 0x7fb9941013f0 2024-06-06T03:36:08.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb9837fe700 1 -- 172.21.15.40:0/1247290759 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98400f6c0 con 0x7fb9941013f0 2024-06-06T03:36:08.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.389+0000 7fb99899a700 1 -- 172.21.15.40:0/1247290759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9941a0cb0 con 0x7fb9941013f0 2024-06-06T03:36:08.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.390+0000 7fb99899a700 1 -- 172.21.15.40:0/1247290759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb994050a00 con 0x7fb9941013f0 2024-06-06T03:36:08.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.392+0000 7fb9837fe700 1 -- 172.21.15.40:0/1247290759 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb98402a070 con 0x7fb9941013f0 2024-06-06T03:36:08.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.392+0000 7fb9837fe700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fb97c0a2d60 0x7fb97c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:08.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.392+0000 7fb991d9b700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fb97c0a2d60 0x7fb97c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:08.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.392+0000 7fb9837fe700 1 -- 172.21.15.40:0/1247290759 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fb9840d0090 con 0x7fb9941013f0 2024-06-06T03:36:08.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.393+0000 7fb991d9b700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fb97c0a2d60 0x7fb97c0a5220 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb988007900 tx=0x7fb988008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:08.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.397+0000 7fb9837fe700 1 -- 172.21.15.40:0/1247290759 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb98405e850 con 0x7fb9941013f0 2024-06-06T03:36:08.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.688+0000 7fb99899a700 1 -- 172.21.15.40:0/1247290759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fb9941a1080 con 0x7fb9941013f0 2024-06-06T03:36:08.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.690+0000 7fb9837fe700 1 -- 172.21.15.40:0/1247290759 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fb984013070 con 0x7fb9941013f0 2024-06-06T03:36:08.689 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-06-06T03:36:08.689 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCoLmFmrAcRKRAAT0uE6c6NyU3IhCA46EbpNQ== 2024-06-06T03:36:08.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.693+0000 7fb99899a700 1 -- 172.21.15.40:0/1247290759 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fb97c0a2d60 msgr2=0x7fb97c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.693+0000 7fb99899a700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fb97c0a2d60 0x7fb97c0a5220 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb988007900 tx=0x7fb988008040 comp rx=0 tx=0).stop 2024-06-06T03:36:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.693+0000 7fb99899a700 1 -- 172.21.15.40:0/1247290759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9941013f0 msgr2=0x7fb994198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.693+0000 7fb99899a700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9941013f0 0x7fb994198860 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb98400bff0 tx=0x7fb984003680 comp rx=0 tx=0).stop 2024-06-06T03:36:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.693+0000 7fb99899a700 1 -- 172.21.15.40:0/1247290759 shutdown_connections 2024-06-06T03:36:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.693+0000 7fb99899a700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fb97c0a2d60 0x7fb97c0a5220 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.693+0000 7fb99899a700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb994102f90 0x7fb99419fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.693+0000 7fb99899a700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9941013f0 0x7fb994198860 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.693+0000 7fb99899a700 1 --2- 172.21.15.40:0/1247290759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9941025f0 0x7fb994198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:08.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.693+0000 7fb99899a700 1 -- 172.21.15.40:0/1247290759 >> 172.21.15.40:0/1247290759 conn(0x7fb9940fc990 msgr2=0x7fb9940fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:08.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.694+0000 7fb99899a700 1 -- 172.21.15.40:0/1247290759 shutdown_connections 2024-06-06T03:36:08.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:08.694+0000 7fb99899a700 1 -- 172.21.15.40:0/1247290759 wait complete. 2024-06-06T03:36:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:09 smithi131 ceph-mon[36687]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-06-06T03:36:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:09 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1247290759' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T03:36:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:09 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1247290759' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T03:36:09.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:09 smithi040 ceph-mon[32220]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-06-06T03:36:09.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:09 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1247290759' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T03:36:09.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:09 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1247290759' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T03:36:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:09 smithi104 ceph-mon[36569]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-06-06T03:36:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:09 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1247290759' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T03:36:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:09 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1247290759' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T03:36:09.359 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:36:09.359 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-06T03:36:09.359 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-06T03:36:09.400 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-06T03:36:09.400 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-06T03:36:09.401 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-06T03:36:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:11.122 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:11 smithi131 ceph-mon[36687]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-06-06T03:36:11.122 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:11 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:36:11.167 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:11 smithi040 ceph-mon[32220]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-06-06T03:36:11.167 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:11 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:36:11.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:11 smithi104 ceph-mon[36569]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-06-06T03:36:11.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:11 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:36:11.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.414+0000 7fba7faf0700 1 -- 172.21.15.40:0/2886347143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba78108530 msgr2=0x7fba78108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:11.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.414+0000 7fba7faf0700 1 --2- 172.21.15.40:0/2886347143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba78108530 0x7fba78108910 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fba74009990 tx=0x7fba74009ca0 comp rx=0 tx=0).stop 2024-06-06T03:36:11.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.414+0000 7fba7faf0700 1 -- 172.21.15.40:0/2886347143 shutdown_connections 2024-06-06T03:36:11.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.414+0000 7fba7faf0700 1 --2- 172.21.15.40:0/2886347143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba7806b5c0 0x7fba7810f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:11.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.414+0000 7fba7faf0700 1 --2- 172.21.15.40:0/2886347143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba78108530 0x7fba78108910 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:11.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.414+0000 7fba7faf0700 1 --2- 172.21.15.40:0/2886347143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba7806ac00 0x7fba7806b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:11.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.414+0000 7fba7faf0700 1 -- 172.21.15.40:0/2886347143 >> 172.21.15.40:0/2886347143 conn(0x7fba78077400 msgr2=0x7fba78077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:11.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.414+0000 7fba7faf0700 1 -- 172.21.15.40:0/2886347143 shutdown_connections 2024-06-06T03:36:11.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.414+0000 7fba7faf0700 1 -- 172.21.15.40:0/2886347143 wait complete. 2024-06-06T03:36:11.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.414+0000 7fba7faf0700 1 Processor -- start 2024-06-06T03:36:11.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.414+0000 7fba7faf0700 1 -- start start 2024-06-06T03:36:11.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7faf0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba7806ac00 0x7fba7819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:11.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7faf0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba7806b5c0 0x7fba7819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:11.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7faf0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba78108530 0x7fba78198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:11.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7faf0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fba78112760 con 0x7fba78108530 2024-06-06T03:36:11.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7faf0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fba781125e0 con 0x7fba7806b5c0 2024-06-06T03:36:11.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7faf0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fba781128e0 con 0x7fba7806ac00 2024-06-06T03:36:11.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7e08d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba78108530 0x7fba78198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:11.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7e08d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba78108530 0x7fba78198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54430/0 (socket says 172.21.15.40:54430) 2024-06-06T03:36:11.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7e08d700 1 -- 172.21.15.40:0/3621543661 learned_addr learned my addr 172.21.15.40:0/3621543661 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:11.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7d88c700 1 --2- 172.21.15.40:0/3621543661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba7806ac00 0x7fba7819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:11.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7d08b700 1 --2- 172.21.15.40:0/3621543661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba7806b5c0 0x7fba7819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:11.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7e08d700 1 -- 172.21.15.40:0/3621543661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba7806ac00 msgr2=0x7fba7819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:11.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7e08d700 1 --2- 172.21.15.40:0/3621543661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba7806ac00 0x7fba7819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:11.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7e08d700 1 -- 172.21.15.40:0/3621543661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba7806b5c0 msgr2=0x7fba7819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:11.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7e08d700 1 --2- 172.21.15.40:0/3621543661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba7806b5c0 0x7fba7819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:11.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7e08d700 1 -- 172.21.15.40:0/3621543661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba781991e0 con 0x7fba78108530 2024-06-06T03:36:11.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.415+0000 7fba7e08d700 1 --2- 172.21.15.40:0/3621543661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba78108530 0x7fba78198a80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fba7000ef30 tx=0x7fba7000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:11.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.416+0000 7fba6effd700 1 -- 172.21.15.40:0/3621543661 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba700092e0 con 0x7fba78108530 2024-06-06T03:36:11.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.416+0000 7fba7faf0700 1 -- 172.21.15.40:0/3621543661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba78199470 con 0x7fba78108530 2024-06-06T03:36:11.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.416+0000 7fba6effd700 1 -- 172.21.15.40:0/3621543661 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba70004bb0 con 0x7fba78108530 2024-06-06T03:36:11.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.416+0000 7fba6effd700 1 -- 172.21.15.40:0/3621543661 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba70004280 con 0x7fba78108530 2024-06-06T03:36:11.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.416+0000 7fba7faf0700 1 -- 172.21.15.40:0/3621543661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba781013c0 con 0x7fba78108530 2024-06-06T03:36:11.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.417+0000 7fba7faf0700 1 -- 172.21.15.40:0/3621543661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7810aa30 con 0x7fba78108530 2024-06-06T03:36:11.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.418+0000 7fba6effd700 1 -- 172.21.15.40:0/3621543661 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fba7001b030 con 0x7fba78108530 2024-06-06T03:36:11.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.419+0000 7fba6effd700 1 --2- 172.21.15.40:0/3621543661 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fba640a2cb0 0x7fba640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:11.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.419+0000 7fba7d88c700 1 --2- 172.21.15.40:0/3621543661 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fba640a2cb0 0x7fba640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:11.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.419+0000 7fba6effd700 1 -- 172.21.15.40:0/3621543661 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fba700ceac0 con 0x7fba78108530 2024-06-06T03:36:11.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.420+0000 7fba7d88c700 1 --2- 172.21.15.40:0/3621543661 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fba640a2cb0 0x7fba640a5170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fba7400b2e0 tx=0x7fba7401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:11.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.424+0000 7fba6effd700 1 -- 172.21.15.40:0/3621543661 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fba70022020 con 0x7fba78108530 2024-06-06T03:36:11.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.708+0000 7fba7faf0700 1 -- 172.21.15.40:0/3621543661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fba78199f60 con 0x7fba78108530 2024-06-06T03:36:11.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.713+0000 7fba6effd700 1 -- 172.21.15.40:0/3621543661 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263197 (secure 0 0 0) 0x7fba70010040 con 0x7fba78108530 2024-06-06T03:36:11.715 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:36:11.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.716+0000 7fba7faf0700 1 -- 172.21.15.40:0/3621543661 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fba640a2cb0 msgr2=0x7fba640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:11.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.716+0000 7fba7faf0700 1 --2- 172.21.15.40:0/3621543661 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fba640a2cb0 0x7fba640a5170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fba7400b2e0 tx=0x7fba7401a040 comp rx=0 tx=0).stop 2024-06-06T03:36:11.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.716+0000 7fba7faf0700 1 -- 172.21.15.40:0/3621543661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba78108530 msgr2=0x7fba78198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:11.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.716+0000 7fba7faf0700 1 --2- 172.21.15.40:0/3621543661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba78108530 0x7fba78198a80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fba7000ef30 tx=0x7fba7000c3a0 comp rx=0 tx=0).stop 2024-06-06T03:36:11.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.716+0000 7fba7faf0700 1 -- 172.21.15.40:0/3621543661 shutdown_connections 2024-06-06T03:36:11.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.716+0000 7fba7faf0700 1 --2- 172.21.15.40:0/3621543661 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fba640a2cb0 0x7fba640a5170 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.716+0000 7fba7faf0700 1 --2- 172.21.15.40:0/3621543661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba7806ac00 0x7fba7819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.716+0000 7fba7faf0700 1 --2- 172.21.15.40:0/3621543661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba78108530 0x7fba78198a80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.716+0000 7fba7faf0700 1 --2- 172.21.15.40:0/3621543661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba7806b5c0 0x7fba7819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.716+0000 7fba7faf0700 1 -- 172.21.15.40:0/3621543661 >> 172.21.15.40:0/3621543661 conn(0x7fba78077400 msgr2=0x7fba78100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.716+0000 7fba7faf0700 1 -- 172.21.15.40:0/3621543661 shutdown_connections 2024-06-06T03:36:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:11.717+0000 7fba7faf0700 1 -- 172.21.15.40:0/3621543661 wait complete. 2024-06-06T03:36:12.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:12 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3621543661' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T03:36:12.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:12 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:12.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:12 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:12.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:12 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:12.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:12 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:12.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:12 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3621543661' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T03:36:12.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:12 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:12.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:12 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:12.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:12 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:12.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:12 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:12.425 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":869712339},{"type":"v1","addr":"172.21.15.40:6801","nonce":869712339}]},"active_addr":"172.21.15.40:6801/869712339","active_change":"2024-06-06T03:27:47.578948+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.40:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":1915158516}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":889893607}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":2237379034}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":1294024108}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":3209472289}]}]} 2024-06-06T03:36:12.432 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-06T03:36:12.432 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-06T03:36:12.433 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T03:36:12.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:12 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3621543661' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T03:36:12.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:12 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:12.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:12 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:12.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:12 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:12.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:12 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:12.803 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:13.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:13 smithi040 ceph-mon[32220]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-06-06T03:36:13.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:13 smithi104 ceph-mon[36569]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-06-06T03:36:13.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:13 smithi131 ceph-mon[36687]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-06-06T03:36:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.511+0000 7fd0e61c6700 1 -- 172.21.15.40:0/3930090765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0e0107840 msgr2=0x7fd0e010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.511+0000 7fd0e61c6700 1 --2- 172.21.15.40:0/3930090765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0e0107840 0x7fd0e010f3c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800b0d0 tx=0x7fd0d800b3e0 comp rx=0 tx=0).stop 2024-06-06T03:36:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.511+0000 7fd0e61c6700 1 -- 172.21.15.40:0/3930090765 shutdown_connections 2024-06-06T03:36:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.511+0000 7fd0e61c6700 1 --2- 172.21.15.40:0/3930090765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0e0107840 0x7fd0e010f3c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.511+0000 7fd0e61c6700 1 --2- 172.21.15.40:0/3930090765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0e0102740 0x7fd0e0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.511+0000 7fd0e61c6700 1 --2- 172.21.15.40:0/3930090765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0e01030f0 0x7fd0e0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.511+0000 7fd0e61c6700 1 -- 172.21.15.40:0/3930090765 >> 172.21.15.40:0/3930090765 conn(0x7fd0e00795f0 msgr2=0x7fd0e0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.511+0000 7fd0e61c6700 1 -- 172.21.15.40:0/3930090765 shutdown_connections 2024-06-06T03:36:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.511+0000 7fd0e61c6700 1 -- 172.21.15.40:0/3930090765 wait complete. 2024-06-06T03:36:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.512+0000 7fd0e61c6700 1 Processor -- start 2024-06-06T03:36:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.512+0000 7fd0e61c6700 1 -- start start 2024-06-06T03:36:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.512+0000 7fd0e61c6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0e0102740 0x7fd0e019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.512+0000 7fd0e61c6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0e01030f0 0x7fd0e019ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.512+0000 7fd0e61c6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0e0107840 0x7fd0e0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.512+0000 7fd0e61c6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e0112760 con 0x7fd0e0102740 2024-06-06T03:36:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.512+0000 7fd0e61c6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e01125e0 con 0x7fd0e01030f0 2024-06-06T03:36:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.512+0000 7fd0e61c6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e01128e0 con 0x7fd0e0107840 2024-06-06T03:36:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.512+0000 7fd0df7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0e0102740 0x7fd0e019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.512+0000 7fd0df7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0e0102740 0x7fd0e019e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54452/0 (socket says 172.21.15.40:54452) 2024-06-06T03:36:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.512+0000 7fd0df7fe700 1 -- 172.21.15.40:0/1110377473 learned_addr learned my addr 172.21.15.40:0/1110377473 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.512+0000 7fd0d7fff700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0e01030f0 0x7fd0e019ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.512+0000 7fd0dffff700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0e0107840 0x7fd0e0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.512+0000 7fd0df7fe700 1 -- 172.21.15.40:0/1110377473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0e0107840 msgr2=0x7fd0e0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.513+0000 7fd0df7fe700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0e0107840 0x7fd0e0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.513+0000 7fd0df7fe700 1 -- 172.21.15.40:0/1110377473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0e01030f0 msgr2=0x7fd0e019ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.513+0000 7fd0df7fe700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0e01030f0 0x7fd0e019ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.513+0000 7fd0df7fe700 1 -- 172.21.15.40:0/1110377473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e01990e0 con 0x7fd0e0102740 2024-06-06T03:36:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.513+0000 7fd0dffff700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0e0107840 0x7fd0e0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:36:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.513+0000 7fd0df7fe700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0e0102740 0x7fd0e019e770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000d950 tx=0x7fd0d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.513+0000 7fd0dd7fa700 1 -- 172.21.15.40:0/1110377473 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0016070 con 0x7fd0e0102740 2024-06-06T03:36:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.513+0000 7fd0dd7fa700 1 -- 172.21.15.40:0/1110377473 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0d0009e20 con 0x7fd0e0102740 2024-06-06T03:36:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.513+0000 7fd0e61c6700 1 -- 172.21.15.40:0/1110377473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e01993d0 con 0x7fd0e0102740 2024-06-06T03:36:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.513+0000 7fd0dd7fa700 1 -- 172.21.15.40:0/1110377473 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000f620 con 0x7fd0e0102740 2024-06-06T03:36:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.513+0000 7fd0e61c6700 1 -- 172.21.15.40:0/1110377473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e01a5640 con 0x7fd0e0102740 2024-06-06T03:36:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.514+0000 7fd0e61c6700 1 -- 172.21.15.40:0/1110377473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e0069180 con 0x7fd0e0102740 2024-06-06T03:36:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.516+0000 7fd0dd7fa700 1 -- 172.21.15.40:0/1110377473 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd0d0019070 con 0x7fd0e0102740 2024-06-06T03:36:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.516+0000 7fd0dd7fa700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd0cc0a2d30 0x7fd0cc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.517+0000 7fd0d7fff700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd0cc0a2d30 0x7fd0cc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.517+0000 7fd0dd7fa700 1 -- 172.21.15.40:0/1110377473 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fd0d00cf0b0 con 0x7fd0e0102740 2024-06-06T03:36:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.517+0000 7fd0d7fff700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd0cc0a2d30 0x7fd0cc0a51f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd0e007a050 tx=0x7fd0c800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:14.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.520+0000 7fd0dd7fa700 1 -- 172.21.15.40:0/1110377473 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd0d009bed0 con 0x7fd0e0102740 2024-06-06T03:36:14.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.739+0000 7fd0e61c6700 1 -- 172.21.15.40:0/1110377473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0e002dca0 con 0x7fd0e0102740 2024-06-06T03:36:14.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.740+0000 7fd0dd7fa700 1 -- 172.21.15.40:0/1110377473 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22534 (secure 0 0 0) 0x7fd0d009b7d0 con 0x7fd0e0102740 2024-06-06T03:36:14.739 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:36:14.739 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"fsid":"77c25d68-23b4-11ef-bc9b-c7b262605968","created":"2024-06-06T03:26:53.165489+0000","modified":"2024-06-06T03:35:48.980461+0000","last_up_change":"2024-06-06T03:34:07.114448+0000","last_in_change":"2024-06-06T03:33:46.152948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T03:30:14.102785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T03:35:08.521025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T03:35:12.369035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63c0a320-a2f2-4b84-87e7-68501d335285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6803","nonce":3453526793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6805","nonce":3453526793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6809","nonce":3453526793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6807","nonce":3453526793}]},"public_addr":"172.21.15.40:6803/3453526793","cluster_addr":"172.21.15.40:6805/3453526793","heartbeat_back_addr":"172.21.15.40:6809/3453526793","heartbeat_front_addr":"172.21.15.40:6807/3453526793","state":["exists","up"]},{"osd":1,"uuid":"70a31534-06a1-4ffa-a54d-577d45bccf98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6801","nonce":405854252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6803","nonce":405854252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6807","nonce":405854252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6805","nonce":405854252}]},"public_addr":"172.21.15.104:6801/405854252","cluster_addr":"172.21.15.104:6803/405854252","heartbeat_back_addr":"172.21.15.104:6807/405854252","heartbeat_front_addr":"172.21.15.104:6805/405854252","state":["exists","up"]},{"osd":2,"uuid":"f5a89f43-e533-45f0-a777-bd3e9bfa8072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6801","nonce":3919221141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6803","nonce":3919221141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6807","nonce":3919221141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6805","nonce":3919221141}]},"public_addr":"172.21.15.131:6801/3919221141","cluster_addr":"172.21.15.131:6803/3919221141","heartbeat_back_addr":"172.21.15.131:6807/3919221141","heartbeat_front_addr":"172.21.15.131:6805/3919221141","state":["exists","up"]},{"osd":3,"uuid":"4df19390-6378-4346-a2be-29bce1dd2380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6811","nonce":2008541522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6813","nonce":2008541522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6817","nonce":2008541522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6815","nonce":2008541522}]},"public_addr":"172.21.15.40:6811/2008541522","cluster_addr":"172.21.15.40:6813/2008541522","heartbeat_back_addr":"172.21.15.40:6817/2008541522","heartbeat_front_addr":"172.21.15.40:6815/2008541522","state":["exists","up"]},{"osd":4,"uuid":"508de408-1cb1-4423-8199-db6bde9034f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6809","nonce":1153394847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6811","nonce":1153394847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6815","nonce":1153394847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6813","nonce":1153394847}]},"public_addr":"172.21.15.104:6809/1153394847","cluster_addr":"172.21.15.104:6811/1153394847","heartbeat_back_addr":"172.21.15.104:6815/1153394847","heartbeat_front_addr":"172.21.15.104:6813/1153394847","state":["exists","up"]},{"osd":5,"uuid":"a8c0cec9-a8ad-4a5d-800b-8b891ca6d836","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6809","nonce":1051933303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6811","nonce":1051933303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6815","nonce":1051933303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6813","nonce":1051933303}]},"public_addr":"172.21.15.131:6809/1051933303","cluster_addr":"172.21.15.131:6811/1051933303","heartbeat_back_addr":"172.21.15.131:6815/1051933303","heartbeat_front_addr":"172.21.15.131:6813/1051933303","state":["exists","up"]},{"osd":6,"uuid":"5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6819","nonce":2670922011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6821","nonce":2670922011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6825","nonce":2670922011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6823","nonce":2670922011}]},"public_addr":"172.21.15.40:6819/2670922011","cluster_addr":"172.21.15.40:6821/2670922011","heartbeat_back_addr":"172.21.15.40:6825/2670922011","heartbeat_front_addr":"172.21.15.40:6823/2670922011","state":["exists","up"]},{"osd":7,"uuid":"bfc536bb-6ae4-459e-afd9-8197c7420bd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6817","nonce":2353571051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6819","nonce":2353571051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6823","nonce":2353571051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6821","nonce":2353571051}]},"public_addr":"172.21.15.104:6817/2353571051","cluster_addr":"172.21.15.104:6819/2353571051","heartbeat_back_addr":"172.21.15.104:6823/2353571051","heartbeat_front_addr":"172.21.15.104:6821/2353571051","state":["exists","up"]},{"osd":8,"uuid":"1fe214f2-5ea3-4fc5-83bf-d9bccafe1529","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6817","nonce":1364533794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6819","nonce":1364533794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6823","nonce":1364533794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6821","nonce":1364533794}]},"public_addr":"172.21.15.131:6817/1364533794","cluster_addr":"172.21.15.131:6819/1364533794","heartbeat_back_addr":"172.21.15.131:6823/1364533794","heartbeat_front_addr":"172.21.15.131:6821/1364533794","state":["exists","up"]},{"osd":9,"uuid":"4b2b3555-6870-4ead-be50-b635799265a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6827","nonce":2604133190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6829","nonce":2604133190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6833","nonce":2604133190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6831","nonce":2604133190}]},"public_addr":"172.21.15.40:6827/2604133190","cluster_addr":"172.21.15.40:6829/2604133190","heartbeat_back_addr":"172.21.15.40:6833/2604133190","heartbeat_front_addr":"172.21.15.40:6831/2604133190","state":["exists","up"]},{"osd":10,"uuid":"70049e24-94ce-458e-97e3-dbcf9de05444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6825","nonce":2231164832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6827","nonce":2231164832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6831","nonce":2231164832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6829","nonce":2231164832}]},"public_addr":"172.21.15.104:6825/2231164832","cluster_addr":"172.21.15.104:6827/2231164832","heartbeat_back_addr":"172.21.15.104:6831/2231164832","heartbeat_front_addr":"172.21.15.104:6829/2231164832","state":["exists","up"]},{"osd":11,"uuid":"80447548-1fbe-4892-ac13-eb4bb6acf10d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6825","nonce":4128185670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6827","nonce":4128185670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6831","nonce":4128185670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6829","nonce":4128185670}]},"public_addr":"172.21.15.131:6825/4128185670","cluster_addr":"172.21.15.131:6827/4128185670","heartbeat_back_addr":"172.21.15.131:6831/4128185670","heartbeat_front_addr":"172.21.15.131:6829/4128185670","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:23.289784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:47.271677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:10.866792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:36.781924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:01.995848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:26.904606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:52.997438+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:19.221567+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:44.543977+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:11.948545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:38.402943+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:34:04.712200+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/23942578":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2234980855":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6801/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6800/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2301695357":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/1676880727":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6801/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6800/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/1597665640":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/513375601":"2024-06-07T03:27:19.799499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T03:36:14.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.743+0000 7fd0e61c6700 1 -- 172.21.15.40:0/1110377473 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd0cc0a2d30 msgr2=0x7fd0cc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:14.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.743+0000 7fd0e61c6700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd0cc0a2d30 0x7fd0cc0a51f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd0e007a050 tx=0x7fd0c800a300 comp rx=0 tx=0).stop 2024-06-06T03:36:14.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.743+0000 7fd0e61c6700 1 -- 172.21.15.40:0/1110377473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0e0102740 msgr2=0x7fd0e019e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:14.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.743+0000 7fd0e61c6700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0e0102740 0x7fd0e019e770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000d950 tx=0x7fd0d000dc60 comp rx=0 tx=0).stop 2024-06-06T03:36:14.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.743+0000 7fd0e61c6700 1 -- 172.21.15.40:0/1110377473 shutdown_connections 2024-06-06T03:36:14.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.743+0000 7fd0e61c6700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd0cc0a2d30 0x7fd0cc0a51f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:14.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.743+0000 7fd0e61c6700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0e0107840 0x7fd0e0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:14.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.743+0000 7fd0e61c6700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0e0102740 0x7fd0e019e770 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:14.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.743+0000 7fd0e61c6700 1 --2- 172.21.15.40:0/1110377473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0e01030f0 0x7fd0e019ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:14.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.743+0000 7fd0e61c6700 1 -- 172.21.15.40:0/1110377473 >> 172.21.15.40:0/1110377473 conn(0x7fd0e00795f0 msgr2=0x7fd0e0100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:14.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.743+0000 7fd0e61c6700 1 -- 172.21.15.40:0/1110377473 shutdown_connections 2024-06-06T03:36:14.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:14.743+0000 7fd0e61c6700 1 -- 172.21.15.40:0/1110377473 wait complete. 2024-06-06T03:36:15.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:15 smithi040 ceph-mon[32220]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-06T03:36:15.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:15 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:15.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:15 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:15.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:15 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1110377473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:36:15.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:15 smithi104 ceph-mon[36569]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-06T03:36:15.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:15 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:15.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:15 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:15.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:15 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1110377473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:36:15.378 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-06T03:36:15.379 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T03:36:15.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:15 smithi131 ceph-mon[36687]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-06T03:36:15.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:15 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:15.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:15 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:15.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:15 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1110377473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:36:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:16.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:16 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:36:16.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:16 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:36:16.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:16 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:16.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:16 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:36:16.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:16 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:36:16.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:16 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:16.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:16 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:36:16.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:16 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:36:16.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:16 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:36:17.173 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:17 smithi040 ceph-mon[32220]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-06T03:36:17.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:17 smithi104 ceph-mon[36569]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-06T03:36:17.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.414+0000 7fca2a57d700 1 -- 172.21.15.40:0/380515201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2410ed70 msgr2=0x7fca24111160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:17.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.414+0000 7fca2a57d700 1 --2- 172.21.15.40:0/380515201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2410ed70 0x7fca24111160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00b0a0 tx=0x7fca1c00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:36:17.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.414+0000 7fca2a57d700 1 -- 172.21.15.40:0/380515201 shutdown_connections 2024-06-06T03:36:17.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.414+0000 7fca2a57d700 1 --2- 172.21.15.40:0/380515201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca241033c0 0x7fca2406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:17.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.414+0000 7fca2a57d700 1 --2- 172.21.15.40:0/380515201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2410ed70 0x7fca24111160 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.414+0000 7fca2a57d700 1 --2- 172.21.15.40:0/380515201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca2406b120 0x7fca2406b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.414+0000 7fca2a57d700 1 -- 172.21.15.40:0/380515201 >> 172.21.15.40:0/380515201 conn(0x7fca24078280 msgr2=0x7fca24078690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.414+0000 7fca2a57d700 1 -- 172.21.15.40:0/380515201 shutdown_connections 2024-06-06T03:36:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.414+0000 7fca2a57d700 1 -- 172.21.15.40:0/380515201 wait complete. 2024-06-06T03:36:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.415+0000 7fca2a57d700 1 Processor -- start 2024-06-06T03:36:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.415+0000 7fca2a57d700 1 -- start start 2024-06-06T03:36:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.415+0000 7fca2a57d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2406b120 0x7fca2419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.415+0000 7fca2a57d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca241033c0 0x7fca2419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.415+0000 7fca2a57d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca2410ed70 0x7fca2419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.415+0000 7fca2a57d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca241149e0 con 0x7fca2406b120 2024-06-06T03:36:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.415+0000 7fca2a57d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fca24114860 con 0x7fca241033c0 2024-06-06T03:36:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.415+0000 7fca2a57d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fca24114b60 con 0x7fca2410ed70 2024-06-06T03:36:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.415+0000 7fca23fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2406b120 0x7fca2419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.415+0000 7fca23fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2406b120 0x7fca2419aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54470/0 (socket says 172.21.15.40:54470) 2024-06-06T03:36:17.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.415+0000 7fca23fff700 1 -- 172.21.15.40:0/1016271552 learned_addr learned my addr 172.21.15.40:0/1016271552 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:17.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.415+0000 7fca28b1a700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca2410ed70 0x7fca2419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:17.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.415+0000 7fca237fe700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca241033c0 0x7fca2419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:17.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.416+0000 7fca23fff700 1 -- 172.21.15.40:0/1016271552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca2410ed70 msgr2=0x7fca2419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:17.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.416+0000 7fca23fff700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca2410ed70 0x7fca2419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:17.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.416+0000 7fca23fff700 1 -- 172.21.15.40:0/1016271552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca241033c0 msgr2=0x7fca2419b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:17.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.416+0000 7fca23fff700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca241033c0 0x7fca2419b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:17.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.416+0000 7fca23fff700 1 -- 172.21.15.40:0/1016271552 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2419fe70 con 0x7fca2406b120 2024-06-06T03:36:17.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.416+0000 7fca237fe700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca241033c0 0x7fca2419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:36:17.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.416+0000 7fca23fff700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2406b120 0x7fca2419aeb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fca1400d980 tx=0x7fca1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:17.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.416+0000 7fca217fa700 1 -- 172.21.15.40:0/1016271552 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca14017070 con 0x7fca2406b120 2024-06-06T03:36:17.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.416+0000 7fca217fa700 1 -- 172.21.15.40:0/1016271552 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca14005020 con 0x7fca2406b120 2024-06-06T03:36:17.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.416+0000 7fca2a57d700 1 -- 172.21.15.40:0/1016271552 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca241a0160 con 0x7fca2406b120 2024-06-06T03:36:17.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.416+0000 7fca217fa700 1 -- 172.21.15.40:0/1016271552 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca14009bc0 con 0x7fca2406b120 2024-06-06T03:36:17.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.416+0000 7fca2a57d700 1 -- 172.21.15.40:0/1016271552 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca241a79e0 con 0x7fca2406b120 2024-06-06T03:36:17.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.417+0000 7fca2a57d700 1 -- 172.21.15.40:0/1016271552 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca241019f0 con 0x7fca2406b120 2024-06-06T03:36:17.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.419+0000 7fca217fa700 1 -- 172.21.15.40:0/1016271552 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fca140051d0 con 0x7fca2406b120 2024-06-06T03:36:17.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.419+0000 7fca217fa700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fca100a2d30 0x7fca100a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:17.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.419+0000 7fca237fe700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fca100a2d30 0x7fca100a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:17.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.419+0000 7fca217fa700 1 -- 172.21.15.40:0/1016271552 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fca140cf3d0 con 0x7fca2406b120 2024-06-06T03:36:17.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.420+0000 7fca237fe700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fca100a2d30 0x7fca100a51f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fca0c005950 tx=0x7fca0c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:17.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.423+0000 7fca217fa700 1 -- 172.21.15.40:0/1016271552 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fca14020020 con 0x7fca2406b120 2024-06-06T03:36:17.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:17 smithi131 ceph-mon[36687]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-06T03:36:17.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.632+0000 7fca2a57d700 1 -- 172.21.15.40:0/1016271552 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca241a7c10 con 0x7fca2406b120 2024-06-06T03:36:17.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.633+0000 7fca217fa700 1 -- 172.21.15.40:0/1016271552 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22534 (secure 0 0 0) 0x7fca1409c270 con 0x7fca2406b120 2024-06-06T03:36:17.632 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:36:17.632 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"fsid":"77c25d68-23b4-11ef-bc9b-c7b262605968","created":"2024-06-06T03:26:53.165489+0000","modified":"2024-06-06T03:35:48.980461+0000","last_up_change":"2024-06-06T03:34:07.114448+0000","last_in_change":"2024-06-06T03:33:46.152948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T03:30:14.102785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T03:35:08.521025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T03:35:12.369035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63c0a320-a2f2-4b84-87e7-68501d335285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6803","nonce":3453526793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6805","nonce":3453526793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6809","nonce":3453526793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6807","nonce":3453526793}]},"public_addr":"172.21.15.40:6803/3453526793","cluster_addr":"172.21.15.40:6805/3453526793","heartbeat_back_addr":"172.21.15.40:6809/3453526793","heartbeat_front_addr":"172.21.15.40:6807/3453526793","state":["exists","up"]},{"osd":1,"uuid":"70a31534-06a1-4ffa-a54d-577d45bccf98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6801","nonce":405854252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6803","nonce":405854252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6807","nonce":405854252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6805","nonce":405854252}]},"public_addr":"172.21.15.104:6801/405854252","cluster_addr":"172.21.15.104:6803/405854252","heartbeat_back_addr":"172.21.15.104:6807/405854252","heartbeat_front_addr":"172.21.15.104:6805/405854252","state":["exists","up"]},{"osd":2,"uuid":"f5a89f43-e533-45f0-a777-bd3e9bfa8072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6801","nonce":3919221141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6803","nonce":3919221141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6807","nonce":3919221141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6805","nonce":3919221141}]},"public_addr":"172.21.15.131:6801/3919221141","cluster_addr":"172.21.15.131:6803/3919221141","heartbeat_back_addr":"172.21.15.131:6807/3919221141","heartbeat_front_addr":"172.21.15.131:6805/3919221141","state":["exists","up"]},{"osd":3,"uuid":"4df19390-6378-4346-a2be-29bce1dd2380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6811","nonce":2008541522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6813","nonce":2008541522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6817","nonce":2008541522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6815","nonce":2008541522}]},"public_addr":"172.21.15.40:6811/2008541522","cluster_addr":"172.21.15.40:6813/2008541522","heartbeat_back_addr":"172.21.15.40:6817/2008541522","heartbeat_front_addr":"172.21.15.40:6815/2008541522","state":["exists","up"]},{"osd":4,"uuid":"508de408-1cb1-4423-8199-db6bde9034f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6809","nonce":1153394847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6811","nonce":1153394847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6815","nonce":1153394847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6813","nonce":1153394847}]},"public_addr":"172.21.15.104:6809/1153394847","cluster_addr":"172.21.15.104:6811/1153394847","heartbeat_back_addr":"172.21.15.104:6815/1153394847","heartbeat_front_addr":"172.21.15.104:6813/1153394847","state":["exists","up"]},{"osd":5,"uuid":"a8c0cec9-a8ad-4a5d-800b-8b891ca6d836","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6809","nonce":1051933303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6811","nonce":1051933303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6815","nonce":1051933303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6813","nonce":1051933303}]},"public_addr":"172.21.15.131:6809/1051933303","cluster_addr":"172.21.15.131:6811/1051933303","heartbeat_back_addr":"172.21.15.131:6815/1051933303","heartbeat_front_addr":"172.21.15.131:6813/1051933303","state":["exists","up"]},{"osd":6,"uuid":"5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6819","nonce":2670922011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6821","nonce":2670922011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6825","nonce":2670922011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6823","nonce":2670922011}]},"public_addr":"172.21.15.40:6819/2670922011","cluster_addr":"172.21.15.40:6821/2670922011","heartbeat_back_addr":"172.21.15.40:6825/2670922011","heartbeat_front_addr":"172.21.15.40:6823/2670922011","state":["exists","up"]},{"osd":7,"uuid":"bfc536bb-6ae4-459e-afd9-8197c7420bd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6817","nonce":2353571051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6819","nonce":2353571051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6823","nonce":2353571051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6821","nonce":2353571051}]},"public_addr":"172.21.15.104:6817/2353571051","cluster_addr":"172.21.15.104:6819/2353571051","heartbeat_back_addr":"172.21.15.104:6823/2353571051","heartbeat_front_addr":"172.21.15.104:6821/2353571051","state":["exists","up"]},{"osd":8,"uuid":"1fe214f2-5ea3-4fc5-83bf-d9bccafe1529","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6817","nonce":1364533794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6819","nonce":1364533794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6823","nonce":1364533794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6821","nonce":1364533794}]},"public_addr":"172.21.15.131:6817/1364533794","cluster_addr":"172.21.15.131:6819/1364533794","heartbeat_back_addr":"172.21.15.131:6823/1364533794","heartbeat_front_addr":"172.21.15.131:6821/1364533794","state":["exists","up"]},{"osd":9,"uuid":"4b2b3555-6870-4ead-be50-b635799265a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6827","nonce":2604133190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6829","nonce":2604133190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6833","nonce":2604133190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6831","nonce":2604133190}]},"public_addr":"172.21.15.40:6827/2604133190","cluster_addr":"172.21.15.40:6829/2604133190","heartbeat_back_addr":"172.21.15.40:6833/2604133190","heartbeat_front_addr":"172.21.15.40:6831/2604133190","state":["exists","up"]},{"osd":10,"uuid":"70049e24-94ce-458e-97e3-dbcf9de05444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6825","nonce":2231164832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6827","nonce":2231164832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6831","nonce":2231164832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6829","nonce":2231164832}]},"public_addr":"172.21.15.104:6825/2231164832","cluster_addr":"172.21.15.104:6827/2231164832","heartbeat_back_addr":"172.21.15.104:6831/2231164832","heartbeat_front_addr":"172.21.15.104:6829/2231164832","state":["exists","up"]},{"osd":11,"uuid":"80447548-1fbe-4892-ac13-eb4bb6acf10d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6825","nonce":4128185670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6827","nonce":4128185670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6831","nonce":4128185670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6829","nonce":4128185670}]},"public_addr":"172.21.15.131:6825/4128185670","cluster_addr":"172.21.15.131:6827/4128185670","heartbeat_back_addr":"172.21.15.131:6831/4128185670","heartbeat_front_addr":"172.21.15.131:6829/4128185670","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:23.289784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:47.271677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:10.866792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:36.781924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:01.995848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:26.904606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:52.997438+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:19.221567+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:44.543977+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:11.948545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:38.402943+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:34:04.712200+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/23942578":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2234980855":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6801/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6800/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2301695357":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/1676880727":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6801/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6800/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/1597665640":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/513375601":"2024-06-07T03:27:19.799499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T03:36:17.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.636+0000 7fca2a57d700 1 -- 172.21.15.40:0/1016271552 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fca100a2d30 msgr2=0x7fca100a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:17.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.636+0000 7fca2a57d700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fca100a2d30 0x7fca100a51f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fca0c005950 tx=0x7fca0c0058e0 comp rx=0 tx=0).stop 2024-06-06T03:36:17.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.636+0000 7fca2a57d700 1 -- 172.21.15.40:0/1016271552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2406b120 msgr2=0x7fca2419aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:17.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.636+0000 7fca2a57d700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2406b120 0x7fca2419aeb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fca1400d980 tx=0x7fca1400dc90 comp rx=0 tx=0).stop 2024-06-06T03:36:17.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.636+0000 7fca2a57d700 1 -- 172.21.15.40:0/1016271552 shutdown_connections 2024-06-06T03:36:17.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.636+0000 7fca2a57d700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fca100a2d30 0x7fca100a51f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:17.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.636+0000 7fca2a57d700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca2410ed70 0x7fca2419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:17.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.636+0000 7fca2a57d700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2406b120 0x7fca2419aeb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:17.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.636+0000 7fca2a57d700 1 --2- 172.21.15.40:0/1016271552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca241033c0 0x7fca2419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:17.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.636+0000 7fca2a57d700 1 -- 172.21.15.40:0/1016271552 >> 172.21.15.40:0/1016271552 conn(0x7fca24078280 msgr2=0x7fca24100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:17.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.636+0000 7fca2a57d700 1 -- 172.21.15.40:0/1016271552 shutdown_connections 2024-06-06T03:36:17.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:17.636+0000 7fca2a57d700 1 -- 172.21.15.40:0/1016271552 wait complete. 2024-06-06T03:36:18.294 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:18 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1016271552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:36:18.323 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-06T03:36:18.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-06T03:36:18.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-06T03:36:18.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-06T03:36:18.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-06T03:36:18.326 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-06T03:36:18.326 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-06T03:36:18.327 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-06T03:36:18.327 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-06T03:36:18.328 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-06T03:36:18.328 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-06T03:36:18.329 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-06T03:36:18.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:18 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1016271552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:36:18.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:18 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1016271552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:36:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:19.194 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:19.194 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:19.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:19 smithi040 ceph-mon[32220]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-06T03:36:19.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:19 smithi104 ceph-mon[36569]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-06T03:36:19.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:19 smithi131 ceph-mon[36687]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-06T03:36:20.140 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:20.141 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:20.143 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:21.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:21 smithi040 ceph-mon[32220]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:36:21.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:21 smithi104 ceph-mon[36569]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:36:21.383 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:21.383 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:21.384 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:21.388 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:21.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:21 smithi131 ceph-mon[36687]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:36:23.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:23 smithi040 ceph-mon[32220]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:36:23.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:23 smithi104 ceph-mon[36569]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:36:23.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:23 smithi131 ceph-mon[36687]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:36:24.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.398+0000 7f9a4bcec700 1 -- 172.21.15.40:0/757472778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a441085d0 msgr2=0x7f9a441089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:24.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.398+0000 7f9a4bcec700 1 --2- 172.21.15.40:0/757472778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a441085d0 0x7f9a441089b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9a400099c0 tx=0x7f9a40009cd0 comp rx=0 tx=0).stop 2024-06-06T03:36:24.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.398+0000 7f9a4bcec700 1 -- 172.21.15.40:0/757472778 shutdown_connections 2024-06-06T03:36:24.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.398+0000 7f9a4bcec700 1 --2- 172.21.15.40:0/757472778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a44102ef0 0x7f9a4410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:24.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.398+0000 7f9a4bcec700 1 --2- 172.21.15.40:0/757472778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a441085d0 0x7f9a441089b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:24.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.398+0000 7f9a4bcec700 1 --2- 172.21.15.40:0/757472778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a44102530 0x7f9a441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:24.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.398+0000 7f9a4bcec700 1 -- 172.21.15.40:0/757472778 >> 172.21.15.40:0/757472778 conn(0x7f9a44079640 msgr2=0x7f9a44079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:24.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.398+0000 7f9a4bcec700 1 -- 172.21.15.40:0/757472778 shutdown_connections 2024-06-06T03:36:24.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.398+0000 7f9a4bcec700 1 -- 172.21.15.40:0/757472778 wait complete. 2024-06-06T03:36:24.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.399+0000 7f9a4bcec700 1 Processor -- start 2024-06-06T03:36:24.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.399+0000 7f9a4bcec700 1 -- start start 2024-06-06T03:36:24.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.399+0000 7f9a4bcec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a44102530 0x7f9a44074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:24.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.399+0000 7f9a4bcec700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a44102ef0 0x7f9a44074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:24.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.399+0000 7f9a4bcec700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a441085d0 0x7f9a44075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:24.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.399+0000 7f9a4bcec700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44112810 con 0x7f9a44102530 2024-06-06T03:36:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.399+0000 7f9a4bcec700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44112690 con 0x7f9a44102ef0 2024-06-06T03:36:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.399+0000 7f9a4bcec700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44112990 con 0x7f9a441085d0 2024-06-06T03:36:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a49a88700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a44102530 0x7f9a44074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a49a88700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a44102530 0x7f9a44074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34956/0 (socket says 172.21.15.40:34956) 2024-06-06T03:36:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a49a88700 1 -- 172.21.15.40:0/4072020637 learned_addr learned my addr 172.21.15.40:0/4072020637 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a49287700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a44102ef0 0x7f9a44074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a4a289700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a441085d0 0x7f9a44075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a49a88700 1 -- 172.21.15.40:0/4072020637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a441085d0 msgr2=0x7f9a44075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a49a88700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a441085d0 0x7f9a44075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a49a88700 1 -- 172.21.15.40:0/4072020637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a44102ef0 msgr2=0x7f9a44074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a49a88700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a44102ef0 0x7f9a44074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a49a88700 1 -- 172.21.15.40:0/4072020637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a441a70f0 con 0x7f9a44102530 2024-06-06T03:36:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a4a289700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a441085d0 0x7f9a44075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:36:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a49a88700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a44102530 0x7f9a44074790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9a400059d0 tx=0x7f9a400038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:24.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a3affd700 1 -- 172.21.15.40:0/4072020637 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40027070 con 0x7f9a44102530 2024-06-06T03:36:24.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a3affd700 1 -- 172.21.15.40:0/4072020637 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a40004050 con 0x7f9a44102530 2024-06-06T03:36:24.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.400+0000 7f9a3affd700 1 -- 172.21.15.40:0/4072020637 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40022d80 con 0x7f9a44102530 2024-06-06T03:36:24.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.401+0000 7f9a4bcec700 1 -- 172.21.15.40:0/4072020637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a441a7380 con 0x7f9a44102530 2024-06-06T03:36:24.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.401+0000 7f9a4bcec700 1 -- 172.21.15.40:0/4072020637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a441a7780 con 0x7f9a44102530 2024-06-06T03:36:24.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.402+0000 7f9a4bcec700 1 -- 172.21.15.40:0/4072020637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9a441018d0 con 0x7f9a44102530 2024-06-06T03:36:24.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.403+0000 7f9a3affd700 1 -- 172.21.15.40:0/4072020637 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9a4002a070 con 0x7f9a44102530 2024-06-06T03:36:24.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.404+0000 7f9a3affd700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9a300a2cb0 0x7f9a300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:24.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.404+0000 7f9a49287700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9a300a2cb0 0x7f9a300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:24.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.404+0000 7f9a3affd700 1 -- 172.21.15.40:0/4072020637 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f9a400d0b80 con 0x7f9a44102530 2024-06-06T03:36:24.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.404+0000 7f9a3affd700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051] conn(0x7f9a300ac670 0x7f9a300aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:24.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.404+0000 7f9a3affd700 1 -- 172.21.15.40:0/4072020637 --> [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9a300af1e0 con 0x7f9a300ac670 2024-06-06T03:36:24.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.404+0000 7f9a3affd700 1 -- 172.21.15.40:0/4072020637 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f9a400d0f30 con 0x7f9a44102530 2024-06-06T03:36:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.404+0000 7f9a4a289700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051] conn(0x7f9a300ac670 0x7f9a300aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.404+0000 7f9a49287700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9a300a2cb0 0x7f9a300a5170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9a44108c30 tx=0x7f9a3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.405+0000 7f9a4a289700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051] conn(0x7f9a300ac670 0x7f9a300aeb10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.405+0000 7f9a3affd700 1 -- 172.21.15.40:0/4072020637 <== osd.7 v2:172.21.15.104:6816/2353571051 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a300af1e0 con 0x7f9a300ac670 2024-06-06T03:36:24.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.422+0000 7f9a4bcec700 1 -- 172.21.15.40:0/4072020637 --> [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9a44103c40 con 0x7f9a300ac670 2024-06-06T03:36:24.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.423+0000 7f9a3affd700 1 -- 172.21.15.40:0/4072020637 <== osd.7 v2:172.21.15.104:6816/2353571051 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9a44103c40 con 0x7f9a300ac670 2024-06-06T03:36:24.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.423+0000 7f9a4bcec700 1 -- 172.21.15.40:0/4072020637 >> [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051] conn(0x7f9a300ac670 msgr2=0x7f9a300aeb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:24.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.423+0000 7f9a4bcec700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051] conn(0x7f9a300ac670 0x7f9a300aeb10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:24.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.423+0000 7f9a4bcec700 1 -- 172.21.15.40:0/4072020637 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9a300a2cb0 msgr2=0x7f9a300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.423+0000 7f9a4bcec700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9a300a2cb0 0x7f9a300a5170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9a44108c30 tx=0x7f9a3400b410 comp rx=0 tx=0).stop 2024-06-06T03:36:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.423+0000 7f9a4bcec700 1 -- 172.21.15.40:0/4072020637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a44102530 msgr2=0x7f9a44074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.423+0000 7f9a4bcec700 1 --2- 172.21.15.40:0/4072020637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a44102530 0x7f9a44074790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9a400059d0 tx=0x7f9a400038b0 comp rx=0 tx=0).stop 2024-06-06T03:36:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.423+0000 7f9a4a289700 1 -- 172.21.15.40:0/4072020637 reap_dead start 2024-06-06T03:36:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.423+0000 7f9a4bcec700 1 -- 172.21.15.40:0/4072020637 shutdown_connections 2024-06-06T03:36:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.423+0000 7f9a4bcec700 1 -- 172.21.15.40:0/4072020637 >> 172.21.15.40:0/4072020637 conn(0x7f9a44079640 msgr2=0x7f9a44100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.424+0000 7f9a4bcec700 1 -- 172.21.15.40:0/4072020637 shutdown_connections 2024-06-06T03:36:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:24.424+0000 7f9a4bcec700 1 -- 172.21.15.40:0/4072020637 wait complete. 2024-06-06T03:36:25.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:25 smithi040 ceph-mon[32220]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:36:25.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:25 smithi104 ceph-mon[36569]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:36:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:25 smithi131 ceph-mon[36687]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:36:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.071+0000 7f32fa306700 1 -- 172.21.15.40:0/4019550445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f4101520 msgr2=0x7f32f4111210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.071+0000 7f32fa306700 1 --2- 172.21.15.40:0/4019550445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f4101520 0x7f32f4111210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00b0a0 tx=0x7f32ec00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:36:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.071+0000 7f32fa306700 1 -- 172.21.15.40:0/4019550445 shutdown_connections 2024-06-06T03:36:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.071+0000 7f32fa306700 1 --2- 172.21.15.40:0/4019550445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f4103360 0x7f32f4103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.071+0000 7f32fa306700 1 --2- 172.21.15.40:0/4019550445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f4101520 0x7f32f4111210 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.071+0000 7f32fa306700 1 --2- 172.21.15.40:0/4019550445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32f4100a80 0x7f32f4100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.071+0000 7f32fa306700 1 -- 172.21.15.40:0/4019550445 >> 172.21.15.40:0/4019550445 conn(0x7f32f4077a00 msgr2=0x7f32f40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.071+0000 7f32fa306700 1 -- 172.21.15.40:0/4019550445 shutdown_connections 2024-06-06T03:36:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.071+0000 7f32fa306700 1 -- 172.21.15.40:0/4019550445 wait complete. 2024-06-06T03:36:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.071+0000 7f32fa306700 1 Processor -- start 2024-06-06T03:36:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.071+0000 7f32fa306700 1 -- start start 2024-06-06T03:36:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32fa306700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f4100a80 0x7f32f419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32fa306700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32f4101520 0x7f32f419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32fa306700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f4103360 0x7f32f419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32fa306700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4114960 con 0x7f32f4100a80 2024-06-06T03:36:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32fa306700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41147e0 con 0x7f32f4101520 2024-06-06T03:36:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32fa306700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4114ae0 con 0x7f32f4103360 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32f3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f4100a80 0x7f32f419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32f3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f4100a80 0x7f32f419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34980/0 (socket says 172.21.15.40:34980) 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32f3fff700 1 -- 172.21.15.40:0/1732156499 learned_addr learned my addr 172.21.15.40:0/1732156499 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32f37fe700 1 --2- 172.21.15.40:0/1732156499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32f4101520 0x7f32f419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32f88a3700 1 --2- 172.21.15.40:0/1732156499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f4103360 0x7f32f419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32f3fff700 1 -- 172.21.15.40:0/1732156499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f4103360 msgr2=0x7f32f419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32f3fff700 1 --2- 172.21.15.40:0/1732156499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f4103360 0x7f32f419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32f3fff700 1 -- 172.21.15.40:0/1732156499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32f4101520 msgr2=0x7f32f419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32f3fff700 1 --2- 172.21.15.40:0/1732156499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32f4101520 0x7f32f419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32f3fff700 1 -- 172.21.15.40:0/1732156499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f419fdd0 con 0x7f32f4100a80 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32f3fff700 1 --2- 172.21.15.40:0/1732156499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f4100a80 0x7f32f419ae10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f32e400d980 tx=0x7f32e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.072+0000 7f32f17fa700 1 -- 172.21.15.40:0/1732156499 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4017070 con 0x7f32f4100a80 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.073+0000 7f32fa306700 1 -- 172.21.15.40:0/1732156499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f41a00c0 con 0x7f32f4100a80 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.073+0000 7f32f17fa700 1 -- 172.21.15.40:0/1732156499 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e4005020 con 0x7f32f4100a80 2024-06-06T03:36:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.073+0000 7f32f17fa700 1 -- 172.21.15.40:0/1732156499 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e400fbf0 con 0x7f32f4100a80 2024-06-06T03:36:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.073+0000 7f32fa306700 1 -- 172.21.15.40:0/1732156499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f41a7940 con 0x7f32f4100a80 2024-06-06T03:36:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.074+0000 7f32fa306700 1 -- 172.21.15.40:0/1732156499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32f4062c20 con 0x7f32f4100a80 2024-06-06T03:36:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.075+0000 7f32f17fa700 1 -- 172.21.15.40:0/1732156499 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f32e40051d0 con 0x7f32f4100a80 2024-06-06T03:36:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.076+0000 7f32f17fa700 1 --2- 172.21.15.40:0/1732156499 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f32dc0a2cb0 0x7f32dc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.076+0000 7f32f37fe700 1 --2- 172.21.15.40:0/1732156499 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f32dc0a2cb0 0x7f32dc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.076+0000 7f32f17fa700 1 -- 172.21.15.40:0/1732156499 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f32e40cf4b0 con 0x7f32f4100a80 2024-06-06T03:36:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.076+0000 7f32f17fa700 1 --2- 172.21.15.40:0/1732156499 >> [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832] conn(0x7f32dc0ac670 0x7f32dc0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.076+0000 7f32f17fa700 1 -- 172.21.15.40:0/1732156499 --> [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32e40044e0 con 0x7f32dc0ac670 2024-06-06T03:36:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.076+0000 7f32f17fa700 1 -- 172.21.15.40:0/1732156499 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f32e40cf860 con 0x7f32f4100a80 2024-06-06T03:36:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.076+0000 7f32f37fe700 1 --2- 172.21.15.40:0/1732156499 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f32dc0a2cb0 0x7f32dc0a5170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f32f4100030 tx=0x7f32e80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.077+0000 7f32f88a3700 1 --2- 172.21.15.40:0/1732156499 >> [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832] conn(0x7f32dc0ac670 0x7f32dc0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.077+0000 7f32f88a3700 1 --2- 172.21.15.40:0/1732156499 >> [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832] conn(0x7f32dc0ac670 0x7f32dc0aeb10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.078+0000 7f32f17fa700 1 -- 172.21.15.40:0/1732156499 <== osd.10 v2:172.21.15.104:6824/2231164832 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32e40044e0 con 0x7f32dc0ac670 2024-06-06T03:36:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.095+0000 7f32fa306700 1 -- 172.21.15.40:0/1732156499 --> [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32f40630a0 con 0x7f32dc0ac670 2024-06-06T03:36:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.095+0000 7f32f17fa700 1 -- 172.21.15.40:0/1732156499 <== osd.10 v2:172.21.15.104:6824/2231164832 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f32f40630a0 con 0x7f32dc0ac670 2024-06-06T03:36:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.095+0000 7f32fa306700 1 -- 172.21.15.40:0/1732156499 >> [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832] conn(0x7f32dc0ac670 msgr2=0x7f32dc0aeb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.095+0000 7f32fa306700 1 --2- 172.21.15.40:0/1732156499 >> [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832] conn(0x7f32dc0ac670 0x7f32dc0aeb10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.095+0000 7f32fa306700 1 -- 172.21.15.40:0/1732156499 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f32dc0a2cb0 msgr2=0x7f32dc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.095+0000 7f32fa306700 1 --2- 172.21.15.40:0/1732156499 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f32dc0a2cb0 0x7f32dc0a5170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f32f4100030 tx=0x7f32e80073d0 comp rx=0 tx=0).stop 2024-06-06T03:36:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.095+0000 7f32fa306700 1 -- 172.21.15.40:0/1732156499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f4100a80 msgr2=0x7f32f419ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.095+0000 7f32fa306700 1 --2- 172.21.15.40:0/1732156499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f4100a80 0x7f32f419ae10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f32e400d980 tx=0x7f32e400dc90 comp rx=0 tx=0).stop 2024-06-06T03:36:26.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.096+0000 7f32f88a3700 1 -- 172.21.15.40:0/1732156499 reap_dead start 2024-06-06T03:36:26.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.096+0000 7f32fa306700 1 -- 172.21.15.40:0/1732156499 shutdown_connections 2024-06-06T03:36:26.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.096+0000 7f32fa306700 1 -- 172.21.15.40:0/1732156499 >> 172.21.15.40:0/1732156499 conn(0x7f32f4077a00 msgr2=0x7f32f40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:26.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.096+0000 7f32fa306700 1 -- 172.21.15.40:0/1732156499 shutdown_connections 2024-06-06T03:36:26.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:25.096+0000 7f32fa306700 1 -- 172.21.15.40:0/1732156499 wait complete. 2024-06-06T03:36:26.712 INFO:teuthology.orchestra.run.smithi040.stdout:309237645348 2024-06-06T03:36:26.712 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-06T03:36:27.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:27 smithi040 ceph-mon[32220]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-06T03:36:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:27 smithi104 ceph-mon[36569]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-06T03:36:27.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:27 smithi131 ceph-mon[36687]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-06T03:36:29.152 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:29 smithi040 ceph-mon[32220]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:36:29.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:29 smithi104 ceph-mon[36569]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:36:29.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.366+0000 7f8d0ea76700 1 -- 172.21.15.40:0/365407503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d08102530 msgr2=0x7f8d081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:29.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.366+0000 7f8d0ea76700 1 --2- 172.21.15.40:0/365407503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d08102530 0x7f8d081029b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40099c0 tx=0x7f8cf4009cd0 comp rx=0 tx=0).stop 2024-06-06T03:36:29.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.366+0000 7f8d0ea76700 1 -- 172.21.15.40:0/365407503 shutdown_connections 2024-06-06T03:36:29.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.366+0000 7f8d0ea76700 1 --2- 172.21.15.40:0/365407503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d081085d0 0x7f8d081089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:29.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.366+0000 7f8d0ea76700 1 --2- 172.21.15.40:0/365407503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d08102ef0 0x7f8d0810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:29.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.366+0000 7f8d0ea76700 1 --2- 172.21.15.40:0/365407503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d08102530 0x7f8d081029b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:29.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.366+0000 7f8d0ea76700 1 -- 172.21.15.40:0/365407503 >> 172.21.15.40:0/365407503 conn(0x7f8d08079640 msgr2=0x7f8d08079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.366+0000 7f8d0ea76700 1 -- 172.21.15.40:0/365407503 shutdown_connections 2024-06-06T03:36:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.366+0000 7f8d0ea76700 1 -- 172.21.15.40:0/365407503 wait complete. 2024-06-06T03:36:29.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.367+0000 7f8d0ea76700 1 Processor -- start 2024-06-06T03:36:29.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.367+0000 7f8d0ea76700 1 -- start start 2024-06-06T03:36:29.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.367+0000 7f8d0ea76700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d08102530 0x7f8d0819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:29.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.367+0000 7f8d0ea76700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d08102ef0 0x7f8d0819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:29.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.367+0000 7f8d0ea76700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d081085d0 0x7f8d0819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:29.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.367+0000 7f8d0ea76700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8d08112810 con 0x7f8d08102ef0 2024-06-06T03:36:29.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.367+0000 7f8d0ea76700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8d08112690 con 0x7f8d081085d0 2024-06-06T03:36:29.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.367+0000 7f8d0ea76700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8d08112990 con 0x7f8d08102530 2024-06-06T03:36:29.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.367+0000 7f8d07fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d08102ef0 0x7f8d0819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:29.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.367+0000 7f8d07fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d08102ef0 0x7f8d0819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50790/0 (socket says 172.21.15.40:50790) 2024-06-06T03:36:29.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.367+0000 7f8d07fff700 1 -- 172.21.15.40:0/3228023629 learned_addr learned my addr 172.21.15.40:0/3228023629 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.367+0000 7f8d0c812700 1 --2- 172.21.15.40:0/3228023629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d08102530 0x7f8d0819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.368+0000 7f8d0d013700 1 --2- 172.21.15.40:0/3228023629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d081085d0 0x7f8d0819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.368+0000 7f8d07fff700 1 -- 172.21.15.40:0/3228023629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d08102530 msgr2=0x7f8d0819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.368+0000 7f8d07fff700 1 --2- 172.21.15.40:0/3228023629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d08102530 0x7f8d0819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.368+0000 7f8d07fff700 1 -- 172.21.15.40:0/3228023629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d081085d0 msgr2=0x7f8d0819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.368+0000 7f8d07fff700 1 --2- 172.21.15.40:0/3228023629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d081085d0 0x7f8d0819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.368+0000 7f8d07fff700 1 -- 172.21.15.40:0/3228023629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0819fe20 con 0x7f8d08102ef0 2024-06-06T03:36:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.368+0000 7f8d07fff700 1 --2- 172.21.15.40:0/3228023629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d08102ef0 0x7f8d0819b350 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4006010 tx=0x7f8cf4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.368+0000 7f8d05ffb700 1 -- 172.21.15.40:0/3228023629 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4026070 con 0x7f8d08102ef0 2024-06-06T03:36:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.368+0000 7f8d0ea76700 1 -- 172.21.15.40:0/3228023629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d081a00b0 con 0x7f8d08102ef0 2024-06-06T03:36:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.368+0000 7f8d05ffb700 1 -- 172.21.15.40:0/3228023629 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cf4004cd0 con 0x7f8d08102ef0 2024-06-06T03:36:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.368+0000 7f8d05ffb700 1 -- 172.21.15.40:0/3228023629 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4003ba0 con 0x7f8d08102ef0 2024-06-06T03:36:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.369+0000 7f8d0ea76700 1 -- 172.21.15.40:0/3228023629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d081a7990 con 0x7f8d08102ef0 2024-06-06T03:36:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.369+0000 7f8d0ea76700 1 -- 172.21.15.40:0/3228023629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8d080762e0 con 0x7f8d08102ef0 2024-06-06T03:36:29.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.371+0000 7f8d05ffb700 1 -- 172.21.15.40:0/3228023629 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8cf4029070 con 0x7f8d08102ef0 2024-06-06T03:36:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.371+0000 7f8d05ffb700 1 --2- 172.21.15.40:0/3228023629 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8cf80a2cb0 0x7f8cf80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.372+0000 7f8d05ffb700 1 -- 172.21.15.40:0/3228023629 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f8cf40d00d0 con 0x7f8d08102ef0 2024-06-06T03:36:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.372+0000 7f8d0c812700 1 --2- 172.21.15.40:0/3228023629 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8cf80a2cb0 0x7f8cf80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.372+0000 7f8d05ffb700 1 --2- 172.21.15.40:0/3228023629 >> [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190] conn(0x7f8cf80ac6d0 0x7f8cf80aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.372+0000 7f8d05ffb700 1 -- 172.21.15.40:0/3228023629 --> [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8cf80af240 con 0x7f8cf80ac6d0 2024-06-06T03:36:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.372+0000 7f8d05ffb700 1 -- 172.21.15.40:0/3228023629 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f8cf40d05b0 con 0x7f8d08102ef0 2024-06-06T03:36:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.372+0000 7f8d0d013700 1 --2- 172.21.15.40:0/3228023629 >> [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190] conn(0x7f8cf80ac6d0 0x7f8cf80aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.372+0000 7f8d0c812700 1 --2- 172.21.15.40:0/3228023629 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8cf80a2cb0 0x7f8cf80a5170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005950 tx=0x7f8cfc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.372+0000 7f8d0d013700 1 --2- 172.21.15.40:0/3228023629 >> [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190] conn(0x7f8cf80ac6d0 0x7f8cf80aeb70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:29.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.373+0000 7f8d05ffb700 1 -- 172.21.15.40:0/3228023629 <== osd.9 v2:172.21.15.40:6826/2604133190 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8cf80af240 con 0x7f8cf80ac6d0 2024-06-06T03:36:29.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.389+0000 7f8d0ea76700 1 -- 172.21.15.40:0/3228023629 --> [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8d08103c40 con 0x7f8cf80ac6d0 2024-06-06T03:36:29.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.389+0000 7f8d05ffb700 1 -- 172.21.15.40:0/3228023629 <== osd.9 v2:172.21.15.40:6826/2604133190 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8d08103c40 con 0x7f8cf80ac6d0 2024-06-06T03:36:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.390+0000 7f8d0ea76700 1 -- 172.21.15.40:0/3228023629 >> [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190] conn(0x7f8cf80ac6d0 msgr2=0x7f8cf80aeb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.390+0000 7f8d0ea76700 1 --2- 172.21.15.40:0/3228023629 >> [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190] conn(0x7f8cf80ac6d0 0x7f8cf80aeb70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.390+0000 7f8d0ea76700 1 -- 172.21.15.40:0/3228023629 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8cf80a2cb0 msgr2=0x7f8cf80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.390+0000 7f8d0ea76700 1 --2- 172.21.15.40:0/3228023629 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f8cf80a2cb0 0x7f8cf80a5170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005950 tx=0x7f8cfc00b410 comp rx=0 tx=0).stop 2024-06-06T03:36:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.390+0000 7f8d0ea76700 1 -- 172.21.15.40:0/3228023629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d08102ef0 msgr2=0x7f8d0819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.390+0000 7f8d0ea76700 1 --2- 172.21.15.40:0/3228023629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d08102ef0 0x7f8d0819b350 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4006010 tx=0x7f8cf4009f80 comp rx=0 tx=0).stop 2024-06-06T03:36:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.390+0000 7f8d0d013700 1 -- 172.21.15.40:0/3228023629 reap_dead start 2024-06-06T03:36:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.390+0000 7f8d0ea76700 1 -- 172.21.15.40:0/3228023629 shutdown_connections 2024-06-06T03:36:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.390+0000 7f8d0ea76700 1 -- 172.21.15.40:0/3228023629 >> 172.21.15.40:0/3228023629 conn(0x7f8d08079640 msgr2=0x7f8d080748f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.390+0000 7f8d0ea76700 1 -- 172.21.15.40:0/3228023629 shutdown_connections 2024-06-06T03:36:29.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:29.390+0000 7f8d0ea76700 1 -- 172.21.15.40:0/3228023629 wait complete. 2024-06-06T03:36:29.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:29 smithi131 ceph-mon[36687]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:36:31.193 INFO:teuthology.orchestra.run.smithi040.stdout:287762808874 2024-06-06T03:36:31.193 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-06T03:36:31.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:31 smithi040 ceph-mon[32220]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:36:31.334 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:31 smithi131 ceph-mon[36687]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:36:31.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:31 smithi104 ceph-mon[36569]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:36:33.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.123+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3832236094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eb8101d50 msgr2=0x7f6eb810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.123+0000 7f6ebe5cc700 1 --2- 172.21.15.40:0/3832236094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eb8101d50 0x7f6eb810e240 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000b0a0 tx=0x7f6eb000b3b0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.123+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3832236094 shutdown_connections 2024-06-06T03:36:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.123+0000 7f6ebe5cc700 1 --2- 172.21.15.40:0/3832236094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eb8101d50 0x7f6eb810e240 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.123+0000 7f6ebe5cc700 1 --2- 172.21.15.40:0/3832236094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6eb8107400 0x7f6eb81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.123+0000 7f6ebe5cc700 1 --2- 172.21.15.40:0/3832236094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb8101390 0x7f6eb8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.123+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3832236094 >> 172.21.15.40:0/3832236094 conn(0x7f6eb8078340 msgr2=0x7f6eb8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.123+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3832236094 shutdown_connections 2024-06-06T03:36:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.123+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3832236094 wait complete. 2024-06-06T03:36:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.124+0000 7f6ebe5cc700 1 Processor -- start 2024-06-06T03:36:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.124+0000 7f6ebe5cc700 1 -- start start 2024-06-06T03:36:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.124+0000 7f6ebe5cc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6eb8101390 0x7f6eb819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.124+0000 7f6ebe5cc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb8101d50 0x7f6eb819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.124+0000 7f6ebe5cc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eb8107400 0x7f6eb81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.124+0000 7f6ebe5cc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb81116a0 con 0x7f6eb8107400 2024-06-06T03:36:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.124+0000 7f6ebe5cc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb8111520 con 0x7f6eb8101390 2024-06-06T03:36:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.124+0000 7f6ebe5cc700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb8111820 con 0x7f6eb8101d50 2024-06-06T03:36:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.124+0000 7f6ebcb69700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eb8107400 0x7f6eb81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6ebcb69700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eb8107400 0x7f6eb81989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50820/0 (socket says 172.21.15.40:50820) 2024-06-06T03:36:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6ebcb69700 1 -- 172.21.15.40:0/3260685908 learned_addr learned my addr 172.21.15.40:0/3260685908 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6eb7fff700 1 --2- 172.21.15.40:0/3260685908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6eb8101390 0x7f6eb819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6eaffff700 1 --2- 172.21.15.40:0/3260685908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb8101d50 0x7f6eb819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6ebcb69700 1 -- 172.21.15.40:0/3260685908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb8101d50 msgr2=0x7f6eb819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6ebcb69700 1 --2- 172.21.15.40:0/3260685908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb8101d50 0x7f6eb819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6ebcb69700 1 -- 172.21.15.40:0/3260685908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6eb8101390 msgr2=0x7f6eb819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6ebcb69700 1 --2- 172.21.15.40:0/3260685908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6eb8101390 0x7f6eb819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6ebcb69700 1 -- 172.21.15.40:0/3260685908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb8199100 con 0x7f6eb8107400 2024-06-06T03:36:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6ebcb69700 1 --2- 172.21.15.40:0/3260685908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eb8107400 0x7f6eb81989a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00062a0 tx=0x7f6eb0004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:33.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6eb5ffb700 1 -- 172.21.15.40:0/3260685908 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0021070 con 0x7f6eb8107400 2024-06-06T03:36:33.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3260685908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb8199390 con 0x7f6eb8107400 2024-06-06T03:36:33.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6eb5ffb700 1 -- 172.21.15.40:0/3260685908 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb0007c00 con 0x7f6eb8107400 2024-06-06T03:36:33.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6eb5ffb700 1 -- 172.21.15.40:0/3260685908 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb001d450 con 0x7f6eb8107400 2024-06-06T03:36:33.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.125+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3260685908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb81a56b0 con 0x7f6eb8107400 2024-06-06T03:36:33.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.126+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3260685908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6eb81054c0 con 0x7f6eb8107400 2024-06-06T03:36:33.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.128+0000 7f6eb5ffb700 1 -- 172.21.15.40:0/3260685908 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6eb0004000 con 0x7f6eb8107400 2024-06-06T03:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.128+0000 7f6eb5ffb700 1 --2- 172.21.15.40:0/3260685908 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6ea40a2cb0 0x7f6ea40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.128+0000 7f6eb7fff700 1 --2- 172.21.15.40:0/3260685908 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6ea40a2cb0 0x7f6ea40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.128+0000 7f6eb5ffb700 1 -- 172.21.15.40:0/3260685908 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f6eb000f020 con 0x7f6eb8107400 2024-06-06T03:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.128+0000 7f6eb5ffb700 1 --2- 172.21.15.40:0/3260685908 >> [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522] conn(0x7f6ea40ac670 0x7f6ea40aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.129+0000 7f6eb5ffb700 1 -- 172.21.15.40:0/3260685908 --> [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6eb0008ea0 con 0x7f6ea40ac670 2024-06-06T03:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.129+0000 7f6eb5ffb700 1 -- 172.21.15.40:0/3260685908 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f6eb00d0ce0 con 0x7f6eb8107400 2024-06-06T03:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.129+0000 7f6eaffff700 1 --2- 172.21.15.40:0/3260685908 >> [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522] conn(0x7f6ea40ac670 0x7f6ea40aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.129+0000 7f6eb7fff700 1 --2- 172.21.15.40:0/3260685908 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6ea40a2cb0 0x7f6ea40a5170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8199ae0 tx=0x7f6ea800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.129+0000 7f6eaffff700 1 --2- 172.21.15.40:0/3260685908 >> [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522] conn(0x7f6ea40ac670 0x7f6ea40aeb10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.130+0000 7f6eb5ffb700 1 -- 172.21.15.40:0/3260685908 <== osd.3 v2:172.21.15.40:6810/2008541522 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6eb0008ea0 con 0x7f6ea40ac670 2024-06-06T03:36:33.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.146+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3260685908 --> [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6eb8102ab0 con 0x7f6ea40ac670 2024-06-06T03:36:33.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.147+0000 7f6eb5ffb700 1 -- 172.21.15.40:0/3260685908 <== osd.3 v2:172.21.15.40:6810/2008541522 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6eb8102ab0 con 0x7f6ea40ac670 2024-06-06T03:36:33.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.147+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3260685908 >> [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522] conn(0x7f6ea40ac670 msgr2=0x7f6ea40aeb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:33.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.147+0000 7f6ebe5cc700 1 --2- 172.21.15.40:0/3260685908 >> [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522] conn(0x7f6ea40ac670 0x7f6ea40aeb10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.147+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3260685908 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6ea40a2cb0 msgr2=0x7f6ea40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:33.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.147+0000 7f6ebe5cc700 1 --2- 172.21.15.40:0/3260685908 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6ea40a2cb0 0x7f6ea40a5170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8199ae0 tx=0x7f6ea800b410 comp rx=0 tx=0).stop 2024-06-06T03:36:33.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.147+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3260685908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eb8107400 msgr2=0x7f6eb81989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:33.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.147+0000 7f6ebe5cc700 1 --2- 172.21.15.40:0/3260685908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eb8107400 0x7f6eb81989a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00062a0 tx=0x7f6eb0004d70 comp rx=0 tx=0).stop 2024-06-06T03:36:33.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.147+0000 7f6ebcb69700 1 -- 172.21.15.40:0/3260685908 reap_dead start 2024-06-06T03:36:33.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.147+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3260685908 shutdown_connections 2024-06-06T03:36:33.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.147+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3260685908 >> 172.21.15.40:0/3260685908 conn(0x7f6eb8078340 msgr2=0x7f6eb8100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:33.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.147+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3260685908 shutdown_connections 2024-06-06T03:36:33.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.147+0000 7f6ebe5cc700 1 -- 172.21.15.40:0/3260685908 wait complete. 2024-06-06T03:36:33.292 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:33 smithi040 ceph-mon[32220]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:36:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.297+0000 7f5e2454a700 1 -- 172.21.15.40:0/2699905807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e1c102ef0 msgr2=0x7f5e1c10f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.297+0000 7f5e2454a700 1 --2- 172.21.15.40:0/2699905807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e1c102ef0 0x7f5e1c10f3a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400b0d0 tx=0x7f5e1400b3e0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.297+0000 7f5e2454a700 1 -- 172.21.15.40:0/2699905807 shutdown_connections 2024-06-06T03:36:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.297+0000 7f5e2454a700 1 --2- 172.21.15.40:0/2699905807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e1c1085d0 0x7f5e1c1089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.297+0000 7f5e2454a700 1 --2- 172.21.15.40:0/2699905807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e1c102ef0 0x7f5e1c10f3a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.297+0000 7f5e2454a700 1 --2- 172.21.15.40:0/2699905807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e1c102530 0x7f5e1c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.297+0000 7f5e2454a700 1 -- 172.21.15.40:0/2699905807 >> 172.21.15.40:0/2699905807 conn(0x7f5e1c079640 msgr2=0x7f5e1c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.297+0000 7f5e2454a700 1 -- 172.21.15.40:0/2699905807 shutdown_connections 2024-06-06T03:36:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.297+0000 7f5e2454a700 1 -- 172.21.15.40:0/2699905807 wait complete. 2024-06-06T03:36:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e2454a700 1 Processor -- start 2024-06-06T03:36:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e2454a700 1 -- start start 2024-06-06T03:36:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e2454a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e1c102530 0x7f5e1c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e2454a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e1c102ef0 0x7f5e1c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e2454a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e1c1085d0 0x7f5e1c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e2454a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c112880 con 0x7f5e1c1085d0 2024-06-06T03:36:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e2454a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c112700 con 0x7f5e1c102ef0 2024-06-06T03:36:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e2454a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c112a00 con 0x7f5e1c102530 2024-06-06T03:36:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e22ae7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e1c1085d0 0x7f5e1c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e22ae7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e1c1085d0 0x7f5e1c198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50840/0 (socket says 172.21.15.40:50840) 2024-06-06T03:36:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e22ae7700 1 -- 172.21.15.40:0/2594211199 learned_addr learned my addr 172.21.15.40:0/2594211199 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e21ae5700 1 --2- 172.21.15.40:0/2594211199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e1c102ef0 0x7f5e1c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e222e6700 1 --2- 172.21.15.40:0/2594211199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e1c102530 0x7f5e1c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e22ae7700 1 -- 172.21.15.40:0/2594211199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e1c102530 msgr2=0x7f5e1c19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e22ae7700 1 --2- 172.21.15.40:0/2594211199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e1c102530 0x7f5e1c19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e22ae7700 1 -- 172.21.15.40:0/2594211199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e1c102ef0 msgr2=0x7f5e1c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e22ae7700 1 --2- 172.21.15.40:0/2594211199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e1c102ef0 0x7f5e1c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.298+0000 7f5e22ae7700 1 -- 172.21.15.40:0/2594211199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1c199240 con 0x7f5e1c1085d0 2024-06-06T03:36:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.299+0000 7f5e22ae7700 1 --2- 172.21.15.40:0/2594211199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e1c1085d0 0x7f5e1c198ae0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5e14006070 tx=0x7f5e14004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.299+0000 7f5e137fe700 1 -- 172.21.15.40:0/2594211199 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14021070 con 0x7f5e1c1085d0 2024-06-06T03:36:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.299+0000 7f5e2454a700 1 -- 172.21.15.40:0/2594211199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1c1994d0 con 0x7f5e1c1085d0 2024-06-06T03:36:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.299+0000 7f5e137fe700 1 -- 172.21.15.40:0/2594211199 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e14003c00 con 0x7f5e1c1085d0 2024-06-06T03:36:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.299+0000 7f5e137fe700 1 -- 172.21.15.40:0/2594211199 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14004830 con 0x7f5e1c1085d0 2024-06-06T03:36:33.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.299+0000 7f5e2454a700 1 -- 172.21.15.40:0/2594211199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e1c1a57a0 con 0x7f5e1c1085d0 2024-06-06T03:36:33.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.300+0000 7f5e2454a700 1 -- 172.21.15.40:0/2594211199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e1c062ba0 con 0x7f5e1c1085d0 2024-06-06T03:36:33.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.301+0000 7f5e137fe700 1 -- 172.21.15.40:0/2594211199 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5e14003db0 con 0x7f5e1c1085d0 2024-06-06T03:36:33.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.302+0000 7f5e137fe700 1 --2- 172.21.15.40:0/2594211199 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5e080a2cb0 0x7f5e080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:33.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.302+0000 7f5e222e6700 1 --2- 172.21.15.40:0/2594211199 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5e080a2cb0 0x7f5e080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:33.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.302+0000 7f5e137fe700 1 -- 172.21.15.40:0/2594211199 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f5e1400f020 con 0x7f5e1c1085d0 2024-06-06T03:36:33.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.302+0000 7f5e137fe700 1 --2- 172.21.15.40:0/2594211199 >> [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793] conn(0x7f5e080ac670 0x7f5e080aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:33.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.302+0000 7f5e137fe700 1 -- 172.21.15.40:0/2594211199 --> [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e080af1e0 con 0x7f5e080ac670 2024-06-06T03:36:33.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.302+0000 7f5e137fe700 1 -- 172.21.15.40:0/2594211199 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f5e140d0be0 con 0x7f5e1c1085d0 2024-06-06T03:36:33.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.302+0000 7f5e21ae5700 1 --2- 172.21.15.40:0/2594211199 >> [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793] conn(0x7f5e080ac670 0x7f5e080aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:33.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.302+0000 7f5e222e6700 1 --2- 172.21.15.40:0/2594211199 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5e080a2cb0 0x7f5e080a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c107c80 tx=0x7f5e1800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:33.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.302+0000 7f5e21ae5700 1 --2- 172.21.15.40:0/2594211199 >> [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793] conn(0x7f5e080ac670 0x7f5e080aeb10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:33.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.303+0000 7f5e137fe700 1 -- 172.21.15.40:0/2594211199 <== osd.0 v2:172.21.15.40:6802/3453526793 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e080af1e0 con 0x7f5e080ac670 2024-06-06T03:36:33.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.320+0000 7f5e2454a700 1 -- 172.21.15.40:0/2594211199 --> [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5e1c062fa0 con 0x7f5e080ac670 2024-06-06T03:36:33.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.320+0000 7f5e137fe700 1 -- 172.21.15.40:0/2594211199 <== osd.0 v2:172.21.15.40:6802/3453526793 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5e1c062fa0 con 0x7f5e080ac670 2024-06-06T03:36:33.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.320+0000 7f5e2454a700 1 -- 172.21.15.40:0/2594211199 >> [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793] conn(0x7f5e080ac670 msgr2=0x7f5e080aeb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:33.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.320+0000 7f5e2454a700 1 --2- 172.21.15.40:0/2594211199 >> [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793] conn(0x7f5e080ac670 0x7f5e080aeb10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.320+0000 7f5e2454a700 1 -- 172.21.15.40:0/2594211199 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5e080a2cb0 msgr2=0x7f5e080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:33.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.320+0000 7f5e2454a700 1 --2- 172.21.15.40:0/2594211199 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5e080a2cb0 0x7f5e080a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c107c80 tx=0x7f5e1800b3c0 comp rx=0 tx=0).stop 2024-06-06T03:36:33.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.320+0000 7f5e2454a700 1 -- 172.21.15.40:0/2594211199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e1c1085d0 msgr2=0x7f5e1c198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:33.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.320+0000 7f5e2454a700 1 --2- 172.21.15.40:0/2594211199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e1c1085d0 0x7f5e1c198ae0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5e14006070 tx=0x7f5e14004d50 comp rx=0 tx=0).stop 2024-06-06T03:36:33.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.320+0000 7f5e22ae7700 1 -- 172.21.15.40:0/2594211199 reap_dead start 2024-06-06T03:36:33.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.320+0000 7f5e2454a700 1 -- 172.21.15.40:0/2594211199 shutdown_connections 2024-06-06T03:36:33.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.320+0000 7f5e2454a700 1 -- 172.21.15.40:0/2594211199 >> 172.21.15.40:0/2594211199 conn(0x7f5e1c079640 msgr2=0x7f5e1c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:33.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.320+0000 7f5e2454a700 1 -- 172.21.15.40:0/2594211199 shutdown_connections 2024-06-06T03:36:33.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:33.321+0000 7f5e2454a700 1 -- 172.21.15.40:0/2594211199 wait complete. 2024-06-06T03:36:33.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:33 smithi104 ceph-mon[36569]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:36:33.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:33 smithi131 ceph-mon[36687]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:36:35.174 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:35 smithi040 ceph-mon[32220]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:36:35.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:35 smithi104 ceph-mon[36569]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:36:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:35 smithi131 ceph-mon[36687]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.265+0000 7f1a947e3700 1 -- 172.21.15.40:0/713893040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a8c107840 msgr2=0x7f1a8c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.265+0000 7f1a947e3700 1 --2- 172.21.15.40:0/713893040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a8c107840 0x7f1a8c10f3c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400b0d0 tx=0x7f1a8400b3e0 comp rx=0 tx=0).stop 2024-06-06T03:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.265+0000 7f1a947e3700 1 -- 172.21.15.40:0/713893040 shutdown_connections 2024-06-06T03:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.265+0000 7f1a947e3700 1 --2- 172.21.15.40:0/713893040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a8c107840 0x7f1a8c10f3c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.265+0000 7f1a947e3700 1 --2- 172.21.15.40:0/713893040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a8c102740 0x7f1a8c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.265+0000 7f1a947e3700 1 --2- 172.21.15.40:0/713893040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a8c1030f0 0x7f1a8c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.265+0000 7f1a947e3700 1 -- 172.21.15.40:0/713893040 >> 172.21.15.40:0/713893040 conn(0x7f1a8c0795f0 msgr2=0x7f1a8c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.265+0000 7f1a947e3700 1 -- 172.21.15.40:0/713893040 shutdown_connections 2024-06-06T03:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.265+0000 7f1a947e3700 1 -- 172.21.15.40:0/713893040 wait complete. 2024-06-06T03:36:36.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a947e3700 1 Processor -- start 2024-06-06T03:36:36.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a947e3700 1 -- start start 2024-06-06T03:36:36.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a947e3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a8c102740 0x7f1a8c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:36.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a947e3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a8c1030f0 0x7f1a8c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:36.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a947e3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a8c107840 0x7f1a8c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:36.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a947e3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c112760 con 0x7f1a8c107840 2024-06-06T03:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a947e3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1125e0 con 0x7f1a8c1030f0 2024-06-06T03:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a947e3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1128e0 con 0x7f1a8c102740 2024-06-06T03:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a92d80700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a8c107840 0x7f1a8c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a92d80700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a8c107840 0x7f1a8c075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50860/0 (socket says 172.21.15.40:50860) 2024-06-06T03:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a92d80700 1 -- 172.21.15.40:0/2031387508 learned_addr learned my addr 172.21.15.40:0/2031387508 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a9257f700 1 --2- 172.21.15.40:0/2031387508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a8c102740 0x7f1a8c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a91d7e700 1 --2- 172.21.15.40:0/2031387508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a8c1030f0 0x7f1a8c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a92d80700 1 -- 172.21.15.40:0/2031387508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a8c102740 msgr2=0x7f1a8c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a92d80700 1 --2- 172.21.15.40:0/2031387508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a8c102740 0x7f1a8c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a92d80700 1 -- 172.21.15.40:0/2031387508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a8c1030f0 msgr2=0x7f1a8c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a92d80700 1 --2- 172.21.15.40:0/2031387508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a8c1030f0 0x7f1a8c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a92d80700 1 -- 172.21.15.40:0/2031387508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c1ab440 con 0x7f1a8c107840 2024-06-06T03:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.266+0000 7f1a92d80700 1 --2- 172.21.15.40:0/2031387508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a8c107840 0x7f1a8c075210 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1a84006070 tx=0x7f1a84004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.267+0000 7f1a837fe700 1 -- 172.21.15.40:0/2031387508 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84021070 con 0x7f1a8c107840 2024-06-06T03:36:36.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.267+0000 7f1a947e3700 1 -- 172.21.15.40:0/2031387508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8c1ab670 con 0x7f1a8c107840 2024-06-06T03:36:36.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.267+0000 7f1a837fe700 1 -- 172.21.15.40:0/2031387508 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a84007d00 con 0x7f1a8c107840 2024-06-06T03:36:36.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.267+0000 7f1a837fe700 1 -- 172.21.15.40:0/2031387508 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84005470 con 0x7f1a8c107840 2024-06-06T03:36:36.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.267+0000 7f1a947e3700 1 -- 172.21.15.40:0/2031387508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a8c1abb80 con 0x7f1a8c107840 2024-06-06T03:36:36.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.268+0000 7f1a947e3700 1 -- 172.21.15.40:0/2031387508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a8c100eb0 con 0x7f1a8c107840 2024-06-06T03:36:36.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.269+0000 7f1a837fe700 1 -- 172.21.15.40:0/2031387508 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1a84024070 con 0x7f1a8c107840 2024-06-06T03:36:36.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.270+0000 7f1a837fe700 1 --2- 172.21.15.40:0/2031387508 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1a780a2cb0 0x7f1a780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:36.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.270+0000 7f1a9257f700 1 --2- 172.21.15.40:0/2031387508 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1a780a2cb0 0x7f1a780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:36.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.270+0000 7f1a837fe700 1 -- 172.21.15.40:0/2031387508 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f1a840d02a0 con 0x7f1a8c107840 2024-06-06T03:36:36.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.270+0000 7f1a837fe700 1 --2- 172.21.15.40:0/2031387508 >> [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252] conn(0x7f1a780ac670 0x7f1a780aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:36.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.270+0000 7f1a837fe700 1 -- 172.21.15.40:0/2031387508 --> [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a8400cbc0 con 0x7f1a780ac670 2024-06-06T03:36:36.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.270+0000 7f1a837fe700 1 -- 172.21.15.40:0/2031387508 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f1a840d0780 con 0x7f1a8c107840 2024-06-06T03:36:36.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.270+0000 7f1a9257f700 1 --2- 172.21.15.40:0/2031387508 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1a780a2cb0 0x7f1a780a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a88005950 tx=0x7f1a8800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:36.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.270+0000 7f1a91d7e700 1 --2- 172.21.15.40:0/2031387508 >> [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252] conn(0x7f1a780ac670 0x7f1a780aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:36.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.271+0000 7f1a91d7e700 1 --2- 172.21.15.40:0/2031387508 >> [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252] conn(0x7f1a780ac670 0x7f1a780aeb10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:36.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.272+0000 7f1a837fe700 1 -- 172.21.15.40:0/2031387508 <== osd.1 v2:172.21.15.104:6800/405854252 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a8400cbc0 con 0x7f1a780ac670 2024-06-06T03:36:36.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.289+0000 7f1a947e3700 1 -- 172.21.15.40:0/2031387508 --> [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a8c103bd0 con 0x7f1a780ac670 2024-06-06T03:36:36.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.289+0000 7f1a837fe700 1 -- 172.21.15.40:0/2031387508 <== osd.1 v2:172.21.15.104:6800/405854252 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1a8c103bd0 con 0x7f1a780ac670 2024-06-06T03:36:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.289+0000 7f1a947e3700 1 -- 172.21.15.40:0/2031387508 >> [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252] conn(0x7f1a780ac670 msgr2=0x7f1a780aeb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.289+0000 7f1a947e3700 1 --2- 172.21.15.40:0/2031387508 >> [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252] conn(0x7f1a780ac670 0x7f1a780aeb10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.289+0000 7f1a947e3700 1 -- 172.21.15.40:0/2031387508 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1a780a2cb0 msgr2=0x7f1a780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.289+0000 7f1a947e3700 1 --2- 172.21.15.40:0/2031387508 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1a780a2cb0 0x7f1a780a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a88005950 tx=0x7f1a8800a300 comp rx=0 tx=0).stop 2024-06-06T03:36:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.289+0000 7f1a947e3700 1 -- 172.21.15.40:0/2031387508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a8c107840 msgr2=0x7f1a8c075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.289+0000 7f1a947e3700 1 --2- 172.21.15.40:0/2031387508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a8c107840 0x7f1a8c075210 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1a84006070 tx=0x7f1a84004e40 comp rx=0 tx=0).stop 2024-06-06T03:36:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.289+0000 7f1a92d80700 1 -- 172.21.15.40:0/2031387508 reap_dead start 2024-06-06T03:36:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.290+0000 7f1a947e3700 1 -- 172.21.15.40:0/2031387508 shutdown_connections 2024-06-06T03:36:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.290+0000 7f1a947e3700 1 -- 172.21.15.40:0/2031387508 >> 172.21.15.40:0/2031387508 conn(0x7f1a8c0795f0 msgr2=0x7f1a8c100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.290+0000 7f1a947e3700 1 -- 172.21.15.40:0/2031387508 shutdown_connections 2024-06-06T03:36:36.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:36.290+0000 7f1a947e3700 1 -- 172.21.15.40:0/2031387508 wait complete. 2024-06-06T03:36:36.929 INFO:teuthology.orchestra.run.smithi040.stdout:107374182473 2024-06-06T03:36:36.929 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-06T03:36:37.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:37 smithi040 ceph-mon[32220]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:36:37.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:37 smithi104 ceph-mon[36569]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:36:37.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:37 smithi131 ceph-mon[36687]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:36:38.111 INFO:teuthology.orchestra.run.smithi040.stdout:219043332147 2024-06-06T03:36:38.111 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-06T03:36:38.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.783+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2000145289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b98101d50 msgr2=0x7f0b9810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:38.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.783+0000 7f0b9ff0e700 1 --2- 172.21.15.40:0/2000145289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b98101d50 0x7f0b9810e240 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000b0a0 tx=0x7f0b9000b3b0 comp rx=0 tx=0).stop 2024-06-06T03:36:38.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.783+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2000145289 shutdown_connections 2024-06-06T03:36:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.783+0000 7f0b9ff0e700 1 --2- 172.21.15.40:0/2000145289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b98101d50 0x7f0b9810e240 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.783+0000 7f0b9ff0e700 1 --2- 172.21.15.40:0/2000145289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b98107400 0x7f0b981077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.783+0000 7f0b9ff0e700 1 --2- 172.21.15.40:0/2000145289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b98101390 0x7f0b98101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.783+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2000145289 >> 172.21.15.40:0/2000145289 conn(0x7f0b98078340 msgr2=0x7f0b98078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.783+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2000145289 shutdown_connections 2024-06-06T03:36:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.783+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2000145289 wait complete. 2024-06-06T03:36:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.784+0000 7f0b9ff0e700 1 Processor -- start 2024-06-06T03:36:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.784+0000 7f0b9ff0e700 1 -- start start 2024-06-06T03:36:38.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.784+0000 7f0b9ff0e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b98101390 0x7f0b98103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:38.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.784+0000 7f0b9ff0e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b98101d50 0x7f0b981035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.784+0000 7f0b9ff0e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b98107400 0x7f0b98103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.784+0000 7f0b9ff0e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0b98111770 con 0x7f0b98101d50 2024-06-06T03:36:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.784+0000 7f0b9ff0e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0b981115f0 con 0x7f0b98107400 2024-06-06T03:36:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.784+0000 7f0b9ff0e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0b981118f0 con 0x7f0b98101390 2024-06-06T03:36:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.784+0000 7f0b9d4a9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b98101d50 0x7f0b981035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b9d4a9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b98101d50 0x7f0b981035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50876/0 (socket says 172.21.15.40:50876) 2024-06-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b9d4a9700 1 -- 172.21.15.40:0/2143383925 learned_addr learned my addr 172.21.15.40:0/2143383925 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b9e4ab700 1 --2- 172.21.15.40:0/2143383925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b98107400 0x7f0b98103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b9dcaa700 1 --2- 172.21.15.40:0/2143383925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b98101390 0x7f0b98103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b9d4a9700 1 -- 172.21.15.40:0/2143383925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b98101390 msgr2=0x7f0b98103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b9d4a9700 1 --2- 172.21.15.40:0/2143383925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b98101390 0x7f0b98103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b9d4a9700 1 -- 172.21.15.40:0/2143383925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b98107400 msgr2=0x7f0b98103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b9d4a9700 1 --2- 172.21.15.40:0/2143383925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b98107400 0x7f0b98103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b9d4a9700 1 -- 172.21.15.40:0/2143383925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b981a7200 con 0x7f0b98101d50 2024-06-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b9d4a9700 1 --2- 172.21.15.40:0/2143383925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b98101d50 0x7f0b981035b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800b530 tx=0x7f0b8800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b8effd700 1 -- 172.21.15.40:0/2143383925 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88015020 con 0x7f0b98101d50 2024-06-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b8effd700 1 -- 172.21.15.40:0/2143383925 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b88011d10 con 0x7f0b98101d50 2024-06-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2143383925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b981a74f0 con 0x7f0b98101d50 2024-06-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b8effd700 1 -- 172.21.15.40:0/2143383925 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8800f2e0 con 0x7f0b98101d50 2024-06-06T03:36:38.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.785+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2143383925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b981a7a80 con 0x7f0b98101d50 2024-06-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.786+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2143383925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b98105480 con 0x7f0b98101d50 2024-06-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.788+0000 7f0b8effd700 1 -- 172.21.15.40:0/2143383925 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0b88018020 con 0x7f0b98101d50 2024-06-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.788+0000 7f0b8effd700 1 --2- 172.21.15.40:0/2143383925 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0b840a2d30 0x7f0b840a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.788+0000 7f0b9dcaa700 1 --2- 172.21.15.40:0/2143383925 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0b840a2d30 0x7f0b840a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.788+0000 7f0b8effd700 1 -- 172.21.15.40:0/2143383925 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f0b880cec60 con 0x7f0b98101d50 2024-06-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.789+0000 7f0b8effd700 1 --2- 172.21.15.40:0/2143383925 >> [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303] conn(0x7f0b840ac6f0 0x7f0b840aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:38.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.789+0000 7f0b8effd700 1 -- 172.21.15.40:0/2143383925 --> [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b880044e0 con 0x7f0b840ac6f0 2024-06-06T03:36:38.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.789+0000 7f0b8effd700 1 -- 172.21.15.40:0/2143383925 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f0b880cf140 con 0x7f0b98101d50 2024-06-06T03:36:38.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.789+0000 7f0b9dcaa700 1 --2- 172.21.15.40:0/2143383925 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0b840a2d30 0x7f0b840a51f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009a10 tx=0x7f0b94009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:38.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.789+0000 7f0b9e4ab700 1 --2- 172.21.15.40:0/2143383925 >> [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303] conn(0x7f0b840ac6f0 0x7f0b840aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:38.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.789+0000 7f0b9e4ab700 1 --2- 172.21.15.40:0/2143383925 >> [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303] conn(0x7f0b840ac6f0 0x7f0b840aeb90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:38.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.790+0000 7f0b8effd700 1 -- 172.21.15.40:0/2143383925 <== osd.5 v2:172.21.15.131:6808/1051933303 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b880044e0 con 0x7f0b840ac6f0 2024-06-06T03:36:38.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.807+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2143383925 --> [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b98102ab0 con 0x7f0b840ac6f0 2024-06-06T03:36:38.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.807+0000 7f0b8effd700 1 -- 172.21.15.40:0/2143383925 <== osd.5 v2:172.21.15.131:6808/1051933303 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0b98102ab0 con 0x7f0b840ac6f0 2024-06-06T03:36:38.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.807+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2143383925 >> [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303] conn(0x7f0b840ac6f0 msgr2=0x7f0b840aeb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:38.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.807+0000 7f0b9ff0e700 1 --2- 172.21.15.40:0/2143383925 >> [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303] conn(0x7f0b840ac6f0 0x7f0b840aeb90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.808+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2143383925 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0b840a2d30 msgr2=0x7f0b840a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.808+0000 7f0b9ff0e700 1 --2- 172.21.15.40:0/2143383925 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0b840a2d30 0x7f0b840a51f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009a10 tx=0x7f0b94009190 comp rx=0 tx=0).stop 2024-06-06T03:36:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.808+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2143383925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b98101d50 msgr2=0x7f0b981035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.808+0000 7f0b9ff0e700 1 --2- 172.21.15.40:0/2143383925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b98101d50 0x7f0b981035b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800b530 tx=0x7f0b8800b840 comp rx=0 tx=0).stop 2024-06-06T03:36:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.808+0000 7f0b9e4ab700 1 -- 172.21.15.40:0/2143383925 reap_dead start 2024-06-06T03:36:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.808+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2143383925 shutdown_connections 2024-06-06T03:36:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.808+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2143383925 >> 172.21.15.40:0/2143383925 conn(0x7f0b98078340 msgr2=0x7f0b981005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.808+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2143383925 shutdown_connections 2024-06-06T03:36:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:38.808+0000 7f0b9ff0e700 1 -- 172.21.15.40:0/2143383925 wait complete. 2024-06-06T03:36:39.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:39 smithi040 ceph-mon[32220]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-06T03:36:39.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:39 smithi104 ceph-mon[36569]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-06T03:36:39.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:39 smithi131 ceph-mon[36687]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-06T03:36:39.705 INFO:teuthology.orchestra.run.smithi040.stdout:163208757312 2024-06-06T03:36:39.705 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-06T03:36:39.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.049+0000 7f84828b8700 1 -- 172.21.15.40:0/1106736624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f847c100e50 msgr2=0x7f847c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:39.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.049+0000 7f84828b8700 1 --2- 172.21.15.40:0/1106736624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f847c100e50 0x7f847c1012d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8470009990 tx=0x7f8470009ca0 comp rx=0 tx=0).stop 2024-06-06T03:36:39.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.049+0000 7f84828b8700 1 -- 172.21.15.40:0/1106736624 shutdown_connections 2024-06-06T03:36:39.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.049+0000 7f84828b8700 1 --2- 172.21.15.40:0/1106736624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f847c107570 0x7f847c107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:39.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.049+0000 7f84828b8700 1 --2- 172.21.15.40:0/1106736624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f847c101810 0x7f847c10f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:39.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.049+0000 7f84828b8700 1 --2- 172.21.15.40:0/1106736624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f847c100e50 0x7f847c1012d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:39.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.049+0000 7f84828b8700 1 -- 172.21.15.40:0/1106736624 >> 172.21.15.40:0/1106736624 conn(0x7f847c0766b0 msgr2=0x7f847c076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:39.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.049+0000 7f84828b8700 1 -- 172.21.15.40:0/1106736624 shutdown_connections 2024-06-06T03:36:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.049+0000 7f84828b8700 1 -- 172.21.15.40:0/1106736624 wait complete. 2024-06-06T03:36:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.049+0000 7f84828b8700 1 Processor -- start 2024-06-06T03:36:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.049+0000 7f84828b8700 1 -- start start 2024-06-06T03:36:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f84828b8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f847c100e50 0x7f847c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f84828b8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f847c101810 0x7f847c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f84828b8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f847c107570 0x7f847c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f84828b8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f847c1127b0 con 0x7f847c100e50 2024-06-06T03:36:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f84828b8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f847c112630 con 0x7f847c101810 2024-06-06T03:36:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f84828b8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f847c112930 con 0x7f847c107570 2024-06-06T03:36:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f847bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f847c100e50 0x7f847c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f847bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f847c100e50 0x7f847c19e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39062/0 (socket says 172.21.15.40:39062) 2024-06-06T03:36:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f847bfff700 1 -- 172.21.15.40:0/1683817952 learned_addr learned my addr 172.21.15.40:0/1683817952 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f847b7fe700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f847c101810 0x7f847c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f8480e55700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f847c107570 0x7f847c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f847bfff700 1 -- 172.21.15.40:0/1683817952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f847c107570 msgr2=0x7f847c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f847bfff700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f847c107570 0x7f847c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f847bfff700 1 -- 172.21.15.40:0/1683817952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f847c101810 msgr2=0x7f847c19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f847bfff700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f847c101810 0x7f847c19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f847bfff700 1 -- 172.21.15.40:0/1683817952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847c1991d0 con 0x7f847c100e50 2024-06-06T03:36:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f8480e55700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f847c107570 0x7f847c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:36:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f847bfff700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f847c100e50 0x7f847c19e8b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f846c00d980 tx=0x7f846c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f84797fa700 1 -- 172.21.15.40:0/1683817952 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c017030 con 0x7f847c100e50 2024-06-06T03:36:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f84797fa700 1 -- 172.21.15.40:0/1683817952 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f846c004500 con 0x7f847c100e50 2024-06-06T03:36:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f84828b8700 1 -- 172.21.15.40:0/1683817952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847c1994c0 con 0x7f847c100e50 2024-06-06T03:36:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.050+0000 7f84797fa700 1 -- 172.21.15.40:0/1683817952 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c003aa0 con 0x7f847c100e50 2024-06-06T03:36:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.051+0000 7f84828b8700 1 -- 172.21.15.40:0/1683817952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f847c1a5780 con 0x7f847c100e50 2024-06-06T03:36:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.051+0000 7f84828b8700 1 -- 172.21.15.40:0/1683817952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f847c062c20 con 0x7f847c100e50 2024-06-06T03:36:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.053+0000 7f84797fa700 1 -- 172.21.15.40:0/1683817952 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f846c01a070 con 0x7f847c100e50 2024-06-06T03:36:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.053+0000 7f84797fa700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f84640a2d30 0x7f84640a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.053+0000 7f847b7fe700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f84640a2d30 0x7f84640a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.053+0000 7f84797fa700 1 -- 172.21.15.40:0/1683817952 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f846c0cf0a0 con 0x7f847c100e50 2024-06-06T03:36:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.054+0000 7f84797fa700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670] conn(0x7f84640ac6f0 0x7f84640aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.054+0000 7f84797fa700 1 -- 172.21.15.40:0/1683817952 --> [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f846c020e50 con 0x7f84640ac6f0 2024-06-06T03:36:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.054+0000 7f84797fa700 1 -- 172.21.15.40:0/1683817952 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f846c0cf580 con 0x7f847c100e50 2024-06-06T03:36:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.054+0000 7f847b7fe700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f84640a2d30 0x7f84640a51f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f847000b5c0 tx=0x7f8470005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.054+0000 7f8480e55700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670] conn(0x7f84640ac6f0 0x7f84640aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.054+0000 7f8480e55700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670] conn(0x7f84640ac6f0 0x7f84640aeb90 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.055+0000 7f84797fa700 1 -- 172.21.15.40:0/1683817952 <== osd.11 v2:172.21.15.131:6824/4128185670 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f846c020e50 con 0x7f84640ac6f0 2024-06-06T03:36:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.072+0000 7f84828b8700 1 -- 172.21.15.40:0/1683817952 --> [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f847c0630a0 con 0x7f84640ac6f0 2024-06-06T03:36:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.072+0000 7f84797fa700 1 -- 172.21.15.40:0/1683817952 <== osd.11 v2:172.21.15.131:6824/4128185670 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f847c0630a0 con 0x7f84640ac6f0 2024-06-06T03:36:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.072+0000 7f84828b8700 1 -- 172.21.15.40:0/1683817952 >> [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670] conn(0x7f84640ac6f0 msgr2=0x7f84640aeb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.073+0000 7f84828b8700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670] conn(0x7f84640ac6f0 0x7f84640aeb90 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.073+0000 7f84828b8700 1 -- 172.21.15.40:0/1683817952 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f84640a2d30 msgr2=0x7f84640a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.073+0000 7f84828b8700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f84640a2d30 0x7f84640a51f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f847000b5c0 tx=0x7f8470005960 comp rx=0 tx=0).stop 2024-06-06T03:36:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.073+0000 7f84828b8700 1 -- 172.21.15.40:0/1683817952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f847c100e50 msgr2=0x7f847c19e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.073+0000 7f84828b8700 1 --2- 172.21.15.40:0/1683817952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f847c100e50 0x7f847c19e8b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f846c00d980 tx=0x7f846c00dc90 comp rx=0 tx=0).stop 2024-06-06T03:36:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.073+0000 7f8480e55700 1 -- 172.21.15.40:0/1683817952 reap_dead start 2024-06-06T03:36:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.073+0000 7f84828b8700 1 -- 172.21.15.40:0/1683817952 shutdown_connections 2024-06-06T03:36:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.073+0000 7f84828b8700 1 -- 172.21.15.40:0/1683817952 >> 172.21.15.40:0/1683817952 conn(0x7f847c0766b0 msgr2=0x7f847c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.073+0000 7f84828b8700 1 -- 172.21.15.40:0/1683817952 shutdown_connections 2024-06-06T03:36:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:39.073+0000 7f84828b8700 1 -- 172.21.15.40:0/1683817952 wait complete. 2024-06-06T03:36:40.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.565+0000 7fa73a18c700 1 -- 172.21.15.40:0/2747288006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa73406a170 msgr2=0x7fa73406a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:40.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.565+0000 7fa73a18c700 1 --2- 172.21.15.40:0/2747288006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa73406a170 0x7fa73406a5d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0099c0 tx=0x7fa71c009cd0 comp rx=0 tx=0).stop 2024-06-06T03:36:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.565+0000 7fa73a18c700 1 -- 172.21.15.40:0/2747288006 shutdown_connections 2024-06-06T03:36:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.565+0000 7fa73a18c700 1 --2- 172.21.15.40:0/2747288006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa73406ab10 0x7fa73410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.565+0000 7fa73a18c700 1 --2- 172.21.15.40:0/2747288006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7341084e0 0x7fa7341088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.565+0000 7fa73a18c700 1 --2- 172.21.15.40:0/2747288006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa73406a170 0x7fa73406a5d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.565+0000 7fa73a18c700 1 -- 172.21.15.40:0/2747288006 >> 172.21.15.40:0/2747288006 conn(0x7fa7340782c0 msgr2=0x7fa7340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.565+0000 7fa73a18c700 1 -- 172.21.15.40:0/2747288006 shutdown_connections 2024-06-06T03:36:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.565+0000 7fa73a18c700 1 -- 172.21.15.40:0/2747288006 wait complete. 2024-06-06T03:36:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.566+0000 7fa73a18c700 1 Processor -- start 2024-06-06T03:36:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.566+0000 7fa73a18c700 1 -- start start 2024-06-06T03:36:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.566+0000 7fa73a18c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa73406a170 0x7fa734101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.566+0000 7fa73a18c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa73406ab10 0x7fa7341019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.566+0000 7fa73a18c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7341084e0 0x7fa734101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.566+0000 7fa73a18c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa734112820 con 0x7fa7341084e0 2024-06-06T03:36:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.566+0000 7fa73a18c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa7341126a0 con 0x7fa73406ab10 2024-06-06T03:36:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.566+0000 7fa73a18c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa7341129a0 con 0x7fa73406a170 2024-06-06T03:36:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.566+0000 7fa733fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7341084e0 0x7fa734101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.566+0000 7fa733fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7341084e0 0x7fa734101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39102/0 (socket says 172.21.15.40:39102) 2024-06-06T03:36:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.566+0000 7fa733fff700 1 -- 172.21.15.40:0/3488634075 learned_addr learned my addr 172.21.15.40:0/3488634075 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.566+0000 7fa7337fe700 1 --2- 172.21.15.40:0/3488634075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa73406a170 0x7fa734101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.566+0000 7fa72affd700 1 --2- 172.21.15.40:0/3488634075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa73406ab10 0x7fa7341019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.567+0000 7fa733fff700 1 -- 172.21.15.40:0/3488634075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa73406a170 msgr2=0x7fa734101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.567+0000 7fa733fff700 1 --2- 172.21.15.40:0/3488634075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa73406a170 0x7fa734101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.567+0000 7fa733fff700 1 -- 172.21.15.40:0/3488634075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa73406ab10 msgr2=0x7fa7341019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.567+0000 7fa733fff700 1 --2- 172.21.15.40:0/3488634075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa73406ab10 0x7fa7341019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.567+0000 7fa733fff700 1 -- 172.21.15.40:0/3488634075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7341a7280 con 0x7fa7341084e0 2024-06-06T03:36:40.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.567+0000 7fa733fff700 1 --2- 172.21.15.40:0/3488634075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7341084e0 0x7fa734101ee0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00b970 tx=0x7fa72c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:40.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.567+0000 7fa7317fa700 1 -- 172.21.15.40:0/3488634075 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c019070 con 0x7fa7341084e0 2024-06-06T03:36:40.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.567+0000 7fa73a18c700 1 -- 172.21.15.40:0/3488634075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7341a7570 con 0x7fa7341084e0 2024-06-06T03:36:40.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.567+0000 7fa7317fa700 1 -- 172.21.15.40:0/3488634075 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa72c00ce10 con 0x7fa7341084e0 2024-06-06T03:36:40.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.567+0000 7fa7317fa700 1 -- 172.21.15.40:0/3488634075 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c007370 con 0x7fa7341084e0 2024-06-06T03:36:40.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.567+0000 7fa73a18c700 1 -- 172.21.15.40:0/3488634075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7341a7a80 con 0x7fa7341084e0 2024-06-06T03:36:40.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.568+0000 7fa73a18c700 1 -- 172.21.15.40:0/3488634075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa734062ba0 con 0x7fa7341084e0 2024-06-06T03:36:40.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.569+0000 7fa7317fa700 1 -- 172.21.15.40:0/3488634075 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa72c01c070 con 0x7fa7341084e0 2024-06-06T03:36:40.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.570+0000 7fa7317fa700 1 --2- 172.21.15.40:0/3488634075 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa7200a2cb0 0x7fa7200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:40.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.570+0000 7fa7337fe700 1 --2- 172.21.15.40:0/3488634075 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa7200a2cb0 0x7fa7200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:40.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.570+0000 7fa7317fa700 1 -- 172.21.15.40:0/3488634075 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fa72c0ceb30 con 0x7fa7341084e0 2024-06-06T03:36:40.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.570+0000 7fa7317fa700 1 --2- 172.21.15.40:0/3488634075 >> [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847] conn(0x7fa7200ac670 0x7fa7200aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:40.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.570+0000 7fa7317fa700 1 -- 172.21.15.40:0/3488634075 --> [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa72c01ce50 con 0x7fa7200ac670 2024-06-06T03:36:40.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.570+0000 7fa7317fa700 1 -- 172.21.15.40:0/3488634075 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa72c0cf010 con 0x7fa7341084e0 2024-06-06T03:36:40.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.570+0000 7fa7337fe700 1 --2- 172.21.15.40:0/3488634075 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa7200a2cb0 0x7fa7200a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa724009a10 tx=0x7fa724009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:40.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.571+0000 7fa72affd700 1 --2- 172.21.15.40:0/3488634075 >> [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847] conn(0x7fa7200ac670 0x7fa7200aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:40.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.571+0000 7fa72affd700 1 --2- 172.21.15.40:0/3488634075 >> [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847] conn(0x7fa7200ac670 0x7fa7200aeb10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:40.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.572+0000 7fa7317fa700 1 -- 172.21.15.40:0/3488634075 <== osd.4 v2:172.21.15.104:6808/1153394847 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa72c01ce50 con 0x7fa7200ac670 2024-06-06T03:36:40.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.589+0000 7fa73a18c700 1 -- 172.21.15.40:0/3488634075 --> [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa734062fa0 con 0x7fa7200ac670 2024-06-06T03:36:40.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.589+0000 7fa7317fa700 1 -- 172.21.15.40:0/3488634075 <== osd.4 v2:172.21.15.104:6808/1153394847 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa734062fa0 con 0x7fa7200ac670 2024-06-06T03:36:40.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.590+0000 7fa73a18c700 1 -- 172.21.15.40:0/3488634075 >> [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847] conn(0x7fa7200ac670 msgr2=0x7fa7200aeb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:40.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.590+0000 7fa73a18c700 1 --2- 172.21.15.40:0/3488634075 >> [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847] conn(0x7fa7200ac670 0x7fa7200aeb10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:40.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.590+0000 7fa73a18c700 1 -- 172.21.15.40:0/3488634075 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa7200a2cb0 msgr2=0x7fa7200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:40.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.590+0000 7fa73a18c700 1 --2- 172.21.15.40:0/3488634075 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa7200a2cb0 0x7fa7200a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa724009a10 tx=0x7fa724009190 comp rx=0 tx=0).stop 2024-06-06T03:36:40.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.590+0000 7fa73a18c700 1 -- 172.21.15.40:0/3488634075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7341084e0 msgr2=0x7fa734101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:40.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.590+0000 7fa73a18c700 1 --2- 172.21.15.40:0/3488634075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7341084e0 0x7fa734101ee0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00b970 tx=0x7fa72c00bc80 comp rx=0 tx=0).stop 2024-06-06T03:36:40.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.590+0000 7fa733fff700 1 -- 172.21.15.40:0/3488634075 reap_dead start 2024-06-06T03:36:40.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.590+0000 7fa73a18c700 1 -- 172.21.15.40:0/3488634075 shutdown_connections 2024-06-06T03:36:40.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.590+0000 7fa73a18c700 1 -- 172.21.15.40:0/3488634075 >> 172.21.15.40:0/3488634075 conn(0x7fa7340782c0 msgr2=0x7fa7340ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:40.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.590+0000 7fa73a18c700 1 -- 172.21.15.40:0/3488634075 shutdown_connections 2024-06-06T03:36:40.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.590+0000 7fa73a18c700 1 -- 172.21.15.40:0/3488634075 wait complete. 2024-06-06T03:36:40.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.452+0000 7f08141f3700 1 -- 172.21.15.40:0/3864897805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c102550 msgr2=0x7f080c1029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:40.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.452+0000 7f08141f3700 1 --2- 172.21.15.40:0/3864897805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c102550 0x7f080c1029d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009990 tx=0x7f07fc009ca0 comp rx=0 tx=0).stop 2024-06-06T03:36:40.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.452+0000 7f08141f3700 1 -- 172.21.15.40:0/3864897805 shutdown_connections 2024-06-06T03:36:40.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.452+0000 7f08141f3700 1 --2- 172.21.15.40:0/3864897805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f080c1084f0 0x7f080c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:40.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.452+0000 7f08141f3700 1 --2- 172.21.15.40:0/3864897805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c102f10 0x7f080c10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:40.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.452+0000 7f08141f3700 1 --2- 172.21.15.40:0/3864897805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c102550 0x7f080c1029d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:40.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.452+0000 7f08141f3700 1 -- 172.21.15.40:0/3864897805 >> 172.21.15.40:0/3864897805 conn(0x7f080c079650 msgr2=0x7f080c079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:40.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.452+0000 7f08141f3700 1 -- 172.21.15.40:0/3864897805 shutdown_connections 2024-06-06T03:36:40.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.452+0000 7f08141f3700 1 -- 172.21.15.40:0/3864897805 wait complete. 2024-06-06T03:36:40.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.452+0000 7f08141f3700 1 Processor -- start 2024-06-06T03:36:40.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.452+0000 7f08141f3700 1 -- start start 2024-06-06T03:36:40.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f08141f3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c102550 0x7f080c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:40.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f08141f3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f080c102f10 0x7f080c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:40.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f08141f3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c1084f0 0x7f080c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:40.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f08141f3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f080c112760 con 0x7f080c1084f0 2024-06-06T03:36:40.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f08141f3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f080c1125e0 con 0x7f080c102550 2024-06-06T03:36:40.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f08141f3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f080c1128e0 con 0x7f080c102f10 2024-06-06T03:36:40.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f0812790700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c1084f0 0x7f080c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:40.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f0812790700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c1084f0 0x7f080c198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39080/0 (socket says 172.21.15.40:39080) 2024-06-06T03:36:40.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f0812790700 1 -- 172.21.15.40:0/232393274 learned_addr learned my addr 172.21.15.40:0/232393274 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:40.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f081178e700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f080c102f10 0x7f080c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:40.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f0811f8f700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c102550 0x7f080c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:40.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f0812790700 1 -- 172.21.15.40:0/232393274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f080c102f10 msgr2=0x7f080c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:40.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f0812790700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f080c102f10 0x7f080c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:40.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f0812790700 1 -- 172.21.15.40:0/232393274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c102550 msgr2=0x7f080c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:40.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f0812790700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c102550 0x7f080c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:40.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f0812790700 1 -- 172.21.15.40:0/232393274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c199180 con 0x7f080c1084f0 2024-06-06T03:36:40.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f0811f8f700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c102550 0x7f080c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:36:40.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f0812790700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c1084f0 0x7f080c198a20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0804007d70 tx=0x7f080400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:40.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f0802ffd700 1 -- 172.21.15.40:0/232393274 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804016020 con 0x7f080c1084f0 2024-06-06T03:36:40.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f08141f3700 1 -- 172.21.15.40:0/232393274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080c199470 con 0x7f080c1084f0 2024-06-06T03:36:40.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f0802ffd700 1 -- 172.21.15.40:0/232393274 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f080400ad40 con 0x7f080c1084f0 2024-06-06T03:36:40.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f0802ffd700 1 -- 172.21.15.40:0/232393274 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08040116f0 con 0x7f080c1084f0 2024-06-06T03:36:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.453+0000 7f08141f3700 1 -- 172.21.15.40:0/232393274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080c1a56e0 con 0x7f080c1084f0 2024-06-06T03:36:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.454+0000 7f08141f3700 1 -- 172.21.15.40:0/232393274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f080c062c20 con 0x7f080c1084f0 2024-06-06T03:36:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.456+0000 7f0802ffd700 1 -- 172.21.15.40:0/232393274 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0804019020 con 0x7f080c1084f0 2024-06-06T03:36:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.456+0000 7f0802ffd700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f07f80a2cb0 0x7f07f80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.456+0000 7f0811f8f700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f07f80a2cb0 0x7f07f80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.456+0000 7f0802ffd700 1 -- 172.21.15.40:0/232393274 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f08040cf620 con 0x7f080c1084f0 2024-06-06T03:36:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.457+0000 7f0802ffd700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141] conn(0x7f07f80ac670 0x7f07f80aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.457+0000 7f0802ffd700 1 -- 172.21.15.40:0/232393274 --> [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0804015e70 con 0x7f07f80ac670 2024-06-06T03:36:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.457+0000 7f0802ffd700 1 -- 172.21.15.40:0/232393274 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f08040cfb00 con 0x7f080c1084f0 2024-06-06T03:36:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.457+0000 7f0811f8f700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f07f80a2cb0 0x7f07f80a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0808005fd0 tx=0x7f0808005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.457+0000 7f081178e700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141] conn(0x7f07f80ac670 0x7f07f80aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.457+0000 7f081178e700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141] conn(0x7f07f80ac670 0x7f07f80aeb10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.458+0000 7f0802ffd700 1 -- 172.21.15.40:0/232393274 <== osd.2 v2:172.21.15.131:6800/3919221141 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0804015e70 con 0x7f07f80ac670 2024-06-06T03:36:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.483+0000 7f08141f3700 1 -- 172.21.15.40:0/232393274 --> [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f080c063050 con 0x7f07f80ac670 2024-06-06T03:36:40.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.483+0000 7f0802ffd700 1 -- 172.21.15.40:0/232393274 <== osd.2 v2:172.21.15.131:6800/3919221141 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f080c063050 con 0x7f07f80ac670 2024-06-06T03:36:40.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.484+0000 7f08141f3700 1 -- 172.21.15.40:0/232393274 >> [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141] conn(0x7f07f80ac670 msgr2=0x7f07f80aeb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:40.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.484+0000 7f08141f3700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141] conn(0x7f07f80ac670 0x7f07f80aeb10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:40.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.484+0000 7f08141f3700 1 -- 172.21.15.40:0/232393274 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f07f80a2cb0 msgr2=0x7f07f80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:40.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.484+0000 7f08141f3700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f07f80a2cb0 0x7f07f80a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0808005fd0 tx=0x7f0808005d60 comp rx=0 tx=0).stop 2024-06-06T03:36:40.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.484+0000 7f08141f3700 1 -- 172.21.15.40:0/232393274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c1084f0 msgr2=0x7f080c198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:40.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.484+0000 7f08141f3700 1 --2- 172.21.15.40:0/232393274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c1084f0 0x7f080c198a20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0804007d70 tx=0x7f080400a340 comp rx=0 tx=0).stop 2024-06-06T03:36:40.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.484+0000 7f0812790700 1 -- 172.21.15.40:0/232393274 reap_dead start 2024-06-06T03:36:40.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.484+0000 7f08141f3700 1 -- 172.21.15.40:0/232393274 shutdown_connections 2024-06-06T03:36:40.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.484+0000 7f08141f3700 1 -- 172.21.15.40:0/232393274 >> 172.21.15.40:0/232393274 conn(0x7f080c079650 msgr2=0x7f080c1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:40.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.484+0000 7f08141f3700 1 -- 172.21.15.40:0/232393274 shutdown_connections 2024-06-06T03:36:40.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:40.484+0000 7f08141f3700 1 -- 172.21.15.40:0/232393274 wait complete. 2024-06-06T03:36:41.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:41 smithi040 ceph-mon[32220]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:36:41.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:41 smithi104 ceph-mon[36569]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:36:41.410 INFO:teuthology.orchestra.run.smithi040.stdout:343597383714 2024-06-06T03:36:41.410 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-06T03:36:41.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:41 smithi131 ceph-mon[36687]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:36:42.088 INFO:teuthology.orchestra.run.smithi040.stdout:34359738457 2024-06-06T03:36:42.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-06T03:36:42.504 INFO:teuthology.orchestra.run.smithi040.stdout:55834574933 2024-06-06T03:36:42.504 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-06T03:36:42.900 INFO:teuthology.orchestra.run.smithi040.stdout:141733920838 2024-06-06T03:36:42.900 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-06T03:36:43.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:43 smithi040 ceph-mon[32220]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T03:36:43.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:43 smithi104 ceph-mon[36569]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T03:36:43.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:43 smithi131 ceph-mon[36687]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T03:36:43.922 INFO:teuthology.orchestra.run.smithi040.stdout:77309411409 2024-06-06T03:36:43.923 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-06T03:36:44.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.417+0000 7f7cffbf8700 1 -- 172.21.15.40:0/668922924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf8102740 msgr2=0x7f7cf8102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:44.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.417+0000 7f7cffbf8700 1 --2- 172.21.15.40:0/668922924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf8102740 0x7f7cf8102b20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40099c0 tx=0x7f7cf4009cd0 comp rx=0 tx=0).stop 2024-06-06T03:36:44.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.417+0000 7f7cffbf8700 1 -- 172.21.15.40:0/668922924 shutdown_connections 2024-06-06T03:36:44.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.417+0000 7f7cffbf8700 1 --2- 172.21.15.40:0/668922924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cf8107840 0x7f7cf810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:44.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.417+0000 7f7cffbf8700 1 --2- 172.21.15.40:0/668922924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf8102740 0x7f7cf8102b20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:44.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.417+0000 7f7cffbf8700 1 --2- 172.21.15.40:0/668922924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cf81030f0 0x7f7cf8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:44.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.417+0000 7f7cffbf8700 1 -- 172.21.15.40:0/668922924 >> 172.21.15.40:0/668922924 conn(0x7f7cf80795f0 msgr2=0x7f7cf8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:44.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.417+0000 7f7cffbf8700 1 -- 172.21.15.40:0/668922924 shutdown_connections 2024-06-06T03:36:44.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.417+0000 7f7cffbf8700 1 -- 172.21.15.40:0/668922924 wait complete. 2024-06-06T03:36:44.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.418+0000 7f7cffbf8700 1 Processor -- start 2024-06-06T03:36:44.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.418+0000 7f7cffbf8700 1 -- start start 2024-06-06T03:36:44.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.418+0000 7f7cffbf8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cf8102740 0x7f7cf8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:44.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.418+0000 7f7cffbf8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cf81030f0 0x7f7cf8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:44.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.418+0000 7f7cffbf8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf8107840 0x7f7cf8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:44.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.418+0000 7f7cffbf8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8112760 con 0x7f7cf8107840 2024-06-06T03:36:44.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.418+0000 7f7cffbf8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf81125e0 con 0x7f7cf8102740 2024-06-06T03:36:44.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.418+0000 7f7cffbf8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf81128e0 con 0x7f7cf81030f0 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.418+0000 7f7cfe195700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf8107840 0x7f7cf8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.418+0000 7f7cfe195700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf8107840 0x7f7cf8075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39128/0 (socket says 172.21.15.40:39128) 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.418+0000 7f7cfe195700 1 -- 172.21.15.40:0/1529471840 learned_addr learned my addr 172.21.15.40:0/1529471840 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.418+0000 7f7cfd994700 1 --2- 172.21.15.40:0/1529471840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cf8102740 0x7f7cf8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.418+0000 7f7cfd193700 1 --2- 172.21.15.40:0/1529471840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cf81030f0 0x7f7cf8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.419+0000 7f7cfe195700 1 -- 172.21.15.40:0/1529471840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cf81030f0 msgr2=0x7f7cf8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.419+0000 7f7cfe195700 1 --2- 172.21.15.40:0/1529471840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cf81030f0 0x7f7cf8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.419+0000 7f7cfe195700 1 -- 172.21.15.40:0/1529471840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cf8102740 msgr2=0x7f7cf8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.419+0000 7f7cfe195700 1 --2- 172.21.15.40:0/1529471840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cf8102740 0x7f7cf8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.419+0000 7f7cfe195700 1 -- 172.21.15.40:0/1529471840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf81a7100 con 0x7f7cf8107840 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.419+0000 7f7cfe195700 1 --2- 172.21.15.40:0/1529471840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf8107840 0x7f7cf8075210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0007ee0 tx=0x7f7cf000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.419+0000 7f7ceeffd700 1 -- 172.21.15.40:0/1529471840 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0004bb0 con 0x7f7cf8107840 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.419+0000 7f7cffbf8700 1 -- 172.21.15.40:0/1529471840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf81a73f0 con 0x7f7cf8107840 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.419+0000 7f7ceeffd700 1 -- 172.21.15.40:0/1529471840 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cf0004d50 con 0x7f7cf8107840 2024-06-06T03:36:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.419+0000 7f7ceeffd700 1 -- 172.21.15.40:0/1529471840 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0005960 con 0x7f7cf8107840 2024-06-06T03:36:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.419+0000 7f7cffbf8700 1 -- 172.21.15.40:0/1529471840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf81a78e0 con 0x7f7cf8107840 2024-06-06T03:36:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.420+0000 7f7cffbf8700 1 -- 172.21.15.40:0/1529471840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7cf8062c20 con 0x7f7cf8107840 2024-06-06T03:36:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.421+0000 7f7ceeffd700 1 -- 172.21.15.40:0/1529471840 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7cf001a020 con 0x7f7cf8107840 2024-06-06T03:36:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.422+0000 7f7ceeffd700 1 --2- 172.21.15.40:0/1529471840 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f7ce40a2cb0 0x7f7ce40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.422+0000 7f7cfd994700 1 --2- 172.21.15.40:0/1529471840 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f7ce40a2cb0 0x7f7ce40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.422+0000 7f7ceeffd700 1 -- 172.21.15.40:0/1529471840 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f7cf00ceef0 con 0x7f7cf8107840 2024-06-06T03:36:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.422+0000 7f7ceeffd700 1 --2- 172.21.15.40:0/1529471840 >> [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011] conn(0x7f7ce40ac670 0x7f7ce40aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.422+0000 7f7ceeffd700 1 -- 172.21.15.40:0/1529471840 --> [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ce40af1e0 con 0x7f7ce40ac670 2024-06-06T03:36:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.422+0000 7f7ceeffd700 1 -- 172.21.15.40:0/1529471840 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f7cf00cf2a0 con 0x7f7cf8107840 2024-06-06T03:36:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.422+0000 7f7cfd193700 1 --2- 172.21.15.40:0/1529471840 >> [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011] conn(0x7f7ce40ac670 0x7f7ce40aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.422+0000 7f7cfd994700 1 --2- 172.21.15.40:0/1529471840 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f7ce40a2cb0 0x7f7ce40a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4000c00 tx=0x7f7cf4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.422+0000 7f7cfd193700 1 --2- 172.21.15.40:0/1529471840 >> [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011] conn(0x7f7ce40ac670 0x7f7ce40aeb10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.423+0000 7f7ceeffd700 1 -- 172.21.15.40:0/1529471840 <== osd.6 v2:172.21.15.40:6818/2670922011 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ce40af1e0 con 0x7f7ce40ac670 2024-06-06T03:36:44.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.440+0000 7f7cffbf8700 1 -- 172.21.15.40:0/1529471840 --> [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7cf80630a0 con 0x7f7ce40ac670 2024-06-06T03:36:44.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.440+0000 7f7ceeffd700 1 -- 172.21.15.40:0/1529471840 <== osd.6 v2:172.21.15.40:6818/2670922011 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7cf80630a0 con 0x7f7ce40ac670 2024-06-06T03:36:44.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.441+0000 7f7cffbf8700 1 -- 172.21.15.40:0/1529471840 >> [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011] conn(0x7f7ce40ac670 msgr2=0x7f7ce40aeb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:44.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.441+0000 7f7cffbf8700 1 --2- 172.21.15.40:0/1529471840 >> [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011] conn(0x7f7ce40ac670 0x7f7ce40aeb10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:44.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.441+0000 7f7cffbf8700 1 -- 172.21.15.40:0/1529471840 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f7ce40a2cb0 msgr2=0x7f7ce40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:44.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.441+0000 7f7cffbf8700 1 --2- 172.21.15.40:0/1529471840 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f7ce40a2cb0 0x7f7ce40a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4000c00 tx=0x7f7cf4005fb0 comp rx=0 tx=0).stop 2024-06-06T03:36:44.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.441+0000 7f7cffbf8700 1 -- 172.21.15.40:0/1529471840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf8107840 msgr2=0x7f7cf8075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:44.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.441+0000 7f7cffbf8700 1 --2- 172.21.15.40:0/1529471840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7cf8107840 0x7f7cf8075210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0007ee0 tx=0x7f7cf000d3b0 comp rx=0 tx=0).stop 2024-06-06T03:36:44.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.441+0000 7f7cfe195700 1 -- 172.21.15.40:0/1529471840 reap_dead start 2024-06-06T03:36:44.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.441+0000 7f7cffbf8700 1 -- 172.21.15.40:0/1529471840 shutdown_connections 2024-06-06T03:36:44.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.441+0000 7f7cffbf8700 1 -- 172.21.15.40:0/1529471840 >> 172.21.15.40:0/1529471840 conn(0x7f7cf80795f0 msgr2=0x7f7cf8108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:44.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.441+0000 7f7cffbf8700 1 -- 172.21.15.40:0/1529471840 shutdown_connections 2024-06-06T03:36:44.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:44.441+0000 7f7cffbf8700 1 -- 172.21.15.40:0/1529471840 wait complete. 2024-06-06T03:36:44.691 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:44 smithi040 ceph-mon[32220]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:36:45.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:44 smithi131 ceph-mon[36687]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:36:45.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:44 smithi104 ceph-mon[36569]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:36:45.218 INFO:teuthology.orchestra.run.smithi040.stdout:197568495676 2024-06-06T03:36:45.219 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-06T03:36:45.317 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:45.317 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:45.328 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:45.331 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:45.645 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:47.028 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:46 smithi040 ceph-mon[32220]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:36:47.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:46 smithi131 ceph-mon[36687]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:36:47.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:46 smithi104 ceph-mon[36569]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:36:49.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:48 smithi131 ceph-mon[36687]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:36:49.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.050+0000 7f2199f84700 1 -- 172.21.15.40:0/753031922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21941073c0 msgr2=0x7f21941077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:49.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.050+0000 7f2199f84700 1 --2- 172.21.15.40:0/753031922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21941073c0 0x7f21941077a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2184009990 tx=0x7f2184009ca0 comp rx=0 tx=0).stop 2024-06-06T03:36:49.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.050+0000 7f2199f84700 1 -- 172.21.15.40:0/753031922 shutdown_connections 2024-06-06T03:36:49.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.050+0000 7f2199f84700 1 --2- 172.21.15.40:0/753031922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2194101d40 0x7f219410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:49.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.050+0000 7f2199f84700 1 --2- 172.21.15.40:0/753031922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21941073c0 0x7f21941077a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:49.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.050+0000 7f2199f84700 1 --2- 172.21.15.40:0/753031922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2194101380 0x7f2194101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.050+0000 7f2199f84700 1 -- 172.21.15.40:0/753031922 >> 172.21.15.40:0/753031922 conn(0x7f2194078330 msgr2=0x7f2194078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.050+0000 7f2199f84700 1 -- 172.21.15.40:0/753031922 shutdown_connections 2024-06-06T03:36:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.050+0000 7f2199f84700 1 -- 172.21.15.40:0/753031922 wait complete. 2024-06-06T03:36:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.051+0000 7f2199f84700 1 Processor -- start 2024-06-06T03:36:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.051+0000 7f2199f84700 1 -- start start 2024-06-06T03:36:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.051+0000 7f2199f84700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2194101380 0x7f219419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.051+0000 7f2199f84700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2194101d40 0x7f219419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.051+0000 7f2199f84700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21941073c0 0x7f21941989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.051+0000 7f2199f84700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2194111630 con 0x7f21941073c0 2024-06-06T03:36:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.051+0000 7f2199f84700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f21941114b0 con 0x7f2194101d40 2024-06-06T03:36:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.051+0000 7f2199f84700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f21941117b0 con 0x7f2194101380 2024-06-06T03:36:49.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2193fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21941073c0 0x7f21941989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2192ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2194101d40 0x7f219419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2193fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21941073c0 0x7f21941989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39140/0 (socket says 172.21.15.40:39140) 2024-06-06T03:36:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2193fff700 1 -- 172.21.15.40:0/527222776 learned_addr learned my addr 172.21.15.40:0/527222776 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f21937fe700 1 --2- 172.21.15.40:0/527222776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2194101380 0x7f219419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2193fff700 1 -- 172.21.15.40:0/527222776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2194101380 msgr2=0x7f219419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2193fff700 1 --2- 172.21.15.40:0/527222776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2194101380 0x7f219419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2193fff700 1 -- 172.21.15.40:0/527222776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2194101d40 msgr2=0x7f219419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2193fff700 1 --2- 172.21.15.40:0/527222776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2194101d40 0x7f219419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2193fff700 1 -- 172.21.15.40:0/527222776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2194199110 con 0x7f21941073c0 2024-06-06T03:36:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2192ffd700 1 --2- 172.21.15.40:0/527222776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2194101d40 0x7f219419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:36:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2193fff700 1 --2- 172.21.15.40:0/527222776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21941073c0 0x7f21941989b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f218c00ef30 tx=0x7f218c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2190ff9700 1 -- 172.21.15.40:0/527222776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c0092e0 con 0x7f21941073c0 2024-06-06T03:36:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2199f84700 1 -- 172.21.15.40:0/527222776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21941993a0 con 0x7f21941073c0 2024-06-06T03:36:49.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2190ff9700 1 -- 172.21.15.40:0/527222776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f218c004bb0 con 0x7f21941073c0 2024-06-06T03:36:49.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2190ff9700 1 -- 172.21.15.40:0/527222776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c004280 con 0x7f21941073c0 2024-06-06T03:36:49.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.052+0000 7f2199f84700 1 -- 172.21.15.40:0/527222776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21941a56c0 con 0x7f21941073c0 2024-06-06T03:36:49.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.053+0000 7f2199f84700 1 -- 172.21.15.40:0/527222776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f21941054c0 con 0x7f21941073c0 2024-06-06T03:36:49.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.055+0000 7f2190ff9700 1 -- 172.21.15.40:0/527222776 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f218c01b030 con 0x7f21941073c0 2024-06-06T03:36:49.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.055+0000 7f2190ff9700 1 --2- 172.21.15.40:0/527222776 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f217c0a2cb0 0x7f217c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:49.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.055+0000 7f21937fe700 1 --2- 172.21.15.40:0/527222776 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f217c0a2cb0 0x7f217c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:49.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.056+0000 7f2190ff9700 1 -- 172.21.15.40:0/527222776 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f218c0cf460 con 0x7f21941073c0 2024-06-06T03:36:49.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.056+0000 7f2190ff9700 1 --2- 172.21.15.40:0/527222776 >> [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794] conn(0x7f217c0ac670 0x7f217c0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.056+0000 7f2190ff9700 1 -- 172.21.15.40:0/527222776 --> [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f218c01fe50 con 0x7f217c0ac670 2024-06-06T03:36:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.056+0000 7f2190ff9700 1 -- 172.21.15.40:0/527222776 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f218c0cf940 con 0x7f21941073c0 2024-06-06T03:36:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.056+0000 7f21937fe700 1 --2- 172.21.15.40:0/527222776 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f217c0a2cb0 0x7f217c0a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f218400b2e0 tx=0x7f218401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.056+0000 7f2192ffd700 1 --2- 172.21.15.40:0/527222776 >> [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794] conn(0x7f217c0ac670 0x7f217c0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.056+0000 7f2192ffd700 1 --2- 172.21.15.40:0/527222776 >> [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794] conn(0x7f217c0ac670 0x7f217c0aeb10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.057+0000 7f2190ff9700 1 -- 172.21.15.40:0/527222776 <== osd.8 v2:172.21.15.131:6816/1364533794 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f218c01fe50 con 0x7f217c0ac670 2024-06-06T03:36:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.074+0000 7f2199f84700 1 -- 172.21.15.40:0/527222776 --> [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2194102a70 con 0x7f217c0ac670 2024-06-06T03:36:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.075+0000 7f2190ff9700 1 -- 172.21.15.40:0/527222776 <== osd.8 v2:172.21.15.131:6816/1364533794 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2194102a70 con 0x7f217c0ac670 2024-06-06T03:36:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.075+0000 7f2199f84700 1 -- 172.21.15.40:0/527222776 >> [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794] conn(0x7f217c0ac670 msgr2=0x7f217c0aeb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.075+0000 7f2199f84700 1 --2- 172.21.15.40:0/527222776 >> [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794] conn(0x7f217c0ac670 0x7f217c0aeb10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.075+0000 7f2199f84700 1 -- 172.21.15.40:0/527222776 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f217c0a2cb0 msgr2=0x7f217c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.075+0000 7f2199f84700 1 --2- 172.21.15.40:0/527222776 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f217c0a2cb0 0x7f217c0a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f218400b2e0 tx=0x7f218401a040 comp rx=0 tx=0).stop 2024-06-06T03:36:49.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.075+0000 7f2199f84700 1 -- 172.21.15.40:0/527222776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21941073c0 msgr2=0x7f21941989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:49.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.075+0000 7f2199f84700 1 --2- 172.21.15.40:0/527222776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21941073c0 0x7f21941989b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f218c00ef30 tx=0x7f218c00c3a0 comp rx=0 tx=0).stop 2024-06-06T03:36:49.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.075+0000 7f2193fff700 1 -- 172.21.15.40:0/527222776 reap_dead start 2024-06-06T03:36:49.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.075+0000 7f2199f84700 1 -- 172.21.15.40:0/527222776 shutdown_connections 2024-06-06T03:36:49.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.075+0000 7f2199f84700 1 -- 172.21.15.40:0/527222776 >> 172.21.15.40:0/527222776 conn(0x7f2194078330 msgr2=0x7f2194100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:49.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.075+0000 7f2199f84700 1 -- 172.21.15.40:0/527222776 shutdown_connections 2024-06-06T03:36:49.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:47.075+0000 7f2199f84700 1 -- 172.21.15.40:0/527222776 wait complete. 2024-06-06T03:36:49.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:48 smithi040 ceph-mon[32220]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:36:49.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:48 smithi104 ceph-mon[36569]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:36:50.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:50 smithi040 ceph-mon[32220]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:36:51.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:50 smithi131 ceph-mon[36687]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:36:51.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:50 smithi104 ceph-mon[36569]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:36:51.209 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:36:52.966 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:52 smithi040 ceph-mon[32220]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:36:53.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:52 smithi131 ceph-mon[36687]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:36:53.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:52 smithi104 ceph-mon[36569]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:36:53.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.954+0000 7f1033706700 1 -- 172.21.15.40:0/4043835643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f102c102ee0 msgr2=0x7f102c10f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:53.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.954+0000 7f1033706700 1 --2- 172.21.15.40:0/4043835643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f102c102ee0 0x7f102c10f390 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f102400b0d0 tx=0x7f102400b3e0 comp rx=0 tx=0).stop 2024-06-06T03:36:53.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.954+0000 7f1033706700 1 -- 172.21.15.40:0/4043835643 shutdown_connections 2024-06-06T03:36:53.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.954+0000 7f1033706700 1 --2- 172.21.15.40:0/4043835643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f102c1085c0 0x7f102c1089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:53.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.954+0000 7f1033706700 1 --2- 172.21.15.40:0/4043835643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f102c102ee0 0x7f102c10f390 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:53.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.954+0000 7f1033706700 1 --2- 172.21.15.40:0/4043835643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f102c102520 0x7f102c1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:53.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.954+0000 7f1033706700 1 -- 172.21.15.40:0/4043835643 >> 172.21.15.40:0/4043835643 conn(0x7f102c079680 msgr2=0x7f102c079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:53.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.954+0000 7f1033706700 1 -- 172.21.15.40:0/4043835643 shutdown_connections 2024-06-06T03:36:53.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.954+0000 7f1033706700 1 -- 172.21.15.40:0/4043835643 wait complete. 2024-06-06T03:36:53.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.954+0000 7f1033706700 1 Processor -- start 2024-06-06T03:36:53.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f1033706700 1 -- start start 2024-06-06T03:36:53.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f1033706700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f102c102520 0x7f102c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:53.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f1033706700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f102c102ee0 0x7f102c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:53.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f1033706700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f102c1085c0 0x7f102c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:53.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f1033706700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f102c112800 con 0x7f102c102520 2024-06-06T03:36:53.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f1033706700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f102c112680 con 0x7f102c1085c0 2024-06-06T03:36:53.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f1033706700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f102c112980 con 0x7f102c102ee0 2024-06-06T03:36:53.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f10314a2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f102c102520 0x7f102c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:53.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f10314a2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f102c102520 0x7f102c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60476/0 (socket says 172.21.15.40:60476) 2024-06-06T03:36:53.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f10314a2700 1 -- 172.21.15.40:0/1012906339 learned_addr learned my addr 172.21.15.40:0/1012906339 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:53.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f1031ca3700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f102c1085c0 0x7f102c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:53.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f1030ca1700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f102c102ee0 0x7f102c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:53.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f10314a2700 1 -- 172.21.15.40:0/1012906339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f102c102ee0 msgr2=0x7f102c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:53.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f10314a2700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f102c102ee0 0x7f102c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:53.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f10314a2700 1 -- 172.21.15.40:0/1012906339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f102c1085c0 msgr2=0x7f102c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:53.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f10314a2700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f102c1085c0 0x7f102c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:53.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.955+0000 7f10314a2700 1 -- 172.21.15.40:0/1012906339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c1a6fe0 con 0x7f102c102520 2024-06-06T03:36:53.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.956+0000 7f1031ca3700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f102c1085c0 0x7f102c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:36:53.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.956+0000 7f10314a2700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f102c102520 0x7f102c074790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f102800c9b0 tx=0x7f102800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:53.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.956+0000 7f10227fc700 1 -- 172.21.15.40:0/1012906339 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028007b00 con 0x7f102c102520 2024-06-06T03:36:53.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.956+0000 7f1033706700 1 -- 172.21.15.40:0/1012906339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102c1a72d0 con 0x7f102c102520 2024-06-06T03:36:53.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.956+0000 7f10227fc700 1 -- 172.21.15.40:0/1012906339 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f102800ce50 con 0x7f102c102520 2024-06-06T03:36:53.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.956+0000 7f10227fc700 1 -- 172.21.15.40:0/1012906339 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028018660 con 0x7f102c102520 2024-06-06T03:36:53.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.956+0000 7f1033706700 1 -- 172.21.15.40:0/1012906339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102c1a7810 con 0x7f102c102520 2024-06-06T03:36:53.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.957+0000 7f1033706700 1 -- 172.21.15.40:0/1012906339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102c069180 con 0x7f102c102520 2024-06-06T03:36:53.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.959+0000 7f10227fc700 1 -- 172.21.15.40:0/1012906339 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f102800f450 con 0x7f102c102520 2024-06-06T03:36:53.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.960+0000 7f10227fc700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f10180a2cb0 0x7f10180a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:53.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.960+0000 7f1030ca1700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f10180a2cb0 0x7f10180a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:53.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.960+0000 7f10227fc700 1 -- 172.21.15.40:0/1012906339 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f10280cf1e0 con 0x7f102c102520 2024-06-06T03:36:53.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.961+0000 7f1030ca1700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f10180a2cb0 0x7f10180a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f102c079aa0 tx=0x7f101c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:53.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:53.963+0000 7f10227fc700 1 -- 172.21.15.40:0/1012906339 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1028014030 con 0x7f102c102520 2024-06-06T03:36:54.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:54.181+0000 7f1033706700 1 -- 172.21.15.40:0/1012906339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f102c075cc0 con 0x7f102c102520 2024-06-06T03:36:54.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:54.181+0000 7f10227fc700 1 -- 172.21.15.40:0/1012906339 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f102c075cc0 con 0x7f102c102520 2024-06-06T03:36:54.181 INFO:teuthology.orchestra.run.smithi040.stdout:77309411412 2024-06-06T03:36:54.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:54.184+0000 7f1033706700 1 -- 172.21.15.40:0/1012906339 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f10180a2cb0 msgr2=0x7f10180a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:54.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:54.184+0000 7f1033706700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f10180a2cb0 0x7f10180a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f102c079aa0 tx=0x7f101c00a380 comp rx=0 tx=0).stop 2024-06-06T03:36:54.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:54.184+0000 7f1033706700 1 -- 172.21.15.40:0/1012906339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f102c102520 msgr2=0x7f102c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:54.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:54.184+0000 7f1033706700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f102c102520 0x7f102c074790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f102800c9b0 tx=0x7f102800ccc0 comp rx=0 tx=0).stop 2024-06-06T03:36:54.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:54.184+0000 7f1033706700 1 -- 172.21.15.40:0/1012906339 shutdown_connections 2024-06-06T03:36:54.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:54.184+0000 7f1033706700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f10180a2cb0 0x7f10180a5170 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:54.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:54.184+0000 7f1033706700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f102c1085c0 0x7f102c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:54.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:54.184+0000 7f1033706700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f102c102520 0x7f102c074790 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:54.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:54.184+0000 7f1033706700 1 --2- 172.21.15.40:0/1012906339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f102c102ee0 0x7f102c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:54.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:54.184+0000 7f1033706700 1 -- 172.21.15.40:0/1012906339 >> 172.21.15.40:0/1012906339 conn(0x7f102c079680 msgr2=0x7f102c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:54.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:54.184+0000 7f1033706700 1 -- 172.21.15.40:0/1012906339 shutdown_connections 2024-06-06T03:36:54.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:54.184+0000 7f1033706700 1 -- 172.21.15.40:0/1012906339 wait complete. 2024-06-06T03:36:55.011 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:54 smithi040 ceph-mon[32220]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:36:55.011 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:54 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1012906339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T03:36:55.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:54 smithi131 ceph-mon[36687]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:36:55.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:54 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1012906339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T03:36:55.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:54 smithi104 ceph-mon[36569]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:36:55.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:54 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1012906339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T03:36:56.254 INFO:teuthology.orchestra.run.smithi040.stdout:253403070515 2024-06-06T03:36:56.255 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-06T03:36:56.805 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:56 smithi040 ceph-mon[32220]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:36:57.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:56 smithi131 ceph-mon[36687]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:36:57.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:56 smithi104 ceph-mon[36569]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:36:57.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.757+0000 7f8546745700 1 -- 172.21.15.40:0/3397614744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8540107840 msgr2=0x7f854010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:57.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.757+0000 7f8546745700 1 --2- 172.21.15.40:0/3397614744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8540107840 0x7f854010f3c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f853800b0d0 tx=0x7f853800b3e0 comp rx=0 tx=0).stop 2024-06-06T03:36:57.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.757+0000 7f8546745700 1 -- 172.21.15.40:0/3397614744 shutdown_connections 2024-06-06T03:36:57.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.757+0000 7f8546745700 1 --2- 172.21.15.40:0/3397614744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8540107840 0x7f854010f3c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:57.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.757+0000 7f8546745700 1 --2- 172.21.15.40:0/3397614744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8540102740 0x7f8540102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:57.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.757+0000 7f8546745700 1 --2- 172.21.15.40:0/3397614744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85401030f0 0x7f8540107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:57.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.757+0000 7f8546745700 1 -- 172.21.15.40:0/3397614744 >> 172.21.15.40:0/3397614744 conn(0x7f85400795f0 msgr2=0x7f8540079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:57.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.757+0000 7f8546745700 1 -- 172.21.15.40:0/3397614744 shutdown_connections 2024-06-06T03:36:57.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.757+0000 7f8546745700 1 -- 172.21.15.40:0/3397614744 wait complete. 2024-06-06T03:36:57.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.758+0000 7f8546745700 1 Processor -- start 2024-06-06T03:36:57.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.758+0000 7f8546745700 1 -- start start 2024-06-06T03:36:57.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.758+0000 7f8546745700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8540102740 0x7f854019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:57.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.758+0000 7f8546745700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85401030f0 0x7f854019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:57.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.758+0000 7f8546745700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8540107840 0x7f85401989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:57.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.758+0000 7f8546745700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85401127b0 con 0x7f8540107840 2024-06-06T03:36:57.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.758+0000 7f8546745700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8540112630 con 0x7f8540102740 2024-06-06T03:36:57.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.758+0000 7f8546745700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8540112930 con 0x7f85401030f0 2024-06-06T03:36:57.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.758+0000 7f8544ce2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8540107840 0x7f85401989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:57.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.758+0000 7f8544ce2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8540107840 0x7f85401989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60492/0 (socket says 172.21.15.40:60492) 2024-06-06T03:36:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.758+0000 7f8544ce2700 1 -- 172.21.15.40:0/2769980659 learned_addr learned my addr 172.21.15.40:0/2769980659 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.759+0000 7f853f7fe700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85401030f0 0x7f854019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.759+0000 7f853ffff700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8540102740 0x7f854019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.759+0000 7f8544ce2700 1 -- 172.21.15.40:0/2769980659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85401030f0 msgr2=0x7f854019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.759+0000 7f8544ce2700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85401030f0 0x7f854019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.759+0000 7f8544ce2700 1 -- 172.21.15.40:0/2769980659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8540102740 msgr2=0x7f854019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.759+0000 7f8544ce2700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8540102740 0x7f854019e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.759+0000 7f8544ce2700 1 -- 172.21.15.40:0/2769980659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8540199110 con 0x7f8540107840 2024-06-06T03:36:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.759+0000 7f853ffff700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8540102740 0x7f854019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:36:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.759+0000 7f8544ce2700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8540107840 0x7f85401989b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8538006070 tx=0x7f8538004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.759+0000 7f853d7fa700 1 -- 172.21.15.40:0/2769980659 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538021070 con 0x7f8540107840 2024-06-06T03:36:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.759+0000 7f8546745700 1 -- 172.21.15.40:0/2769980659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85401993a0 con 0x7f8540107840 2024-06-06T03:36:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.759+0000 7f853d7fa700 1 -- 172.21.15.40:0/2769980659 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8538003c00 con 0x7f8540107840 2024-06-06T03:36:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.759+0000 7f853d7fa700 1 -- 172.21.15.40:0/2769980659 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853801d450 con 0x7f8540107840 2024-06-06T03:36:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.759+0000 7f8546745700 1 -- 172.21.15.40:0/2769980659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85401a5670 con 0x7f8540107840 2024-06-06T03:36:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.760+0000 7f8546745700 1 -- 172.21.15.40:0/2769980659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f852c005330 con 0x7f8540107840 2024-06-06T03:36:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.761+0000 7f853d7fa700 1 -- 172.21.15.40:0/2769980659 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8538004020 con 0x7f8540107840 2024-06-06T03:36:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.762+0000 7f853d7fa700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f85280a2cb0 0x7f85280a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.762+0000 7f853ffff700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f85280a2cb0 0x7f85280a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.762+0000 7f853d7fa700 1 -- 172.21.15.40:0/2769980659 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f853800f020 con 0x7f8540107840 2024-06-06T03:36:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.762+0000 7f853ffff700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f85280a2cb0 0x7f85280a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8530005fd0 tx=0x7f8530005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:57.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.766+0000 7f853d7fa700 1 -- 172.21.15.40:0/2769980659 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f853802a020 con 0x7f8540107840 2024-06-06T03:36:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.984+0000 7f8546745700 1 -- 172.21.15.40:0/2769980659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f852c0058f0 con 0x7f8540107840 2024-06-06T03:36:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.984+0000 7f853d7fa700 1 -- 172.21.15.40:0/2769980659 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f853809d480 con 0x7f8540107840 2024-06-06T03:36:57.984 INFO:teuthology.orchestra.run.smithi040.stdout:163208757316 2024-06-06T03:36:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.986+0000 7f8546745700 1 -- 172.21.15.40:0/2769980659 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f85280a2cb0 msgr2=0x7f85280a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.986+0000 7f8546745700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f85280a2cb0 0x7f85280a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8530005fd0 tx=0x7f8530005f00 comp rx=0 tx=0).stop 2024-06-06T03:36:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.986+0000 7f8546745700 1 -- 172.21.15.40:0/2769980659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8540107840 msgr2=0x7f85401989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.986+0000 7f8546745700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8540107840 0x7f85401989b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8538006070 tx=0x7f8538004d50 comp rx=0 tx=0).stop 2024-06-06T03:36:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.986+0000 7f8546745700 1 -- 172.21.15.40:0/2769980659 shutdown_connections 2024-06-06T03:36:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.986+0000 7f8546745700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f85280a2cb0 0x7f85280a5170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.986+0000 7f8546745700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8540107840 0x7f85401989b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.986+0000 7f8546745700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8540102740 0x7f854019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.986+0000 7f8546745700 1 --2- 172.21.15.40:0/2769980659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85401030f0 0x7f854019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.986+0000 7f8546745700 1 -- 172.21.15.40:0/2769980659 >> 172.21.15.40:0/2769980659 conn(0x7f85400795f0 msgr2=0x7f8540100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.987+0000 7f8546745700 1 -- 172.21.15.40:0/2769980659 shutdown_connections 2024-06-06T03:36:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:57.987+0000 7f8546745700 1 -- 172.21.15.40:0/2769980659 wait complete. 2024-06-06T03:36:58.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.370+0000 7f17be8d3700 1 -- 172.21.15.40:0/2504772818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b8101390 msgr2=0x7f17b8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.370+0000 7f17be8d3700 1 --2- 172.21.15.40:0/2504772818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b8101390 0x7f17b8101810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009990 tx=0x7f17a0009ca0 comp rx=0 tx=0).stop 2024-06-06T03:36:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.370+0000 7f17be8d3700 1 -- 172.21.15.40:0/2504772818 shutdown_connections 2024-06-06T03:36:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.370+0000 7f17be8d3700 1 --2- 172.21.15.40:0/2504772818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b8101d50 0x7f17b810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.370+0000 7f17be8d3700 1 --2- 172.21.15.40:0/2504772818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b8107400 0x7f17b81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.370+0000 7f17be8d3700 1 --2- 172.21.15.40:0/2504772818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b8101390 0x7f17b8101810 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.370+0000 7f17be8d3700 1 -- 172.21.15.40:0/2504772818 >> 172.21.15.40:0/2504772818 conn(0x7f17b8078340 msgr2=0x7f17b8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.370+0000 7f17be8d3700 1 -- 172.21.15.40:0/2504772818 shutdown_connections 2024-06-06T03:36:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.370+0000 7f17be8d3700 1 -- 172.21.15.40:0/2504772818 wait complete. 2024-06-06T03:36:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.371+0000 7f17be8d3700 1 Processor -- start 2024-06-06T03:36:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.371+0000 7f17be8d3700 1 -- start start 2024-06-06T03:36:58.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17be8d3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b8101390 0x7f17b8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17be8d3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b8101d50 0x7f17b819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17be8d3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b8107400 0x7f17b81a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17be8d3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81116a0 con 0x7f17b8101d50 2024-06-06T03:36:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17be8d3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f17b8111520 con 0x7f17b8101390 2024-06-06T03:36:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17be8d3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f17b8111820 con 0x7f17b8107400 2024-06-06T03:36:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17b77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b8101d50 0x7f17b819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17b77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b8101d50 0x7f17b819a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60510/0 (socket says 172.21.15.40:60510) 2024-06-06T03:36:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17b77fe700 1 -- 172.21.15.40:0/3418314683 learned_addr learned my addr 172.21.15.40:0/3418314683 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:36:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17b7fff700 1 --2- 172.21.15.40:0/3418314683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b8101390 0x7f17b8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17bce70700 1 --2- 172.21.15.40:0/3418314683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b8107400 0x7f17b81a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17b77fe700 1 -- 172.21.15.40:0/3418314683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b8107400 msgr2=0x7f17b81a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17b77fe700 1 --2- 172.21.15.40:0/3418314683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b8107400 0x7f17b81a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17b77fe700 1 -- 172.21.15.40:0/3418314683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b8101390 msgr2=0x7f17b8199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17b77fe700 1 --2- 172.21.15.40:0/3418314683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b8101390 0x7f17b8199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17b77fe700 1 -- 172.21.15.40:0/3418314683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b81a50a0 con 0x7f17b8101d50 2024-06-06T03:36:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17b77fe700 1 --2- 172.21.15.40:0/3418314683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b8101d50 0x7f17b819a020 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f17a0000c00 tx=0x7f17a00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17b57fa700 1 -- 172.21.15.40:0/3418314683 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0027070 con 0x7f17b8101d50 2024-06-06T03:36:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17be8d3700 1 -- 172.21.15.40:0/3418314683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b81a5270 con 0x7f17b8101d50 2024-06-06T03:36:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17b57fa700 1 -- 172.21.15.40:0/3418314683 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a0003780 con 0x7f17b8101d50 2024-06-06T03:36:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.372+0000 7f17b57fa700 1 -- 172.21.15.40:0/3418314683 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0022dd0 con 0x7f17b8101d50 2024-06-06T03:36:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.373+0000 7f17be8d3700 1 -- 172.21.15.40:0/3418314683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b81a5720 con 0x7f17b8101d50 2024-06-06T03:36:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.373+0000 7f17be8d3700 1 -- 172.21.15.40:0/3418314683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b8069180 con 0x7f17b8101d50 2024-06-06T03:36:58.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.375+0000 7f17b57fa700 1 -- 172.21.15.40:0/3418314683 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f17a002a070 con 0x7f17b8101d50 2024-06-06T03:36:58.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.375+0000 7f17b57fa700 1 --2- 172.21.15.40:0/3418314683 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f17a40a2d30 0x7f17a40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:36:58.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.376+0000 7f17b7fff700 1 --2- 172.21.15.40:0/3418314683 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f17a40a2d30 0x7f17a40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:36:58.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.376+0000 7f17b57fa700 1 -- 172.21.15.40:0/3418314683 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f17a00d00a0 con 0x7f17b8101d50 2024-06-06T03:36:58.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.376+0000 7f17b7fff700 1 --2- 172.21.15.40:0/3418314683 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f17a40a2d30 0x7f17a40a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17a8005950 tx=0x7f17a800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:36:58.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.379+0000 7f17b57fa700 1 -- 172.21.15.40:0/3418314683 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f17a0013070 con 0x7f17b8101d50 2024-06-06T03:36:58.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.587+0000 7f17be8d3700 1 -- 172.21.15.40:0/3418314683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f17b802dca0 con 0x7f17b8101d50 2024-06-06T03:36:58.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.587+0000 7f17b57fa700 1 -- 172.21.15.40:0/3418314683 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f17a009d0e0 con 0x7f17b8101d50 2024-06-06T03:36:58.587 INFO:teuthology.orchestra.run.smithi040.stdout:309237645355 2024-06-06T03:36:58.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.589+0000 7f17be8d3700 1 -- 172.21.15.40:0/3418314683 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f17a40a2d30 msgr2=0x7f17a40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:58.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.589+0000 7f17be8d3700 1 --2- 172.21.15.40:0/3418314683 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f17a40a2d30 0x7f17a40a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17a8005950 tx=0x7f17a800b410 comp rx=0 tx=0).stop 2024-06-06T03:36:58.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.589+0000 7f17be8d3700 1 -- 172.21.15.40:0/3418314683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b8101d50 msgr2=0x7f17b819a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:36:58.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.589+0000 7f17be8d3700 1 --2- 172.21.15.40:0/3418314683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b8101d50 0x7f17b819a020 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f17a0000c00 tx=0x7f17a00047b0 comp rx=0 tx=0).stop 2024-06-06T03:36:58.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.589+0000 7f17be8d3700 1 -- 172.21.15.40:0/3418314683 shutdown_connections 2024-06-06T03:36:58.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.589+0000 7f17be8d3700 1 --2- 172.21.15.40:0/3418314683 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f17a40a2d30 0x7f17a40a51f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:58.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.589+0000 7f17be8d3700 1 --2- 172.21.15.40:0/3418314683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b8107400 0x7f17b81a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:58.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.589+0000 7f17be8d3700 1 --2- 172.21.15.40:0/3418314683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b8101390 0x7f17b8199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:58.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.589+0000 7f17be8d3700 1 --2- 172.21.15.40:0/3418314683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b8101d50 0x7f17b819a020 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:36:58.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.589+0000 7f17be8d3700 1 -- 172.21.15.40:0/3418314683 >> 172.21.15.40:0/3418314683 conn(0x7f17b8078340 msgr2=0x7f17b80fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:36:58.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.590+0000 7f17be8d3700 1 -- 172.21.15.40:0/3418314683 shutdown_connections 2024-06-06T03:36:58.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:36:58.590+0000 7f17be8d3700 1 -- 172.21.15.40:0/3418314683 wait complete. 2024-06-06T03:36:58.841 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:58 smithi040 ceph-mon[32220]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:36:58.841 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:58 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2769980659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T03:36:58.841 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:36:58 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3418314683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T03:36:59.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:58 smithi131 ceph-mon[36687]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:36:59.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:58 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2769980659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T03:36:59.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:36:58 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3418314683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T03:36:59.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:58 smithi104 ceph-mon[36569]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:36:59.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:58 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2769980659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T03:36:59.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:36:58 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3418314683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T03:36:59.376 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411409 got 77309411412 for osd.2 2024-06-06T03:36:59.377 DEBUG:teuthology.parallel:result is None 2024-06-06T03:36:59.954 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757312 got 163208757316 for osd.5 2024-06-06T03:36:59.954 DEBUG:teuthology.parallel:result is None 2024-06-06T03:37:00.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.162+0000 7fbb2d629700 1 -- 172.21.15.40:0/3771470879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb28107840 msgr2=0x7fbb2810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.162+0000 7fbb2d629700 1 --2- 172.21.15.40:0/3771470879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb28107840 0x7fbb2810f3c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000b0d0 tx=0x7fbb2000b3e0 comp rx=0 tx=0).stop 2024-06-06T03:37:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.162+0000 7fbb2d629700 1 -- 172.21.15.40:0/3771470879 shutdown_connections 2024-06-06T03:37:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.162+0000 7fbb2d629700 1 --2- 172.21.15.40:0/3771470879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb28102740 0x7fbb28102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.162+0000 7fbb2d629700 1 --2- 172.21.15.40:0/3771470879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb28107840 0x7fbb2810f3c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.162+0000 7fbb2d629700 1 --2- 172.21.15.40:0/3771470879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb281030f0 0x7fbb28107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.162+0000 7fbb2d629700 1 -- 172.21.15.40:0/3771470879 >> 172.21.15.40:0/3771470879 conn(0x7fbb280795f0 msgr2=0x7fbb28079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.162+0000 7fbb2d629700 1 -- 172.21.15.40:0/3771470879 shutdown_connections 2024-06-06T03:37:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.162+0000 7fbb2d629700 1 -- 172.21.15.40:0/3771470879 wait complete. 2024-06-06T03:37:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.162+0000 7fbb2d629700 1 Processor -- start 2024-06-06T03:37:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb2d629700 1 -- start start 2024-06-06T03:37:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb2d629700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb28102740 0x7fbb28074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb2d629700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb281030f0 0x7fbb28074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb2d629700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb28107840 0x7fbb28075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb2d629700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbb281127e0 con 0x7fbb28102740 2024-06-06T03:37:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb2d629700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbb28112660 con 0x7fbb281030f0 2024-06-06T03:37:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb2d629700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbb28112960 con 0x7fbb28107840 2024-06-06T03:37:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb26ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb28102740 0x7fbb28074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb26ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb28102740 0x7fbb28074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39694/0 (socket says 172.21.15.40:39694) 2024-06-06T03:37:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb26ffd700 1 -- 172.21.15.40:0/257541184 learned_addr learned my addr 172.21.15.40:0/257541184 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb277fe700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb28107840 0x7fbb28075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb267fc700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb281030f0 0x7fbb28074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb26ffd700 1 -- 172.21.15.40:0/257541184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb28107840 msgr2=0x7fbb28075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb26ffd700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb28107840 0x7fbb28075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb26ffd700 1 -- 172.21.15.40:0/257541184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb281030f0 msgr2=0x7fbb28074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb26ffd700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb281030f0 0x7fbb28074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb26ffd700 1 -- 172.21.15.40:0/257541184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb281a7060 con 0x7fbb28102740 2024-06-06T03:37:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.163+0000 7fbb267fc700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb281030f0 0x7fbb28074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:37:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.164+0000 7fbb26ffd700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb28102740 0x7fbb28074790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800b530 tx=0x7fbb1800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.164+0000 7fbb0ffff700 1 -- 172.21.15.40:0/257541184 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18015020 con 0x7fbb28102740 2024-06-06T03:37:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.164+0000 7fbb0ffff700 1 -- 172.21.15.40:0/257541184 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb18011d10 con 0x7fbb28102740 2024-06-06T03:37:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.164+0000 7fbb2d629700 1 -- 172.21.15.40:0/257541184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb281a7350 con 0x7fbb28102740 2024-06-06T03:37:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.164+0000 7fbb0ffff700 1 -- 172.21.15.40:0/257541184 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800f2e0 con 0x7fbb28102740 2024-06-06T03:37:00.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.164+0000 7fbb2d629700 1 -- 172.21.15.40:0/257541184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb281a7890 con 0x7fbb28102740 2024-06-06T03:37:00.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.165+0000 7fbb2d629700 1 -- 172.21.15.40:0/257541184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb28069180 con 0x7fbb28102740 2024-06-06T03:37:00.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.166+0000 7fbb0ffff700 1 -- 172.21.15.40:0/257541184 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbb18018020 con 0x7fbb28102740 2024-06-06T03:37:00.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.167+0000 7fbb0ffff700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbb100a2cb0 0x7fbb100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:00.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.167+0000 7fbb267fc700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbb100a2cb0 0x7fbb100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:00.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.167+0000 7fbb0ffff700 1 -- 172.21.15.40:0/257541184 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fbb180cec20 con 0x7fbb28102740 2024-06-06T03:37:00.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.168+0000 7fbb267fc700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbb100a2cb0 0x7fbb100a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c005fd0 tx=0x7fbb1c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:00.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.173+0000 7fbb0ffff700 1 -- 172.21.15.40:0/257541184 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbb1801f020 con 0x7fbb28102740 2024-06-06T03:37:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.386+0000 7fbb2d629700 1 -- 172.21.15.40:0/257541184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fbb28075cc0 con 0x7fbb28102740 2024-06-06T03:37:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.386+0000 7fbb0ffff700 1 -- 172.21.15.40:0/257541184 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbb28075cc0 con 0x7fbb28102740 2024-06-06T03:37:00.386 INFO:teuthology.orchestra.run.smithi040.stdout:107374182479 2024-06-06T03:37:00.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.388+0000 7fbb2d629700 1 -- 172.21.15.40:0/257541184 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbb100a2cb0 msgr2=0x7fbb100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:00.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.388+0000 7fbb2d629700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbb100a2cb0 0x7fbb100a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c005fd0 tx=0x7fbb1c005e30 comp rx=0 tx=0).stop 2024-06-06T03:37:00.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.388+0000 7fbb2d629700 1 -- 172.21.15.40:0/257541184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb28102740 msgr2=0x7fbb28074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:00.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.388+0000 7fbb2d629700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb28102740 0x7fbb28074790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800b530 tx=0x7fbb1800b840 comp rx=0 tx=0).stop 2024-06-06T03:37:00.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.388+0000 7fbb2d629700 1 -- 172.21.15.40:0/257541184 shutdown_connections 2024-06-06T03:37:00.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.388+0000 7fbb2d629700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbb100a2cb0 0x7fbb100a5170 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:00.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.388+0000 7fbb2d629700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb28107840 0x7fbb28075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:00.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.388+0000 7fbb2d629700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb28102740 0x7fbb28074790 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:00.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.388+0000 7fbb2d629700 1 --2- 172.21.15.40:0/257541184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb281030f0 0x7fbb28074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:00.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.388+0000 7fbb2d629700 1 -- 172.21.15.40:0/257541184 >> 172.21.15.40:0/257541184 conn(0x7fbb280795f0 msgr2=0x7fbb281003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:00.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.388+0000 7fbb2d629700 1 -- 172.21.15.40:0/257541184 shutdown_connections 2024-06-06T03:37:00.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:00.388+0000 7fbb2d629700 1 -- 172.21.15.40:0/257541184 wait complete. 2024-06-06T03:37:00.889 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645348 got 309237645355 for osd.10 2024-06-06T03:37:00.889 DEBUG:teuthology.parallel:result is None 2024-06-06T03:37:01.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:00 smithi131 ceph-mon[36687]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:37:01.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:00 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/257541184' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T03:37:01.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:00 smithi040 ceph-mon[32220]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:37:01.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:00 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/257541184' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T03:37:01.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:00 smithi104 ceph-mon[36569]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:37:01.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:00 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/257541184' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T03:37:01.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.253+0000 7f6ea99c7700 1 -- 172.21.15.40:0/2449083604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea4106b30 msgr2=0x7f6ea4106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:01.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.253+0000 7f6ea99c7700 1 --2- 172.21.15.40:0/2449083604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea4106b30 0x7f6ea4106f10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6e940099c0 tx=0x7f6e94009cd0 comp rx=0 tx=0).stop 2024-06-06T03:37:01.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.253+0000 7f6ea99c7700 1 -- 172.21.15.40:0/2449083604 shutdown_connections 2024-06-06T03:37:01.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.253+0000 7f6ea99c7700 1 --2- 172.21.15.40:0/2449083604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ea4101860 0x7f6ea410d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:01.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.253+0000 7f6ea99c7700 1 --2- 172.21.15.40:0/2449083604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea4106b30 0x7f6ea4106f10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:01.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.253+0000 7f6ea99c7700 1 --2- 172.21.15.40:0/2449083604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ea4100ea0 0x7f6ea4101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:01.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.253+0000 7f6ea99c7700 1 -- 172.21.15.40:0/2449083604 >> 172.21.15.40:0/2449083604 conn(0x7f6ea4077590 msgr2=0x7f6ea40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:01.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.253+0000 7f6ea99c7700 1 -- 172.21.15.40:0/2449083604 shutdown_connections 2024-06-06T03:37:01.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.253+0000 7f6ea99c7700 1 -- 172.21.15.40:0/2449083604 wait complete. 2024-06-06T03:37:01.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.254+0000 7f6ea99c7700 1 Processor -- start 2024-06-06T03:37:01.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.254+0000 7f6ea99c7700 1 -- start start 2024-06-06T03:37:01.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea99c7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ea4100ea0 0x7f6ea419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:01.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea99c7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ea4101860 0x7f6ea419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:01.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea99c7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea4106b30 0x7f6ea4196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:01.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea99c7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41105d0 con 0x7f6ea4106b30 2024-06-06T03:37:01.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea99c7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea4110450 con 0x7f6ea4100ea0 2024-06-06T03:37:01.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea99c7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea4110750 con 0x7f6ea4101860 2024-06-06T03:37:01.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea4106b30 0x7f6ea4196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:01.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea4106b30 0x7f6ea4196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39712/0 (socket says 172.21.15.40:39712) 2024-06-06T03:37:01.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea37fe700 1 -- 172.21.15.40:0/1274119262 learned_addr learned my addr 172.21.15.40:0/1274119262 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:01.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6e9bfff700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ea4101860 0x7f6ea419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:01.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea2ffd700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ea4100ea0 0x7f6ea419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:01.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea37fe700 1 -- 172.21.15.40:0/1274119262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ea4101860 msgr2=0x7f6ea419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:01.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea37fe700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ea4101860 0x7f6ea419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:01.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea37fe700 1 -- 172.21.15.40:0/1274119262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ea4100ea0 msgr2=0x7f6ea419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:01.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea37fe700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ea4100ea0 0x7f6ea419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:01.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea37fe700 1 -- 172.21.15.40:0/1274119262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea4197030 con 0x7f6ea4106b30 2024-06-06T03:37:01.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea2ffd700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ea4100ea0 0x7f6ea419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:37:01.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea37fe700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea4106b30 0x7f6ea4196850 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00d140 tx=0x7f6e9c00d500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:01.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea89c5700 1 -- 172.21.15.40:0/1274119262 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c00ddc0 con 0x7f6ea4106b30 2024-06-06T03:37:01.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea99c7700 1 -- 172.21.15.40:0/1274119262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea41972c0 con 0x7f6ea4106b30 2024-06-06T03:37:01.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea89c5700 1 -- 172.21.15.40:0/1274119262 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e9c004bb0 con 0x7f6ea4106b30 2024-06-06T03:37:01.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea89c5700 1 -- 172.21.15.40:0/1274119262 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c014400 con 0x7f6ea4106b30 2024-06-06T03:37:01.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.255+0000 7f6ea99c7700 1 -- 172.21.15.40:0/1274119262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea41a34e0 con 0x7f6ea4106b30 2024-06-06T03:37:01.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.256+0000 7f6ea99c7700 1 -- 172.21.15.40:0/1274119262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea4102200 con 0x7f6ea4106b30 2024-06-06T03:37:01.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.258+0000 7f6ea89c5700 1 -- 172.21.15.40:0/1274119262 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6e9c01b020 con 0x7f6ea4106b30 2024-06-06T03:37:01.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.258+0000 7f6ea89c5700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6e900a2cb0 0x7f6e900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:01.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.258+0000 7f6ea2ffd700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6e900a2cb0 0x7f6e900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:01.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.258+0000 7f6ea89c5700 1 -- 172.21.15.40:0/1274119262 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f6e9c0d0700 con 0x7f6ea4106b30 2024-06-06T03:37:01.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.259+0000 7f6ea2ffd700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6e900a2cb0 0x7f6e900a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400b580 tx=0x7f6e94005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:01.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.262+0000 7f6ea89c5700 1 -- 172.21.15.40:0/1274119262 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6e9c011040 con 0x7f6ea4106b30 2024-06-06T03:37:01.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.470+0000 7f6ea99c7700 1 -- 172.21.15.40:0/1274119262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f6ea402dca0 con 0x7f6ea4106b30 2024-06-06T03:37:01.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.470+0000 7f6ea89c5700 1 -- 172.21.15.40:0/1274119262 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6e9c09d620 con 0x7f6ea4106b30 2024-06-06T03:37:01.470 INFO:teuthology.orchestra.run.smithi040.stdout:287762808880 2024-06-06T03:37:01.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.472+0000 7f6ea99c7700 1 -- 172.21.15.40:0/1274119262 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6e900a2cb0 msgr2=0x7f6e900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:01.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.472+0000 7f6ea99c7700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6e900a2cb0 0x7f6e900a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400b580 tx=0x7f6e94005fb0 comp rx=0 tx=0).stop 2024-06-06T03:37:01.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.472+0000 7f6ea99c7700 1 -- 172.21.15.40:0/1274119262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea4106b30 msgr2=0x7f6ea4196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:01.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.472+0000 7f6ea99c7700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea4106b30 0x7f6ea4196850 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00d140 tx=0x7f6e9c00d500 comp rx=0 tx=0).stop 2024-06-06T03:37:01.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.472+0000 7f6ea99c7700 1 -- 172.21.15.40:0/1274119262 shutdown_connections 2024-06-06T03:37:01.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.472+0000 7f6ea99c7700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f6e900a2cb0 0x7f6e900a5170 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:01.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.472+0000 7f6ea99c7700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea4106b30 0x7f6ea4196850 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:01.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.472+0000 7f6ea99c7700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ea4100ea0 0x7f6ea419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:01.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.472+0000 7f6ea99c7700 1 --2- 172.21.15.40:0/1274119262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ea4101860 0x7f6ea419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:01.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.472+0000 7f6ea99c7700 1 -- 172.21.15.40:0/1274119262 >> 172.21.15.40:0/1274119262 conn(0x7f6ea4077590 msgr2=0x7f6ea4103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:01.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.473+0000 7f6ea99c7700 1 -- 172.21.15.40:0/1274119262 shutdown_connections 2024-06-06T03:37:01.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:01.473+0000 7f6ea99c7700 1 -- 172.21.15.40:0/1274119262 wait complete. 2024-06-06T03:37:01.680 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182473 got 107374182479 for osd.3 2024-06-06T03:37:01.681 DEBUG:teuthology.parallel:result is None 2024-06-06T03:37:02.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:01 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1274119262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T03:37:02.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:01 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1274119262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T03:37:02.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:01 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1274119262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T03:37:02.888 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:02 smithi040 ceph-mon[32220]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:37:03.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:02 smithi131 ceph-mon[36687]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:37:03.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:02 smithi104 ceph-mon[36569]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:37:03.127 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:03.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.150+0000 7f9bdb347700 1 -- 172.21.15.40:0/1992820598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bd4103120 msgr2=0x7f9bd4107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:03.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.150+0000 7f9bdb347700 1 --2- 172.21.15.40:0/1992820598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bd4103120 0x7f9bd4107110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40099c0 tx=0x7f9bc4009cd0 comp rx=0 tx=0).stop 2024-06-06T03:37:03.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.150+0000 7f9bdb347700 1 -- 172.21.15.40:0/1992820598 shutdown_connections 2024-06-06T03:37:03.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.150+0000 7f9bdb347700 1 --2- 172.21.15.40:0/1992820598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bd4102770 0x7f9bd4102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:03.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.150+0000 7f9bdb347700 1 --2- 172.21.15.40:0/1992820598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd4107840 0x7f9bd410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:03.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.150+0000 7f9bdb347700 1 --2- 172.21.15.40:0/1992820598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bd4103120 0x7f9bd4107110 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:03.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.150+0000 7f9bdb347700 1 -- 172.21.15.40:0/1992820598 >> 172.21.15.40:0/1992820598 conn(0x7f9bd4079500 msgr2=0x7f9bd4079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:03.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.150+0000 7f9bdb347700 1 -- 172.21.15.40:0/1992820598 shutdown_connections 2024-06-06T03:37:03.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.151+0000 7f9bdb347700 1 -- 172.21.15.40:0/1992820598 wait complete. 2024-06-06T03:37:03.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.151+0000 7f9bdb347700 1 Processor -- start 2024-06-06T03:37:03.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.151+0000 7f9bdb347700 1 -- start start 2024-06-06T03:37:03.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bdb347700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bd4102770 0x7f9bd419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:03.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bdb347700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bd4103120 0x7f9bd419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:03.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bdb347700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd4107840 0x7f9bd4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:03.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bdb347700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd41127a0 con 0x7f9bd4102770 2024-06-06T03:37:03.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bdb347700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd4112620 con 0x7f9bd4107840 2024-06-06T03:37:03.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bdb347700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd4112920 con 0x7f9bd4103120 2024-06-06T03:37:03.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bd90e3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bd4102770 0x7f9bd419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:03.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bd90e3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bd4102770 0x7f9bd419e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39726/0 (socket says 172.21.15.40:39726) 2024-06-06T03:37:03.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bd90e3700 1 -- 172.21.15.40:0/4054278254 learned_addr learned my addr 172.21.15.40:0/4054278254 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:03.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bd88e2700 1 --2- 172.21.15.40:0/4054278254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bd4103120 0x7f9bd419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:03.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bd98e4700 1 --2- 172.21.15.40:0/4054278254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd4107840 0x7f9bd4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:03.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bd90e3700 1 -- 172.21.15.40:0/4054278254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bd4103120 msgr2=0x7f9bd419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:03.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bd90e3700 1 --2- 172.21.15.40:0/4054278254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bd4103120 0x7f9bd419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:03.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bd90e3700 1 -- 172.21.15.40:0/4054278254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd4107840 msgr2=0x7f9bd4198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:03.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bd90e3700 1 --2- 172.21.15.40:0/4054278254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd4107840 0x7f9bd4198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:03.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bd90e3700 1 -- 172.21.15.40:0/4054278254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd4198f60 con 0x7f9bd4102770 2024-06-06T03:37:03.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.152+0000 7f9bd90e3700 1 --2- 172.21.15.40:0/4054278254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bd4102770 0x7f9bd419e600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000e9a0 tx=0x7f9bd000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:03.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.153+0000 7f9bca7fc700 1 -- 172.21.15.40:0/4054278254 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd000cbd0 con 0x7f9bd4102770 2024-06-06T03:37:03.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.153+0000 7f9bca7fc700 1 -- 172.21.15.40:0/4054278254 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bd000cd70 con 0x7f9bd4102770 2024-06-06T03:37:03.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.153+0000 7f9bdb347700 1 -- 172.21.15.40:0/4054278254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd4199250 con 0x7f9bd4102770 2024-06-06T03:37:03.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.153+0000 7f9bca7fc700 1 -- 172.21.15.40:0/4054278254 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0018750 con 0x7f9bd4102770 2024-06-06T03:37:03.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.153+0000 7f9bdb347700 1 -- 172.21.15.40:0/4054278254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd41a5460 con 0x7f9bd4102770 2024-06-06T03:37:03.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.154+0000 7f9bdb347700 1 -- 172.21.15.40:0/4054278254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bd4103bd0 con 0x7f9bd4102770 2024-06-06T03:37:03.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.155+0000 7f9bca7fc700 1 -- 172.21.15.40:0/4054278254 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9bd0010430 con 0x7f9bd4102770 2024-06-06T03:37:03.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.156+0000 7f9bca7fc700 1 --2- 172.21.15.40:0/4054278254 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9bc00a2cb0 0x7f9bc00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:03.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.156+0000 7f9bd88e2700 1 --2- 172.21.15.40:0/4054278254 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9bc00a2cb0 0x7f9bc00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:03.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.156+0000 7f9bca7fc700 1 -- 172.21.15.40:0/4054278254 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f9bd00cf810 con 0x7f9bd4102770 2024-06-06T03:37:03.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.156+0000 7f9bd88e2700 1 --2- 172.21.15.40:0/4054278254 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9bc00a2cb0 0x7f9bc00a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009350 tx=0x7f9bc4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:03.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.159+0000 7f9bca7fc700 1 -- 172.21.15.40:0/4054278254 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9bd009c730 con 0x7f9bd4102770 2024-06-06T03:37:03.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.368+0000 7f9bdb347700 1 -- 172.21.15.40:0/4054278254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9bd41a5750 con 0x7f9bd4102770 2024-06-06T03:37:03.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.368+0000 7f9bca7fc700 1 -- 172.21.15.40:0/4054278254 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9bd009c030 con 0x7f9bd4102770 2024-06-06T03:37:03.368 INFO:teuthology.orchestra.run.smithi040.stdout:55834574938 2024-06-06T03:37:03.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.370+0000 7f9bdb347700 1 -- 172.21.15.40:0/4054278254 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9bc00a2cb0 msgr2=0x7f9bc00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:03.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.370+0000 7f9bdb347700 1 --2- 172.21.15.40:0/4054278254 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9bc00a2cb0 0x7f9bc00a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009350 tx=0x7f9bc4000bc0 comp rx=0 tx=0).stop 2024-06-06T03:37:03.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.370+0000 7f9bdb347700 1 -- 172.21.15.40:0/4054278254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bd4102770 msgr2=0x7f9bd419e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:03.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.370+0000 7f9bdb347700 1 --2- 172.21.15.40:0/4054278254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bd4102770 0x7f9bd419e600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000e9a0 tx=0x7f9bd000ecb0 comp rx=0 tx=0).stop 2024-06-06T03:37:03.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.370+0000 7f9bdb347700 1 -- 172.21.15.40:0/4054278254 shutdown_connections 2024-06-06T03:37:03.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.370+0000 7f9bdb347700 1 --2- 172.21.15.40:0/4054278254 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9bc00a2cb0 0x7f9bc00a5170 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:03.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.370+0000 7f9bdb347700 1 --2- 172.21.15.40:0/4054278254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd4107840 0x7f9bd4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:03.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.370+0000 7f9bdb347700 1 --2- 172.21.15.40:0/4054278254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bd4102770 0x7f9bd419e600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:03.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.370+0000 7f9bdb347700 1 --2- 172.21.15.40:0/4054278254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bd4103120 0x7f9bd419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:03.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.370+0000 7f9bdb347700 1 -- 172.21.15.40:0/4054278254 >> 172.21.15.40:0/4054278254 conn(0x7f9bd4079500 msgr2=0x7f9bd41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:03.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.370+0000 7f9bdb347700 1 -- 172.21.15.40:0/4054278254 shutdown_connections 2024-06-06T03:37:03.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:03.370+0000 7f9bdb347700 1 -- 172.21.15.40:0/4054278254 wait complete. 2024-06-06T03:37:04.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:03 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/4054278254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T03:37:04.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:03 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/4054278254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T03:37:04.138 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:03 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/4054278254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T03:37:04.170 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574933 got 55834574938 for osd.1 2024-06-06T03:37:04.170 DEBUG:teuthology.parallel:result is None 2024-06-06T03:37:04.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.576+0000 7f58918a1700 1 -- 172.21.15.40:0/3863464626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c107400 msgr2=0x7f588c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:04.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.576+0000 7f58918a1700 1 --2- 172.21.15.40:0/3863464626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c107400 0x7f588c1077e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f587c00b0a0 tx=0x7f587c00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:37:04.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.576+0000 7f58918a1700 1 -- 172.21.15.40:0/3863464626 shutdown_connections 2024-06-06T03:37:04.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.576+0000 7f58918a1700 1 --2- 172.21.15.40:0/3863464626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f588c101d50 0x7f588c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:04.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.576+0000 7f58918a1700 1 --2- 172.21.15.40:0/3863464626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c107400 0x7f588c1077e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:04.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.576+0000 7f58918a1700 1 --2- 172.21.15.40:0/3863464626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c101390 0x7f588c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:04.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.576+0000 7f58918a1700 1 -- 172.21.15.40:0/3863464626 >> 172.21.15.40:0/3863464626 conn(0x7f588c078340 msgr2=0x7f588c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:04.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.576+0000 7f58918a1700 1 -- 172.21.15.40:0/3863464626 shutdown_connections 2024-06-06T03:37:04.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.576+0000 7f58918a1700 1 -- 172.21.15.40:0/3863464626 wait complete. 2024-06-06T03:37:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f58918a1700 1 Processor -- start 2024-06-06T03:37:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f58918a1700 1 -- start start 2024-06-06T03:37:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f58918a1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c101390 0x7f588c103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f58918a1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f588c101d50 0x7f588c1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f58918a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c107400 0x7f588c103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f58918a1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1116f0 con 0x7f588c107400 2024-06-06T03:37:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f58918a1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f588c111570 con 0x7f588c101d50 2024-06-06T03:37:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f58918a1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f588c111870 con 0x7f588c101390 2024-06-06T03:37:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f588b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c107400 0x7f588c103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f588b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c107400 0x7f588c103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39750/0 (socket says 172.21.15.40:39750) 2024-06-06T03:37:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f588b7fe700 1 -- 172.21.15.40:0/4164763301 learned_addr learned my addr 172.21.15.40:0/4164763301 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f58827fc700 1 --2- 172.21.15.40:0/4164763301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f588c101d50 0x7f588c1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f588b7fe700 1 -- 172.21.15.40:0/4164763301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c101390 msgr2=0x7f588c103070 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T03:37:04.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f588b7fe700 1 --2- 172.21.15.40:0/4164763301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c101390 0x7f588c103070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f588b7fe700 1 -- 172.21.15.40:0/4164763301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f588c101d50 msgr2=0x7f588c1035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f588b7fe700 1 --2- 172.21.15.40:0/4164763301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f588c101d50 0x7f588c1035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.577+0000 7f588b7fe700 1 -- 172.21.15.40:0/4164763301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c1a7140 con 0x7f588c107400 2024-06-06T03:37:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.578+0000 7f588b7fe700 1 --2- 172.21.15.40:0/4164763301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c107400 0x7f588c103af0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f588400c900 tx=0x7f588400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.578+0000 7f589089f700 1 -- 172.21.15.40:0/4164763301 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884007b00 con 0x7f588c107400 2024-06-06T03:37:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.578+0000 7f58918a1700 1 -- 172.21.15.40:0/4164763301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588c1a7430 con 0x7f588c107400 2024-06-06T03:37:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.578+0000 7f589089f700 1 -- 172.21.15.40:0/4164763301 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5884007ca0 con 0x7f588c107400 2024-06-06T03:37:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.578+0000 7f589089f700 1 -- 172.21.15.40:0/4164763301 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884018620 con 0x7f588c107400 2024-06-06T03:37:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.578+0000 7f58918a1700 1 -- 172.21.15.40:0/4164763301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f588c1a7970 con 0x7f588c107400 2024-06-06T03:37:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.579+0000 7f58918a1700 1 -- 172.21.15.40:0/4164763301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f588c069180 con 0x7f588c107400 2024-06-06T03:37:04.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.580+0000 7f589089f700 1 -- 172.21.15.40:0/4164763301 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f588400f450 con 0x7f588c107400 2024-06-06T03:37:04.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.581+0000 7f589089f700 1 --2- 172.21.15.40:0/4164763301 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58780a2cb0 0x7f58780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:04.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.581+0000 7f588affd700 1 --2- 172.21.15.40:0/4164763301 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58780a2cb0 0x7f58780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:04.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.581+0000 7f589089f700 1 -- 172.21.15.40:0/4164763301 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f58840cf000 con 0x7f588c107400 2024-06-06T03:37:04.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.581+0000 7f588affd700 1 --2- 172.21.15.40:0/4164763301 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58780a2cb0 0x7f58780a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f587c000f80 tx=0x7f587c0060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:04.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.586+0000 7f589089f700 1 -- 172.21.15.40:0/4164763301 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5884014030 con 0x7f588c107400 2024-06-06T03:37:04.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.797+0000 7f58918a1700 1 -- 172.21.15.40:0/4164763301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f588c1a7c30 con 0x7f588c107400 2024-06-06T03:37:04.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.797+0000 7f589089f700 1 -- 172.21.15.40:0/4164763301 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f588409bf20 con 0x7f588c107400 2024-06-06T03:37:04.797 INFO:teuthology.orchestra.run.smithi040.stdout:219043332155 2024-06-06T03:37:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.799+0000 7f58918a1700 1 -- 172.21.15.40:0/4164763301 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58780a2cb0 msgr2=0x7f58780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.799+0000 7f58918a1700 1 --2- 172.21.15.40:0/4164763301 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58780a2cb0 0x7f58780a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f587c000f80 tx=0x7f587c0060c0 comp rx=0 tx=0).stop 2024-06-06T03:37:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.799+0000 7f58918a1700 1 -- 172.21.15.40:0/4164763301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c107400 msgr2=0x7f588c103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.799+0000 7f58918a1700 1 --2- 172.21.15.40:0/4164763301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c107400 0x7f588c103af0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f588400c900 tx=0x7f588400ccc0 comp rx=0 tx=0).stop 2024-06-06T03:37:04.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.799+0000 7f58918a1700 1 -- 172.21.15.40:0/4164763301 shutdown_connections 2024-06-06T03:37:04.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.799+0000 7f58918a1700 1 --2- 172.21.15.40:0/4164763301 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58780a2cb0 0x7f58780a5170 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:04.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.799+0000 7f58918a1700 1 --2- 172.21.15.40:0/4164763301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c101390 0x7f588c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:04.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.799+0000 7f58918a1700 1 --2- 172.21.15.40:0/4164763301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c107400 0x7f588c103af0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:04.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.799+0000 7f58918a1700 1 --2- 172.21.15.40:0/4164763301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f588c101d50 0x7f588c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:04.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.800+0000 7f58918a1700 1 -- 172.21.15.40:0/4164763301 >> 172.21.15.40:0/4164763301 conn(0x7f588c078340 msgr2=0x7f588c100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:04.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.800+0000 7f58918a1700 1 -- 172.21.15.40:0/4164763301 shutdown_connections 2024-06-06T03:37:04.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:04.800+0000 7f58918a1700 1 -- 172.21.15.40:0/4164763301 wait complete. 2024-06-06T03:37:05.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:04 smithi040 ceph-mon[32220]: pgmap v358: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:37:05.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:04 smithi104 ceph-mon[36569]: pgmap v358: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:37:05.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:04 smithi131 ceph-mon[36687]: pgmap v358: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:37:05.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.284+0000 7fa89630f700 1 -- 172.21.15.40:0/4265052410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890101d50 msgr2=0x7fa89010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:05.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.284+0000 7fa89630f700 1 --2- 172.21.15.40:0/4265052410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890101d50 0x7fa89010e240 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa88800b0a0 tx=0x7fa88800b3b0 comp rx=0 tx=0).stop 2024-06-06T03:37:05.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.284+0000 7fa89630f700 1 -- 172.21.15.40:0/4265052410 shutdown_connections 2024-06-06T03:37:05.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.284+0000 7fa89630f700 1 --2- 172.21.15.40:0/4265052410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa890107400 0x7fa8901077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:05.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.284+0000 7fa89630f700 1 --2- 172.21.15.40:0/4265052410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890101d50 0x7fa89010e240 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:05.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.284+0000 7fa89630f700 1 --2- 172.21.15.40:0/4265052410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa890101390 0x7fa890101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:05.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.284+0000 7fa89630f700 1 -- 172.21.15.40:0/4265052410 >> 172.21.15.40:0/4265052410 conn(0x7fa890078340 msgr2=0x7fa890078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:05.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.284+0000 7fa89630f700 1 -- 172.21.15.40:0/4265052410 shutdown_connections 2024-06-06T03:37:05.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.284+0000 7fa89630f700 1 -- 172.21.15.40:0/4265052410 wait complete. 2024-06-06T03:37:05.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.285+0000 7fa89630f700 1 Processor -- start 2024-06-06T03:37:05.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.285+0000 7fa89630f700 1 -- start start 2024-06-06T03:37:05.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.285+0000 7fa89630f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa890101390 0x7fa89019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:05.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.285+0000 7fa89630f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa890101d50 0x7fa89019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:05.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.285+0000 7fa89630f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890107400 0x7fa890198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:05.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.285+0000 7fa89630f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa890111670 con 0x7fa890107400 2024-06-06T03:37:05.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.285+0000 7fa89630f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa8901114f0 con 0x7fa890101390 2024-06-06T03:37:05.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.285+0000 7fa89630f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8901117f0 con 0x7fa890101d50 2024-06-06T03:37:05.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.285+0000 7fa8948ac700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890107400 0x7fa890198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:05.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.285+0000 7fa8948ac700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890107400 0x7fa890198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39772/0 (socket says 172.21.15.40:39772) 2024-06-06T03:37:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.285+0000 7fa8948ac700 1 -- 172.21.15.40:0/2989104778 learned_addr learned my addr 172.21.15.40:0/2989104778 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.285+0000 7fa88ffff700 1 --2- 172.21.15.40:0/2989104778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa890101390 0x7fa89019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.286+0000 7fa88f7fe700 1 --2- 172.21.15.40:0/2989104778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa890101d50 0x7fa89019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.286+0000 7fa8948ac700 1 -- 172.21.15.40:0/2989104778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa890101d50 msgr2=0x7fa89019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.286+0000 7fa8948ac700 1 --2- 172.21.15.40:0/2989104778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa890101d50 0x7fa89019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.286+0000 7fa8948ac700 1 -- 172.21.15.40:0/2989104778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa890101390 msgr2=0x7fa89019e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.286+0000 7fa8948ac700 1 --2- 172.21.15.40:0/2989104778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa890101390 0x7fa89019e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.286+0000 7fa8948ac700 1 -- 172.21.15.40:0/2989104778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8901990d0 con 0x7fa890107400 2024-06-06T03:37:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.286+0000 7fa8948ac700 1 --2- 172.21.15.40:0/2989104778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890107400 0x7fa890198970 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa8880062a0 tx=0x7fa888004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.286+0000 7fa88d7fa700 1 -- 172.21.15.40:0/2989104778 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888021070 con 0x7fa890107400 2024-06-06T03:37:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.286+0000 7fa89630f700 1 -- 172.21.15.40:0/2989104778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa890199360 con 0x7fa890107400 2024-06-06T03:37:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.286+0000 7fa88d7fa700 1 -- 172.21.15.40:0/2989104778 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa888003c00 con 0x7fa890107400 2024-06-06T03:37:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.286+0000 7fa88d7fa700 1 -- 172.21.15.40:0/2989104778 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888004880 con 0x7fa890107400 2024-06-06T03:37:05.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.286+0000 7fa89630f700 1 -- 172.21.15.40:0/2989104778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8901a5630 con 0x7fa890107400 2024-06-06T03:37:05.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.287+0000 7fa89630f700 1 -- 172.21.15.40:0/2989104778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa890069180 con 0x7fa890107400 2024-06-06T03:37:05.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.289+0000 7fa88d7fa700 1 -- 172.21.15.40:0/2989104778 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa888007980 con 0x7fa890107400 2024-06-06T03:37:05.289 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:05.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.290+0000 7fa88d7fa700 1 --2- 172.21.15.40:0/2989104778 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa8780a2d30 0x7fa8780a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.290+0000 7fa88ffff700 1 --2- 172.21.15.40:0/2989104778 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa8780a2d30 0x7fa8780a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.290+0000 7fa88d7fa700 1 -- 172.21.15.40:0/2989104778 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fa8880d0370 con 0x7fa890107400 2024-06-06T03:37:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.291+0000 7fa88ffff700 1 --2- 172.21.15.40:0/2989104778 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa8780a2d30 0x7fa8780a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa8900ffb00 tx=0x7fa88000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:05.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.293+0000 7fa88d7fa700 1 -- 172.21.15.40:0/2989104778 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa88809d3a0 con 0x7fa890107400 2024-06-06T03:37:05.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.506+0000 7fa89630f700 1 -- 172.21.15.40:0/2989104778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa8901a5800 con 0x7fa890107400 2024-06-06T03:37:05.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.506+0000 7fa88d7fa700 1 -- 172.21.15.40:0/2989104778 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa88809cca0 con 0x7fa890107400 2024-06-06T03:37:05.506 INFO:teuthology.orchestra.run.smithi040.stdout:34359738463 2024-06-06T03:37:05.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.508+0000 7fa89630f700 1 -- 172.21.15.40:0/2989104778 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa8780a2d30 msgr2=0x7fa8780a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:05.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.508+0000 7fa89630f700 1 --2- 172.21.15.40:0/2989104778 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa8780a2d30 0x7fa8780a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa8900ffb00 tx=0x7fa88000b410 comp rx=0 tx=0).stop 2024-06-06T03:37:05.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.508+0000 7fa89630f700 1 -- 172.21.15.40:0/2989104778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890107400 msgr2=0x7fa890198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:05.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.508+0000 7fa89630f700 1 --2- 172.21.15.40:0/2989104778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890107400 0x7fa890198970 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa8880062a0 tx=0x7fa888004d70 comp rx=0 tx=0).stop 2024-06-06T03:37:05.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.508+0000 7fa89630f700 1 -- 172.21.15.40:0/2989104778 shutdown_connections 2024-06-06T03:37:05.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.508+0000 7fa89630f700 1 --2- 172.21.15.40:0/2989104778 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa8780a2d30 0x7fa8780a51f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:05.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.508+0000 7fa89630f700 1 --2- 172.21.15.40:0/2989104778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890107400 0x7fa890198970 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:05.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.508+0000 7fa89630f700 1 --2- 172.21.15.40:0/2989104778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa890101390 0x7fa89019e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:05.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.508+0000 7fa89630f700 1 --2- 172.21.15.40:0/2989104778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa890101d50 0x7fa89019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:05.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.508+0000 7fa89630f700 1 -- 172.21.15.40:0/2989104778 >> 172.21.15.40:0/2989104778 conn(0x7fa890078340 msgr2=0x7fa890100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:05.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.508+0000 7fa89630f700 1 -- 172.21.15.40:0/2989104778 shutdown_connections 2024-06-06T03:37:05.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:05.508+0000 7fa89630f700 1 -- 172.21.15.40:0/2989104778 wait complete. 2024-06-06T03:37:06.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:05 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/4164763301' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T03:37:06.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:05 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2989104778' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T03:37:06.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:05 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/4164763301' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T03:37:06.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:05 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2989104778' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T03:37:06.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:05 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/4164763301' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T03:37:06.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:05 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2989104778' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T03:37:06.558 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332147 got 219043332155 for osd.7 2024-06-06T03:37:06.558 DEBUG:teuthology.parallel:result is None 2024-06-06T03:37:07.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:06 smithi040 ceph-mon[32220]: pgmap v359: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:37:07.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:06 smithi104 ceph-mon[36569]: pgmap v359: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:37:07.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:06 smithi131 ceph-mon[36687]: pgmap v359: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:37:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.831+0000 7f376bd8c700 1 -- 172.21.15.40:0/3139819413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3764101390 msgr2=0x7f3764101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.831+0000 7f376bd8c700 1 --2- 172.21.15.40:0/3139819413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3764101390 0x7f3764101810 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3754009990 tx=0x7f3754009ca0 comp rx=0 tx=0).stop 2024-06-06T03:37:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.831+0000 7f376bd8c700 1 -- 172.21.15.40:0/3139819413 shutdown_connections 2024-06-06T03:37:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.831+0000 7f376bd8c700 1 --2- 172.21.15.40:0/3139819413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3764101d50 0x7f376410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:07.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.831+0000 7f376bd8c700 1 --2- 172.21.15.40:0/3139819413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3764107400 0x7f37641077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:07.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.831+0000 7f376bd8c700 1 --2- 172.21.15.40:0/3139819413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3764101390 0x7f3764101810 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:07.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.831+0000 7f376bd8c700 1 -- 172.21.15.40:0/3139819413 >> 172.21.15.40:0/3139819413 conn(0x7f3764078340 msgr2=0x7f3764078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:07.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.831+0000 7f376bd8c700 1 -- 172.21.15.40:0/3139819413 shutdown_connections 2024-06-06T03:37:07.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.831+0000 7f376bd8c700 1 -- 172.21.15.40:0/3139819413 wait complete. 2024-06-06T03:37:07.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.832+0000 7f376bd8c700 1 Processor -- start 2024-06-06T03:37:07.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.832+0000 7f376bd8c700 1 -- start start 2024-06-06T03:37:07.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.832+0000 7f376bd8c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3764101390 0x7f37641979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:07.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.832+0000 7f376bd8c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3764101d50 0x7f3764197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:07.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.832+0000 7f376bd8c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3764107400 0x7f376419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:07.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.832+0000 7f376bd8c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37641116f0 con 0x7f3764101390 2024-06-06T03:37:07.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.832+0000 7f376bd8c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3764111570 con 0x7f3764107400 2024-06-06T03:37:07.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.832+0000 7f376bd8c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3764111870 con 0x7f3764101d50 2024-06-06T03:37:07.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.832+0000 7f3769b28700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3764101390 0x7f37641979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:07.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f3769b28700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3764101390 0x7f37641979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39798/0 (socket says 172.21.15.40:39798) 2024-06-06T03:37:07.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f3769b28700 1 -- 172.21.15.40:0/320495875 learned_addr learned my addr 172.21.15.40:0/320495875 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:07.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f376a329700 1 --2- 172.21.15.40:0/320495875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3764107400 0x7f376419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:07.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f3769327700 1 --2- 172.21.15.40:0/320495875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3764101d50 0x7f3764197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:07.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f3769b28700 1 -- 172.21.15.40:0/320495875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3764101d50 msgr2=0x7f3764197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:07.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f3769b28700 1 --2- 172.21.15.40:0/320495875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3764101d50 0x7f3764197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:07.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f3769b28700 1 -- 172.21.15.40:0/320495875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3764107400 msgr2=0x7f376419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:07.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f3769b28700 1 --2- 172.21.15.40:0/320495875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3764107400 0x7f376419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:07.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f3769b28700 1 -- 172.21.15.40:0/320495875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376419cf70 con 0x7f3764101390 2024-06-06T03:37:07.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f3769b28700 1 --2- 172.21.15.40:0/320495875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3764101390 0x7f37641979e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f376000b530 tx=0x7f376000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:07.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f375affd700 1 -- 172.21.15.40:0/320495875 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760015020 con 0x7f3764101390 2024-06-06T03:37:07.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f375affd700 1 -- 172.21.15.40:0/320495875 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3760011d10 con 0x7f3764101390 2024-06-06T03:37:07.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f376bd8c700 1 -- 172.21.15.40:0/320495875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376419d260 con 0x7f3764101390 2024-06-06T03:37:07.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f375affd700 1 -- 172.21.15.40:0/320495875 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376000f2e0 con 0x7f3764101390 2024-06-06T03:37:07.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.833+0000 7f376bd8c700 1 -- 172.21.15.40:0/320495875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376419d7f0 con 0x7f3764101390 2024-06-06T03:37:07.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.834+0000 7f376bd8c700 1 -- 172.21.15.40:0/320495875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3764109970 con 0x7f3764101390 2024-06-06T03:37:07.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.835+0000 7f375affd700 1 -- 172.21.15.40:0/320495875 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3760018020 con 0x7f3764101390 2024-06-06T03:37:07.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.836+0000 7f375affd700 1 --2- 172.21.15.40:0/320495875 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f37500a2d30 0x7f37500a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:07.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.836+0000 7f3769327700 1 --2- 172.21.15.40:0/320495875 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f37500a2d30 0x7f37500a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:07.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.836+0000 7f375affd700 1 -- 172.21.15.40:0/320495875 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f37600ce7c0 con 0x7f3764101390 2024-06-06T03:37:07.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.837+0000 7f3769327700 1 --2- 172.21.15.40:0/320495875 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f37500a2d30 0x7f37500a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3754009750 tx=0x7f375400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:07.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:07.841+0000 7f375affd700 1 -- 172.21.15.40:0/320495875 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f376001f020 con 0x7f3764101390 2024-06-06T03:37:08.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:08.046+0000 7f376bd8c700 1 -- 172.21.15.40:0/320495875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f376406a770 con 0x7f3764101390 2024-06-06T03:37:08.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:08.047+0000 7f375affd700 1 -- 172.21.15.40:0/320495875 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f376009b7f0 con 0x7f3764101390 2024-06-06T03:37:08.047 INFO:teuthology.orchestra.run.smithi040.stdout:343597383720 2024-06-06T03:37:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:08.049+0000 7f376bd8c700 1 -- 172.21.15.40:0/320495875 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f37500a2d30 msgr2=0x7f37500a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:08.049+0000 7f376bd8c700 1 --2- 172.21.15.40:0/320495875 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f37500a2d30 0x7f37500a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3754009750 tx=0x7f375400b540 comp rx=0 tx=0).stop 2024-06-06T03:37:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:08.049+0000 7f376bd8c700 1 -- 172.21.15.40:0/320495875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3764101390 msgr2=0x7f37641979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:08.049+0000 7f376bd8c700 1 --2- 172.21.15.40:0/320495875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3764101390 0x7f37641979e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f376000b530 tx=0x7f376000b840 comp rx=0 tx=0).stop 2024-06-06T03:37:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:08.049+0000 7f376bd8c700 1 -- 172.21.15.40:0/320495875 shutdown_connections 2024-06-06T03:37:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:08.049+0000 7f376bd8c700 1 --2- 172.21.15.40:0/320495875 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f37500a2d30 0x7f37500a51f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:08.049+0000 7f376bd8c700 1 --2- 172.21.15.40:0/320495875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3764107400 0x7f376419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:08.049+0000 7f376bd8c700 1 --2- 172.21.15.40:0/320495875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3764101390 0x7f37641979e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:08.049+0000 7f376bd8c700 1 --2- 172.21.15.40:0/320495875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3764101d50 0x7f3764197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:08.049+0000 7f376bd8c700 1 -- 172.21.15.40:0/320495875 >> 172.21.15.40:0/320495875 conn(0x7f3764078340 msgr2=0x7f37640fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:08.049+0000 7f376bd8c700 1 -- 172.21.15.40:0/320495875 shutdown_connections 2024-06-06T03:37:08.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:08.049+0000 7f376bd8c700 1 -- 172.21.15.40:0/320495875 wait complete. 2024-06-06T03:37:08.288 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808874 got 287762808880 for osd.9 2024-06-06T03:37:08.288 DEBUG:teuthology.parallel:result is None 2024-06-06T03:37:08.899 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:08 smithi040 ceph-mon[32220]: pgmap v360: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:37:08.899 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:08 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/320495875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T03:37:08.936 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738463 for osd.0 2024-06-06T03:37:08.936 DEBUG:teuthology.parallel:result is None 2024-06-06T03:37:09.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:08 smithi104 ceph-mon[36569]: pgmap v360: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:37:09.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:08 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/320495875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T03:37:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:08 smithi131 ceph-mon[36687]: pgmap v360: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:37:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:08 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/320495875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T03:37:09.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.718+0000 7f49d79b4700 1 -- 172.21.15.40:0/1074343672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d0107840 msgr2=0x7f49d010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:09.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.718+0000 7f49d79b4700 1 --2- 172.21.15.40:0/1074343672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d0107840 0x7f49d010f3c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f49c800b0d0 tx=0x7f49c800b3e0 comp rx=0 tx=0).stop 2024-06-06T03:37:09.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.718+0000 7f49d79b4700 1 -- 172.21.15.40:0/1074343672 shutdown_connections 2024-06-06T03:37:09.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.718+0000 7f49d79b4700 1 --2- 172.21.15.40:0/1074343672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d0107840 0x7f49d010f3c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:09.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.718+0000 7f49d79b4700 1 --2- 172.21.15.40:0/1074343672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49d0102740 0x7f49d0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:09.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.718+0000 7f49d79b4700 1 --2- 172.21.15.40:0/1074343672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d01030f0 0x7f49d0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:09.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.718+0000 7f49d79b4700 1 -- 172.21.15.40:0/1074343672 >> 172.21.15.40:0/1074343672 conn(0x7f49d00795f0 msgr2=0x7f49d0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:09.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.718+0000 7f49d79b4700 1 -- 172.21.15.40:0/1074343672 shutdown_connections 2024-06-06T03:37:09.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.718+0000 7f49d79b4700 1 -- 172.21.15.40:0/1074343672 wait complete. 2024-06-06T03:37:09.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.718+0000 7f49d79b4700 1 Processor -- start 2024-06-06T03:37:09.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.718+0000 7f49d79b4700 1 -- start start 2024-06-06T03:37:09.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d79b4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d0102740 0x7f49d019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:09.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d79b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d01030f0 0x7f49d019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:09.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d79b4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49d0107840 0x7f49d0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:09.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d79b4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0112850 con 0x7f49d01030f0 2024-06-06T03:37:09.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d79b4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f49d01126d0 con 0x7f49d0107840 2024-06-06T03:37:09.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d79b4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f49d01129d0 con 0x7f49d0102740 2024-06-06T03:37:09.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d4f4f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d01030f0 0x7f49d019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:09.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d4f4f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d01030f0 0x7f49d019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35998/0 (socket says 172.21.15.40:35998) 2024-06-06T03:37:09.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d4f4f700 1 -- 172.21.15.40:0/3975594790 learned_addr learned my addr 172.21.15.40:0/3975594790 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:09.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d5f51700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49d0107840 0x7f49d0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:09.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d5750700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d0102740 0x7f49d019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:09.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d4f4f700 1 -- 172.21.15.40:0/3975594790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d0102740 msgr2=0x7f49d019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:09.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d4f4f700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d0102740 0x7f49d019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:09.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d4f4f700 1 -- 172.21.15.40:0/3975594790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49d0107840 msgr2=0x7f49d0198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:09.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d4f4f700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49d0107840 0x7f49d0198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:09.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d4f4f700 1 -- 172.21.15.40:0/3975594790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d0199210 con 0x7f49d01030f0 2024-06-06T03:37:09.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d5f51700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49d0107840 0x7f49d0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:37:09.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d5750700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d0102740 0x7f49d019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:37:09.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d4f4f700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d01030f0 0x7f49d019ee30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f49c000b530 tx=0x7f49c000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:09.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49c67fc700 1 -- 172.21.15.40:0/3975594790 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0015020 con 0x7f49d01030f0 2024-06-06T03:37:09.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49c67fc700 1 -- 172.21.15.40:0/3975594790 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c0011d10 con 0x7f49d01030f0 2024-06-06T03:37:09.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d79b4700 1 -- 172.21.15.40:0/3975594790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d0199500 con 0x7f49d01030f0 2024-06-06T03:37:09.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49c67fc700 1 -- 172.21.15.40:0/3975594790 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0010e50 con 0x7f49d01030f0 2024-06-06T03:37:09.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.719+0000 7f49d79b4700 1 -- 172.21.15.40:0/3975594790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d01a5840 con 0x7f49d01030f0 2024-06-06T03:37:09.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.720+0000 7f49d79b4700 1 -- 172.21.15.40:0/3975594790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d0069180 con 0x7f49d01030f0 2024-06-06T03:37:09.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.721+0000 7f49c67fc700 1 -- 172.21.15.40:0/3975594790 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f49c0018020 con 0x7f49d01030f0 2024-06-06T03:37:09.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.722+0000 7f49c67fc700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f49bc0a2ed0 0x7f49bc0a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:09.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.722+0000 7f49d5750700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f49bc0a2ed0 0x7f49bc0a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:09.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.722+0000 7f49c67fc700 1 -- 172.21.15.40:0/3975594790 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f49c00cece0 con 0x7f49d01030f0 2024-06-06T03:37:09.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.723+0000 7f49d5750700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f49bc0a2ed0 0x7f49bc0a5390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f49d0199bf0 tx=0x7f49cc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:09.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.726+0000 7f49c67fc700 1 -- 172.21.15.40:0/3975594790 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f49c001f020 con 0x7f49d01030f0 2024-06-06T03:37:09.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.938+0000 7f49d79b4700 1 -- 172.21.15.40:0/3975594790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f49d010aa80 con 0x7f49d01030f0 2024-06-06T03:37:09.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.938+0000 7f49c67fc700 1 -- 172.21.15.40:0/3975594790 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f49c009bd10 con 0x7f49d01030f0 2024-06-06T03:37:09.938 INFO:teuthology.orchestra.run.smithi040.stdout:253403070519 2024-06-06T03:37:09.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.940+0000 7f49d79b4700 1 -- 172.21.15.40:0/3975594790 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f49bc0a2ed0 msgr2=0x7f49bc0a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:09.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.941+0000 7f49d79b4700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f49bc0a2ed0 0x7f49bc0a5390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f49d0199bf0 tx=0x7f49cc008040 comp rx=0 tx=0).stop 2024-06-06T03:37:09.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.941+0000 7f49d79b4700 1 -- 172.21.15.40:0/3975594790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d01030f0 msgr2=0x7f49d019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:09.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.941+0000 7f49d79b4700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d01030f0 0x7f49d019ee30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f49c000b530 tx=0x7f49c000b8f0 comp rx=0 tx=0).stop 2024-06-06T03:37:09.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.941+0000 7f49d79b4700 1 -- 172.21.15.40:0/3975594790 shutdown_connections 2024-06-06T03:37:09.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.941+0000 7f49d79b4700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f49bc0a2ed0 0x7f49bc0a5390 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:09.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.941+0000 7f49d79b4700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d0102740 0x7f49d019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:09.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.941+0000 7f49d79b4700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49d0107840 0x7f49d0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:09.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.941+0000 7f49d79b4700 1 --2- 172.21.15.40:0/3975594790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d01030f0 0x7f49d019ee30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:09.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.941+0000 7f49d79b4700 1 -- 172.21.15.40:0/3975594790 >> 172.21.15.40:0/3975594790 conn(0x7f49d00795f0 msgr2=0x7f49d0100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:09.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.941+0000 7f49d79b4700 1 -- 172.21.15.40:0/3975594790 shutdown_connections 2024-06-06T03:37:09.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:09.941+0000 7f49d79b4700 1 -- 172.21.15.40:0/3975594790 wait complete. 2024-06-06T03:37:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.438+0000 7ff6a6215700 1 -- 172.21.15.40:0/1785431754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a0101fd0 msgr2=0x7ff6a0105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.438+0000 7ff6a6215700 1 --2- 172.21.15.40:0/1785431754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a0101fd0 0x7ff6a0105fd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff6940099c0 tx=0x7ff694009cd0 comp rx=0 tx=0).stop 2024-06-06T03:37:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.438+0000 7ff6a6215700 1 -- 172.21.15.40:0/1785431754 shutdown_connections 2024-06-06T03:37:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.438+0000 7ff6a6215700 1 --2- 172.21.15.40:0/1785431754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6a01066b0 0x7ff6a010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.438+0000 7ff6a6215700 1 --2- 172.21.15.40:0/1785431754 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6a0101620 0x7ff6a0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.438+0000 7ff6a6215700 1 --2- 172.21.15.40:0/1785431754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a0101fd0 0x7ff6a0105fd0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.438+0000 7ff6a6215700 1 -- 172.21.15.40:0/1785431754 >> 172.21.15.40:0/1785431754 conn(0x7ff6a0078340 msgr2=0x7ff6a0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.438+0000 7ff6a6215700 1 -- 172.21.15.40:0/1785431754 shutdown_connections 2024-06-06T03:37:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.438+0000 7ff6a6215700 1 -- 172.21.15.40:0/1785431754 wait complete. 2024-06-06T03:37:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.439+0000 7ff6a6215700 1 Processor -- start 2024-06-06T03:37:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.439+0000 7ff6a6215700 1 -- start start 2024-06-06T03:37:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.439+0000 7ff6a6215700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a0101620 0x7ff6a0199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.439+0000 7ff6a6215700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6a0101fd0 0x7ff6a0199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.439+0000 7ff6a6215700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6a01066b0 0x7ff6a01a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.439+0000 7ff6a6215700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a0111670 con 0x7ff6a0101620 2024-06-06T03:37:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.439+0000 7ff6a6215700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01114f0 con 0x7ff6a0101fd0 2024-06-06T03:37:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.439+0000 7ff6a6215700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01117f0 con 0x7ff6a01066b0 2024-06-06T03:37:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.439+0000 7ff69f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a0101620 0x7ff6a0199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff69f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a0101620 0x7ff6a0199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36026/0 (socket says 172.21.15.40:36026) 2024-06-06T03:37:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff69f7fe700 1 -- 172.21.15.40:0/343465948 learned_addr learned my addr 172.21.15.40:0/343465948 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff69effd700 1 --2- 172.21.15.40:0/343465948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6a0101fd0 0x7ff6a0199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff69ffff700 1 --2- 172.21.15.40:0/343465948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6a01066b0 0x7ff6a01a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff69f7fe700 1 -- 172.21.15.40:0/343465948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6a01066b0 msgr2=0x7ff6a01a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff69f7fe700 1 --2- 172.21.15.40:0/343465948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6a01066b0 0x7ff6a01a4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff69f7fe700 1 -- 172.21.15.40:0/343465948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6a0101fd0 msgr2=0x7ff6a0199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff69f7fe700 1 --2- 172.21.15.40:0/343465948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6a0101fd0 0x7ff6a0199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff69f7fe700 1 -- 172.21.15.40:0/343465948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a01a5010 con 0x7ff6a0101620 2024-06-06T03:37:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff69f7fe700 1 --2- 172.21.15.40:0/343465948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a0101620 0x7ff6a0199a50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff69000d950 tx=0x7ff69000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff69cff9700 1 -- 172.21.15.40:0/343465948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690016070 con 0x7ff6a0101620 2024-06-06T03:37:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff69cff9700 1 -- 172.21.15.40:0/343465948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff690009e20 con 0x7ff6a0101620 2024-06-06T03:37:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff6a6215700 1 -- 172.21.15.40:0/343465948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a01a5240 con 0x7ff6a0101620 2024-06-06T03:37:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff69cff9700 1 -- 172.21.15.40:0/343465948 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69000f620 con 0x7ff6a0101620 2024-06-06T03:37:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.440+0000 7ff6a6215700 1 -- 172.21.15.40:0/343465948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a01a57a0 con 0x7ff6a0101620 2024-06-06T03:37:10.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.441+0000 7ff6a6215700 1 -- 172.21.15.40:0/343465948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a0069180 con 0x7ff6a0101620 2024-06-06T03:37:10.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.442+0000 7ff69cff9700 1 -- 172.21.15.40:0/343465948 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff690019070 con 0x7ff6a0101620 2024-06-06T03:37:10.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.443+0000 7ff69cff9700 1 --2- 172.21.15.40:0/343465948 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff6880a2d30 0x7ff6880a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:10.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.443+0000 7ff69effd700 1 --2- 172.21.15.40:0/343465948 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff6880a2d30 0x7ff6880a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:10.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.443+0000 7ff69cff9700 1 -- 172.21.15.40:0/343465948 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7ff6900cec00 con 0x7ff6a0101620 2024-06-06T03:37:10.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.444+0000 7ff69effd700 1 --2- 172.21.15.40:0/343465948 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff6880a2d30 0x7ff6880a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff694000c00 tx=0x7ff694005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.447+0000 7ff69cff9700 1 -- 172.21.15.40:0/343465948 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff6900d4020 con 0x7ff6a0101620 2024-06-06T03:37:10.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.659+0000 7ff6a6215700 1 -- 172.21.15.40:0/343465948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff6a002dca0 con 0x7ff6a0101620 2024-06-06T03:37:10.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.659+0000 7ff69cff9700 1 -- 172.21.15.40:0/343465948 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff69009bb20 con 0x7ff6a0101620 2024-06-06T03:37:10.659 INFO:teuthology.orchestra.run.smithi040.stdout:197568495681 2024-06-06T03:37:10.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.661+0000 7ff6a6215700 1 -- 172.21.15.40:0/343465948 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff6880a2d30 msgr2=0x7ff6880a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:10.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.661+0000 7ff6a6215700 1 --2- 172.21.15.40:0/343465948 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff6880a2d30 0x7ff6880a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff694000c00 tx=0x7ff694005fb0 comp rx=0 tx=0).stop 2024-06-06T03:37:10.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.661+0000 7ff6a6215700 1 -- 172.21.15.40:0/343465948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a0101620 msgr2=0x7ff6a0199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:10.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.661+0000 7ff6a6215700 1 --2- 172.21.15.40:0/343465948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a0101620 0x7ff6a0199a50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff69000d950 tx=0x7ff69000dc60 comp rx=0 tx=0).stop 2024-06-06T03:37:10.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.661+0000 7ff6a6215700 1 -- 172.21.15.40:0/343465948 shutdown_connections 2024-06-06T03:37:10.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.661+0000 7ff6a6215700 1 --2- 172.21.15.40:0/343465948 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff6880a2d30 0x7ff6880a51f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:10.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.661+0000 7ff6a6215700 1 --2- 172.21.15.40:0/343465948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6a01066b0 0x7ff6a01a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:10.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.661+0000 7ff6a6215700 1 --2- 172.21.15.40:0/343465948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a0101620 0x7ff6a0199a50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:10.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.661+0000 7ff6a6215700 1 --2- 172.21.15.40:0/343465948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6a0101fd0 0x7ff6a0199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:10.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.661+0000 7ff6a6215700 1 -- 172.21.15.40:0/343465948 >> 172.21.15.40:0/343465948 conn(0x7ff6a0078340 msgr2=0x7ff6a00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:10.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.662+0000 7ff6a6215700 1 -- 172.21.15.40:0/343465948 shutdown_connections 2024-06-06T03:37:10.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:10.662+0000 7ff6a6215700 1 -- 172.21.15.40:0/343465948 wait complete. 2024-06-06T03:37:10.876 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:10 smithi040 ceph-mon[32220]: pgmap v361: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:37:10.876 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:10 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3975594790' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T03:37:10.876 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:10 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/343465948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T03:37:10.903 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383714 got 343597383720 for osd.11 2024-06-06T03:37:10.903 DEBUG:teuthology.parallel:result is None 2024-06-06T03:37:11.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:10 smithi104 ceph-mon[36569]: pgmap v361: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:37:11.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:10 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3975594790' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T03:37:11.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:10 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/343465948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T03:37:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:10 smithi131 ceph-mon[36687]: pgmap v361: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:37:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:10 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3975594790' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T03:37:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:10 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/343465948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T03:37:11.754 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495676 got 197568495681 for osd.6 2024-06-06T03:37:11.754 DEBUG:teuthology.parallel:result is None 2024-06-06T03:37:11.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.935+0000 7f21c92cd700 1 -- 172.21.15.40:0/2662819844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c406b580 msgr2=0x7f21c410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:11.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.935+0000 7f21c92cd700 1 --2- 172.21.15.40:0/2662819844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c406b580 0x7f21c410f320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00b0a0 tx=0x7f21bc00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:37:11.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.935+0000 7f21c92cd700 1 -- 172.21.15.40:0/2662819844 shutdown_connections 2024-06-06T03:37:11.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.935+0000 7f21c92cd700 1 --2- 172.21.15.40:0/2662819844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c4108500 0x7f21c41088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:11.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.935+0000 7f21c92cd700 1 --2- 172.21.15.40:0/2662819844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c406b580 0x7f21c410f320 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:11.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.935+0000 7f21c92cd700 1 --2- 172.21.15.40:0/2662819844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21c406abc0 0x7f21c406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:11.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.935+0000 7f21c92cd700 1 -- 172.21.15.40:0/2662819844 >> 172.21.15.40:0/2662819844 conn(0x7f21c4078340 msgr2=0x7f21c4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:11.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.935+0000 7f21c92cd700 1 -- 172.21.15.40:0/2662819844 shutdown_connections 2024-06-06T03:37:11.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.935+0000 7f21c92cd700 1 -- 172.21.15.40:0/2662819844 wait complete. 2024-06-06T03:37:11.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.935+0000 7f21c92cd700 1 Processor -- start 2024-06-06T03:37:11.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.935+0000 7f21c92cd700 1 -- start start 2024-06-06T03:37:11.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c92cd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21c406abc0 0x7f21c4101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:11.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c92cd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c406b580 0x7f21c41019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:11.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c92cd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c4108500 0x7f21c4101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:11.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c92cd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f21c4112780 con 0x7f21c406b580 2024-06-06T03:37:11.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c92cd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f21c4112600 con 0x7f21c406abc0 2024-06-06T03:37:11.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c92cd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f21c4112900 con 0x7f21c4108500 2024-06-06T03:37:11.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c27fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c406b580 0x7f21c41019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:11.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c27fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c406b580 0x7f21c41019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36044/0 (socket says 172.21.15.40:36044) 2024-06-06T03:37:11.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c27fc700 1 -- 172.21.15.40:0/1849605276 learned_addr learned my addr 172.21.15.40:0/1849605276 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:11.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c2ffd700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21c406abc0 0x7f21c4101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:11.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c37fe700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c4108500 0x7f21c4101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:11.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c27fc700 1 -- 172.21.15.40:0/1849605276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c4108500 msgr2=0x7f21c4101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:11.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c27fc700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c4108500 0x7f21c4101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:11.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c27fc700 1 -- 172.21.15.40:0/1849605276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21c406abc0 msgr2=0x7f21c4101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:11.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c27fc700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21c406abc0 0x7f21c4101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:11.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c27fc700 1 -- 172.21.15.40:0/1849605276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c41a7100 con 0x7f21c406b580 2024-06-06T03:37:11.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c2ffd700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21c406abc0 0x7f21c4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:37:11.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.936+0000 7f21c27fc700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c406b580 0x7f21c41019d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f21b800d950 tx=0x7f21b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:11.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.937+0000 7f21abfff700 1 -- 172.21.15.40:0/1849605276 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8016070 con 0x7f21c406b580 2024-06-06T03:37:11.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.937+0000 7f21c92cd700 1 -- 172.21.15.40:0/1849605276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c41a73f0 con 0x7f21c406b580 2024-06-06T03:37:11.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.937+0000 7f21abfff700 1 -- 172.21.15.40:0/1849605276 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21b8009e20 con 0x7f21c406b580 2024-06-06T03:37:11.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.937+0000 7f21abfff700 1 -- 172.21.15.40:0/1849605276 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b800f620 con 0x7f21c406b580 2024-06-06T03:37:11.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.937+0000 7f21c92cd700 1 -- 172.21.15.40:0/1849605276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c41a7980 con 0x7f21c406b580 2024-06-06T03:37:11.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.938+0000 7f21c92cd700 1 -- 172.21.15.40:0/1849605276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c410a9e0 con 0x7f21c406b580 2024-06-06T03:37:11.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.939+0000 7f21abfff700 1 -- 172.21.15.40:0/1849605276 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f21b8019070 con 0x7f21c406b580 2024-06-06T03:37:11.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.939+0000 7f21abfff700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f21ac0a2cb0 0x7f21ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:11.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.940+0000 7f21c2ffd700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f21ac0a2cb0 0x7f21ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:11.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.940+0000 7f21abfff700 1 -- 172.21.15.40:0/1849605276 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f21b80cec80 con 0x7f21c406b580 2024-06-06T03:37:11.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.940+0000 7f21c2ffd700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f21ac0a2cb0 0x7f21ac0a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f21b4005e10 tx=0x7f21b4005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:11.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:11.944+0000 7f21abfff700 1 -- 172.21.15.40:0/1849605276 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f21b809bba0 con 0x7f21c406b580 2024-06-06T03:37:12.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:12.152+0000 7f21c92cd700 1 -- 172.21.15.40:0/1849605276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f21c41029c0 con 0x7f21c406b580 2024-06-06T03:37:12.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:12.153+0000 7f21abfff700 1 -- 172.21.15.40:0/1849605276 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f21b809b4a0 con 0x7f21c406b580 2024-06-06T03:37:12.153 INFO:teuthology.orchestra.run.smithi040.stdout:141733920844 2024-06-06T03:37:12.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:12.155+0000 7f21c92cd700 1 -- 172.21.15.40:0/1849605276 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f21ac0a2cb0 msgr2=0x7f21ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:12.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:12.155+0000 7f21c92cd700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f21ac0a2cb0 0x7f21ac0a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f21b4005e10 tx=0x7f21b4005d80 comp rx=0 tx=0).stop 2024-06-06T03:37:12.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:12.155+0000 7f21c92cd700 1 -- 172.21.15.40:0/1849605276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c406b580 msgr2=0x7f21c41019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:12.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:12.155+0000 7f21c92cd700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c406b580 0x7f21c41019d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f21b800d950 tx=0x7f21b800dc60 comp rx=0 tx=0).stop 2024-06-06T03:37:12.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:12.155+0000 7f21c92cd700 1 -- 172.21.15.40:0/1849605276 shutdown_connections 2024-06-06T03:37:12.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:12.155+0000 7f21c92cd700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f21ac0a2cb0 0x7f21ac0a5170 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:12.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:12.155+0000 7f21c92cd700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c4108500 0x7f21c4101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:12.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:12.155+0000 7f21c92cd700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21c406abc0 0x7f21c4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:12.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:12.155+0000 7f21c92cd700 1 --2- 172.21.15.40:0/1849605276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21c406b580 0x7f21c41019d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:12.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:12.155+0000 7f21c92cd700 1 -- 172.21.15.40:0/1849605276 >> 172.21.15.40:0/1849605276 conn(0x7f21c4078340 msgr2=0x7f21c4100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:12.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:12.155+0000 7f21c92cd700 1 -- 172.21.15.40:0/1849605276 shutdown_connections 2024-06-06T03:37:12.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:12.156+0000 7f21c92cd700 1 -- 172.21.15.40:0/1849605276 wait complete. 2024-06-06T03:37:12.465 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070515 got 253403070519 for osd.8 2024-06-06T03:37:12.466 DEBUG:teuthology.parallel:result is None 2024-06-06T03:37:13.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:12 smithi040 ceph-mon[32220]: pgmap v362: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:37:13.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:12 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1849605276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T03:37:13.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:12 smithi104 ceph-mon[36569]: pgmap v362: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:37:13.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:12 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1849605276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T03:37:13.178 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920838 got 141733920844 for osd.4 2024-06-06T03:37:13.179 DEBUG:teuthology.parallel:result is None 2024-06-06T03:37:13.179 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-06T03:37:13.179 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T03:37:13.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:12 smithi131 ceph-mon[36687]: pgmap v362: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:37:13.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:12 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1849605276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T03:37:13.434 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:14.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:14 smithi040 ceph-mon[32220]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:37:15.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:14 smithi104 ceph-mon[36569]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:37:15.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.117+0000 7f610d59b700 1 -- 172.21.15.40:0/2793416385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6108102740 msgr2=0x7f6108102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:15.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.117+0000 7f610d59b700 1 --2- 172.21.15.40:0/2793416385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6108102740 0x7f6108102b20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f60f80099c0 tx=0x7f60f8009cd0 comp rx=0 tx=0).stop 2024-06-06T03:37:15.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.117+0000 7f610d59b700 1 -- 172.21.15.40:0/2793416385 shutdown_connections 2024-06-06T03:37:15.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.117+0000 7f610d59b700 1 --2- 172.21.15.40:0/2793416385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6108107840 0x7f610810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:15.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.117+0000 7f610d59b700 1 --2- 172.21.15.40:0/2793416385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6108102740 0x7f6108102b20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:15.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.117+0000 7f610d59b700 1 --2- 172.21.15.40:0/2793416385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61081030f0 0x7f6108107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:15.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.117+0000 7f610d59b700 1 -- 172.21.15.40:0/2793416385 >> 172.21.15.40:0/2793416385 conn(0x7f61080795f0 msgr2=0x7f6108079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:15.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.117+0000 7f610d59b700 1 -- 172.21.15.40:0/2793416385 shutdown_connections 2024-06-06T03:37:15.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.117+0000 7f610d59b700 1 -- 172.21.15.40:0/2793416385 wait complete. 2024-06-06T03:37:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.118+0000 7f610d59b700 1 Processor -- start 2024-06-06T03:37:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.118+0000 7f610d59b700 1 -- start start 2024-06-06T03:37:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.118+0000 7f610d59b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6108102740 0x7f610819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.118+0000 7f610d59b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61081030f0 0x7f610819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.118+0000 7f610d59b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6108107840 0x7f6108198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.118+0000 7f610d59b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f61081127d0 con 0x7f6108102740 2024-06-06T03:37:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.118+0000 7f610d59b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6108112650 con 0x7f6108107840 2024-06-06T03:37:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.118+0000 7f610d59b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6108112950 con 0x7f61081030f0 2024-06-06T03:37:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.118+0000 7f6106ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6108102740 0x7f610819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.118+0000 7f6106ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6108102740 0x7f610819e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36066/0 (socket says 172.21.15.40:36066) 2024-06-06T03:37:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.118+0000 7f6106ffd700 1 -- 172.21.15.40:0/3541131473 learned_addr learned my addr 172.21.15.40:0/3541131473 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.119+0000 7f61067fc700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61081030f0 0x7f610819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.119+0000 7f61077fe700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6108107840 0x7f6108198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.119+0000 7f6106ffd700 1 -- 172.21.15.40:0/3541131473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61081030f0 msgr2=0x7f610819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.119+0000 7f6106ffd700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61081030f0 0x7f610819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.119+0000 7f6106ffd700 1 -- 172.21.15.40:0/3541131473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6108107840 msgr2=0x7f6108198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.119+0000 7f6106ffd700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6108107840 0x7f6108198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.119+0000 7f6106ffd700 1 -- 172.21.15.40:0/3541131473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6108199230 con 0x7f6108102740 2024-06-06T03:37:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.119+0000 7f61067fc700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61081030f0 0x7f610819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:37:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.119+0000 7f6106ffd700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6108102740 0x7f610819e8c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f60f80059d0 tx=0x7f60f80038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.119+0000 7f60effff700 1 -- 172.21.15.40:0/3541131473 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8027070 con 0x7f6108102740 2024-06-06T03:37:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.119+0000 7f60effff700 1 -- 172.21.15.40:0/3541131473 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f8004050 con 0x7f6108102740 2024-06-06T03:37:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.119+0000 7f610d59b700 1 -- 172.21.15.40:0/3541131473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61081994c0 con 0x7f6108102740 2024-06-06T03:37:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.119+0000 7f60effff700 1 -- 172.21.15.40:0/3541131473 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f800f460 con 0x7f6108102740 2024-06-06T03:37:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.119+0000 7f610d59b700 1 -- 172.21.15.40:0/3541131473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61081a5790 con 0x7f6108102740 2024-06-06T03:37:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.120+0000 7f610d59b700 1 -- 172.21.15.40:0/3541131473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6108069180 con 0x7f6108102740 2024-06-06T03:37:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.121+0000 7f60effff700 1 -- 172.21.15.40:0/3541131473 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f60f802a070 con 0x7f6108102740 2024-06-06T03:37:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.122+0000 7f60effff700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f60f00a2d30 0x7f60f00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.122+0000 7f61067fc700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f60f00a2d30 0x7f60f00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:15.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.122+0000 7f60effff700 1 -- 172.21.15.40:0/3541131473 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f60f80d07f0 con 0x7f6108102740 2024-06-06T03:37:15.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.122+0000 7f61067fc700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f60f00a2d30 0x7f60f00a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6108101fe0 tx=0x7f60fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:15.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.125+0000 7f60effff700 1 -- 172.21.15.40:0/3541131473 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f60f8031020 con 0x7f6108102740 2024-06-06T03:37:15.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:14 smithi131 ceph-mon[36687]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:37:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.328+0000 7f610d59b700 1 -- 172.21.15.40:0/3541131473 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6108199fb0 con 0x7f60f00a2d30 2024-06-06T03:37:15.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.336+0000 7f60effff700 1 -- 172.21.15.40:0/3541131473 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258946 (secure 0 0 0) 0x7f6108199fb0 con 0x7f60f00a2d30 2024-06-06T03:37:15.340 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:37:15.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.339+0000 7f610d59b700 1 -- 172.21.15.40:0/3541131473 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f60f00a2d30 msgr2=0x7f60f00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:15.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.339+0000 7f610d59b700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f60f00a2d30 0x7f60f00a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6108101fe0 tx=0x7f60fc008040 comp rx=0 tx=0).stop 2024-06-06T03:37:15.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.339+0000 7f610d59b700 1 -- 172.21.15.40:0/3541131473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6108102740 msgr2=0x7f610819e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:15.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.339+0000 7f610d59b700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6108102740 0x7f610819e8c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f60f80059d0 tx=0x7f60f80038b0 comp rx=0 tx=0).stop 2024-06-06T03:37:15.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.339+0000 7f610d59b700 1 -- 172.21.15.40:0/3541131473 shutdown_connections 2024-06-06T03:37:15.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.339+0000 7f610d59b700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f60f00a2d30 0x7f60f00a51f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:15.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.339+0000 7f610d59b700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6108107840 0x7f6108198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.339+0000 7f610d59b700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6108102740 0x7f610819e8c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.339+0000 7f610d59b700 1 --2- 172.21.15.40:0/3541131473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61081030f0 0x7f610819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.339+0000 7f610d59b700 1 -- 172.21.15.40:0/3541131473 >> 172.21.15.40:0/3541131473 conn(0x7f61080795f0 msgr2=0x7f6108108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.339+0000 7f610d59b700 1 -- 172.21.15.40:0/3541131473 shutdown_connections 2024-06-06T03:37:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:15.339+0000 7f610d59b700 1 -- 172.21.15.40:0/3541131473 wait complete. 2024-06-06T03:37:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-06-06T03:37:16.031 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:15 smithi040 ceph-mon[32220]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:37:16.031 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:15 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:37:16.060 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-06-06T03:37:13.708598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660138,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1143,"num_read_kb":697,"num_write":428,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365492,"kb_used_data":12156,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576332,"statfs":{"total":1151940427776,"available":1151566163968,"internally_reserved":0,"allocated":12447744,"data_stored":7728774,"data_compressed":107001,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003091"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987328+0000","last_change":"2024-06-06T03:35:13.854938+0000","last_active":"2024-06-06T03:35:48.987328+0000","last_peered":"2024-06-06T03:35:48.987328+0000","last_clean":"2024-06-06T03:35:48.987328+0000","last_became_active":"2024-06-06T03:35:13.853574+0000","last_became_peered":"2024-06-06T03:35:13.853574+0000","last_unstale":"2024-06-06T03:35:48.987328+0000","last_undegraded":"2024-06-06T03:35:48.987328+0000","last_fullsized":"2024-06-06T03:35:48.987328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:21:09.256002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990557+0000","last_change":"2024-06-06T03:35:13.858299+0000","last_active":"2024-06-06T03:35:48.990557+0000","last_peered":"2024-06-06T03:35:48.990557+0000","last_clean":"2024-06-06T03:35:48.990557+0000","last_became_active":"2024-06-06T03:35:13.857993+0000","last_became_peered":"2024-06-06T03:35:13.857993+0000","last_unstale":"2024-06-06T03:35:48.990557+0000","last_undegraded":"2024-06-06T03:35:48.990557+0000","last_fullsized":"2024-06-06T03:35:48.990557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.811056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990502+0000","last_change":"2024-06-06T03:35:13.852783+0000","last_active":"2024-06-06T03:35:48.990502+0000","last_peered":"2024-06-06T03:35:48.990502+0000","last_clean":"2024-06-06T03:35:48.990502+0000","last_became_active":"2024-06-06T03:35:13.852468+0000","last_became_peered":"2024-06-06T03:35:13.852468+0000","last_unstale":"2024-06-06T03:35:48.990502+0000","last_undegraded":"2024-06-06T03:35:48.990502+0000","last_fullsized":"2024-06-06T03:35:48.990502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:04:59.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.004048+0000","last_change":"2024-06-06T03:35:13.845223+0000","last_active":"2024-06-06T03:35:49.004048+0000","last_peered":"2024-06-06T03:35:49.004048+0000","last_clean":"2024-06-06T03:35:49.004048+0000","last_became_active":"2024-06-06T03:35:13.844854+0000","last_became_peered":"2024-06-06T03:35:13.844854+0000","last_unstale":"2024-06-06T03:35:49.004048+0000","last_undegraded":"2024-06-06T03:35:49.004048+0000","last_fullsized":"2024-06-06T03:35:49.004048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:35:20.058188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991571+0000","last_change":"2024-06-06T03:35:13.853703+0000","last_active":"2024-06-06T03:35:48.991571+0000","last_peered":"2024-06-06T03:35:48.991571+0000","last_clean":"2024-06-06T03:35:48.991571+0000","last_became_active":"2024-06-06T03:35:13.853358+0000","last_became_peered":"2024-06-06T03:35:13.853358+0000","last_unstale":"2024-06-06T03:35:48.991571+0000","last_undegraded":"2024-06-06T03:35:48.991571+0000","last_fullsized":"2024-06-06T03:35:48.991571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:23:39.092306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988483+0000","last_change":"2024-06-06T03:35:13.859414+0000","last_active":"2024-06-06T03:35:48.988483+0000","last_peered":"2024-06-06T03:35:48.988483+0000","last_clean":"2024-06-06T03:35:48.988483+0000","last_became_active":"2024-06-06T03:35:13.859038+0000","last_became_peered":"2024-06-06T03:35:13.859038+0000","last_unstale":"2024-06-06T03:35:48.988483+0000","last_undegraded":"2024-06-06T03:35:48.988483+0000","last_fullsized":"2024-06-06T03:35:48.988483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:14:31.780064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985258+0000","last_change":"2024-06-06T03:35:13.859245+0000","last_active":"2024-06-06T03:35:48.985258+0000","last_peered":"2024-06-06T03:35:48.985258+0000","last_clean":"2024-06-06T03:35:48.985258+0000","last_became_active":"2024-06-06T03:35:13.858857+0000","last_became_peered":"2024-06-06T03:35:13.858857+0000","last_unstale":"2024-06-06T03:35:48.985258+0000","last_undegraded":"2024-06-06T03:35:48.985258+0000","last_fullsized":"2024-06-06T03:35:48.985258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:03:23.240007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990694+0000","last_change":"2024-06-06T03:35:13.851249+0000","last_active":"2024-06-06T03:35:48.990694+0000","last_peered":"2024-06-06T03:35:48.990694+0000","last_clean":"2024-06-06T03:35:48.990694+0000","last_became_active":"2024-06-06T03:35:13.850777+0000","last_became_peered":"2024-06-06T03:35:13.850777+0000","last_unstale":"2024-06-06T03:35:48.990694+0000","last_undegraded":"2024-06-06T03:35:48.990694+0000","last_fullsized":"2024-06-06T03:35:48.990694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:53:48.463672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985036+0000","last_change":"2024-06-06T03:35:13.852950+0000","last_active":"2024-06-06T03:35:48.985036+0000","last_peered":"2024-06-06T03:35:48.985036+0000","last_clean":"2024-06-06T03:35:48.985036+0000","last_became_active":"2024-06-06T03:35:13.852160+0000","last_became_peered":"2024-06-06T03:35:13.852160+0000","last_unstale":"2024-06-06T03:35:48.985036+0000","last_undegraded":"2024-06-06T03:35:48.985036+0000","last_fullsized":"2024-06-06T03:35:48.985036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:00.498440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988970+0000","last_change":"2024-06-06T03:35:13.856081+0000","last_active":"2024-06-06T03:35:48.988970+0000","last_peered":"2024-06-06T03:35:48.988970+0000","last_clean":"2024-06-06T03:35:48.988970+0000","last_became_active":"2024-06-06T03:35:13.855718+0000","last_became_peered":"2024-06-06T03:35:13.855718+0000","last_unstale":"2024-06-06T03:35:48.988970+0000","last_undegraded":"2024-06-06T03:35:48.988970+0000","last_fullsized":"2024-06-06T03:35:48.988970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:33:38.171996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985594+0000","last_change":"2024-06-06T03:35:13.861850+0000","last_active":"2024-06-06T03:35:48.985594+0000","last_peered":"2024-06-06T03:35:48.985594+0000","last_clean":"2024-06-06T03:35:48.985594+0000","last_became_active":"2024-06-06T03:35:13.861513+0000","last_became_peered":"2024-06-06T03:35:13.861513+0000","last_unstale":"2024-06-06T03:35:48.985594+0000","last_undegraded":"2024-06-06T03:35:48.985594+0000","last_fullsized":"2024-06-06T03:35:48.985594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:32:45.938302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.003896+0000","last_change":"2024-06-06T03:35:13.847742+0000","last_active":"2024-06-06T03:35:49.003896+0000","last_peered":"2024-06-06T03:35:49.003896+0000","last_clean":"2024-06-06T03:35:49.003896+0000","last_became_active":"2024-06-06T03:35:13.847409+0000","last_became_peered":"2024-06-06T03:35:13.847409+0000","last_unstale":"2024-06-06T03:35:49.003896+0000","last_undegraded":"2024-06-06T03:35:49.003896+0000","last_fullsized":"2024-06-06T03:35:49.003896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:20:31.353919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983746+0000","last_change":"2024-06-06T03:35:13.848785+0000","last_active":"2024-06-06T03:35:48.983746+0000","last_peered":"2024-06-06T03:35:48.983746+0000","last_clean":"2024-06-06T03:35:48.983746+0000","last_became_active":"2024-06-06T03:35:13.848048+0000","last_became_peered":"2024-06-06T03:35:13.848048+0000","last_unstale":"2024-06-06T03:35:48.983746+0000","last_undegraded":"2024-06-06T03:35:48.983746+0000","last_fullsized":"2024-06-06T03:35:48.983746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:43.657761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990820+0000","last_change":"2024-06-06T03:35:13.856083+0000","last_active":"2024-06-06T03:35:48.990820+0000","last_peered":"2024-06-06T03:35:48.990820+0000","last_clean":"2024-06-06T03:35:48.990820+0000","last_became_active":"2024-06-06T03:35:13.854390+0000","last_became_peered":"2024-06-06T03:35:13.854390+0000","last_unstale":"2024-06-06T03:35:48.990820+0000","last_undegraded":"2024-06-06T03:35:48.990820+0000","last_fullsized":"2024-06-06T03:35:48.990820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:51:37.163814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990153+0000","last_change":"2024-06-06T03:35:13.852611+0000","last_active":"2024-06-06T03:35:48.990153+0000","last_peered":"2024-06-06T03:35:48.990153+0000","last_clean":"2024-06-06T03:35:48.990153+0000","last_became_active":"2024-06-06T03:35:13.852244+0000","last_became_peered":"2024-06-06T03:35:13.852244+0000","last_unstale":"2024-06-06T03:35:48.990153+0000","last_undegraded":"2024-06-06T03:35:48.990153+0000","last_fullsized":"2024-06-06T03:35:48.990153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:59:53.513150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984532+0000","last_change":"2024-06-06T03:35:13.857840+0000","last_active":"2024-06-06T03:35:48.984532+0000","last_peered":"2024-06-06T03:35:48.984532+0000","last_clean":"2024-06-06T03:35:48.984532+0000","last_became_active":"2024-06-06T03:35:13.857137+0000","last_became_peered":"2024-06-06T03:35:13.857137+0000","last_unstale":"2024-06-06T03:35:48.984532+0000","last_undegraded":"2024-06-06T03:35:48.984532+0000","last_fullsized":"2024-06-06T03:35:48.984532+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:42:07.924735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987452+0000","last_change":"2024-06-06T03:35:13.857033+0000","last_active":"2024-06-06T03:35:48.987452+0000","last_peered":"2024-06-06T03:35:48.987452+0000","last_clean":"2024-06-06T03:35:48.987452+0000","last_became_active":"2024-06-06T03:35:13.856519+0000","last_became_peered":"2024-06-06T03:35:13.856519+0000","last_unstale":"2024-06-06T03:35:48.987452+0000","last_undegraded":"2024-06-06T03:35:48.987452+0000","last_fullsized":"2024-06-06T03:35:48.987452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:05:25.668344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.986761+0000","last_change":"2024-06-06T03:35:13.851356+0000","last_active":"2024-06-06T03:35:48.986761+0000","last_peered":"2024-06-06T03:35:48.986761+0000","last_clean":"2024-06-06T03:35:48.986761+0000","last_became_active":"2024-06-06T03:35:13.851030+0000","last_became_peered":"2024-06-06T03:35:13.851030+0000","last_unstale":"2024-06-06T03:35:48.986761+0000","last_undegraded":"2024-06-06T03:35:48.986761+0000","last_fullsized":"2024-06-06T03:35:48.986761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:49:38.833022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991734+0000","last_change":"2024-06-06T03:35:13.858613+0000","last_active":"2024-06-06T03:35:48.991734+0000","last_peered":"2024-06-06T03:35:48.991734+0000","last_clean":"2024-06-06T03:35:48.991734+0000","last_became_active":"2024-06-06T03:35:13.858182+0000","last_became_peered":"2024-06-06T03:35:13.858182+0000","last_unstale":"2024-06-06T03:35:48.991734+0000","last_undegraded":"2024-06-06T03:35:48.991734+0000","last_fullsized":"2024-06-06T03:35:48.991734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:23:08.216431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":135,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:08.081745+0000","last_change":"2024-06-06T03:35:09.821896+0000","last_active":"2024-06-06T03:37:08.081745+0000","last_peered":"2024-06-06T03:37:08.081745+0000","last_clean":"2024-06-06T03:37:08.081745+0000","last_became_active":"2024-06-06T03:35:09.821567+0000","last_became_peered":"2024-06-06T03:35:09.821567+0000","last_unstale":"2024-06-06T03:37:08.081745+0000","last_undegraded":"2024-06-06T03:37:08.081745+0000","last_fullsized":"2024-06-06T03:37:08.081745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:04:51.918393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991407+0000","last_change":"2024-06-06T03:35:13.857357+0000","last_active":"2024-06-06T03:35:48.991407+0000","last_peered":"2024-06-06T03:35:48.991407+0000","last_clean":"2024-06-06T03:35:48.991407+0000","last_became_active":"2024-06-06T03:35:13.857016+0000","last_became_peered":"2024-06-06T03:35:13.857016+0000","last_unstale":"2024-06-06T03:35:48.991407+0000","last_undegraded":"2024-06-06T03:35:48.991407+0000","last_fullsized":"2024-06-06T03:35:48.991407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:42.566425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991487+0000","last_change":"2024-06-06T03:35:10.277316+0000","last_active":"2024-06-06T03:35:48.991487+0000","last_peered":"2024-06-06T03:35:48.991487+0000","last_clean":"2024-06-06T03:35:48.991487+0000","last_became_active":"2024-06-06T03:35:10.276986+0000","last_became_peered":"2024-06-06T03:35:10.276986+0000","last_unstale":"2024-06-06T03:35:48.991487+0000","last_undegraded":"2024-06-06T03:35:48.991487+0000","last_fullsized":"2024-06-06T03:35:48.991487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:17:29.034224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990383+0000","last_change":"2024-06-06T03:35:13.863069+0000","last_active":"2024-06-06T03:35:48.990383+0000","last_peered":"2024-06-06T03:35:48.990383+0000","last_clean":"2024-06-06T03:35:48.990383+0000","last_became_active":"2024-06-06T03:35:13.862647+0000","last_became_peered":"2024-06-06T03:35:13.862647+0000","last_unstale":"2024-06-06T03:35:48.990383+0000","last_undegraded":"2024-06-06T03:35:48.990383+0000","last_fullsized":"2024-06-06T03:35:48.990383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:48.252606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992306+0000","last_change":"2024-06-06T03:35:09.824977+0000","last_active":"2024-06-06T03:35:48.992306+0000","last_peered":"2024-06-06T03:35:48.992306+0000","last_clean":"2024-06-06T03:35:48.992306+0000","last_became_active":"2024-06-06T03:35:09.824643+0000","last_became_peered":"2024-06-06T03:35:09.824643+0000","last_unstale":"2024-06-06T03:35:48.992306+0000","last_undegraded":"2024-06-06T03:35:48.992306+0000","last_fullsized":"2024-06-06T03:35:48.992306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:35:37.208023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984591+0000","last_change":"2024-06-06T03:35:13.860572+0000","last_active":"2024-06-06T03:35:48.984591+0000","last_peered":"2024-06-06T03:35:48.984591+0000","last_clean":"2024-06-06T03:35:48.984591+0000","last_became_active":"2024-06-06T03:35:13.858398+0000","last_became_peered":"2024-06-06T03:35:13.858398+0000","last_unstale":"2024-06-06T03:35:48.984591+0000","last_undegraded":"2024-06-06T03:35:48.984591+0000","last_fullsized":"2024-06-06T03:35:48.984591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:02:12.683096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991795+0000","last_change":"2024-06-06T03:35:09.827016+0000","last_active":"2024-06-06T03:35:48.991795+0000","last_peered":"2024-06-06T03:35:48.991795+0000","last_clean":"2024-06-06T03:35:48.991795+0000","last_became_active":"2024-06-06T03:35:09.826576+0000","last_became_peered":"2024-06-06T03:35:09.826576+0000","last_unstale":"2024-06-06T03:35:48.991795+0000","last_undegraded":"2024-06-06T03:35:48.991795+0000","last_fullsized":"2024-06-06T03:35:48.991795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:24.120549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984420+0000","last_change":"2024-06-06T03:35:13.853988+0000","last_active":"2024-06-06T03:35:48.984420+0000","last_peered":"2024-06-06T03:35:48.984420+0000","last_clean":"2024-06-06T03:35:48.984420+0000","last_became_active":"2024-06-06T03:35:13.852343+0000","last_became_peered":"2024-06-06T03:35:13.852343+0000","last_unstale":"2024-06-06T03:35:48.984420+0000","last_undegraded":"2024-06-06T03:35:48.984420+0000","last_fullsized":"2024-06-06T03:35:48.984420+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:39:03.054045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":129,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:07.081441+0000","last_change":"2024-06-06T03:35:09.829109+0000","last_active":"2024-06-06T03:37:07.081441+0000","last_peered":"2024-06-06T03:37:07.081441+0000","last_clean":"2024-06-06T03:37:07.081441+0000","last_became_active":"2024-06-06T03:35:09.828734+0000","last_became_peered":"2024-06-06T03:35:09.828734+0000","last_unstale":"2024-06-06T03:37:07.081441+0000","last_undegraded":"2024-06-06T03:37:07.081441+0000","last_fullsized":"2024-06-06T03:37:07.081441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:50.475600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992478+0000","last_change":"2024-06-06T03:35:13.859198+0000","last_active":"2024-06-06T03:35:48.992478+0000","last_peered":"2024-06-06T03:35:48.992478+0000","last_clean":"2024-06-06T03:35:48.992478+0000","last_became_active":"2024-06-06T03:35:13.858863+0000","last_became_peered":"2024-06-06T03:35:13.858863+0000","last_unstale":"2024-06-06T03:35:48.992478+0000","last_undegraded":"2024-06-06T03:35:48.992478+0000","last_fullsized":"2024-06-06T03:35:48.992478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:57.153174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988385+0000","last_change":"2024-06-06T03:35:09.824352+0000","last_active":"2024-06-06T03:35:48.988385+0000","last_peered":"2024-06-06T03:35:48.988385+0000","last_clean":"2024-06-06T03:35:48.988385+0000","last_became_active":"2024-06-06T03:35:09.823998+0000","last_became_peered":"2024-06-06T03:35:09.823998+0000","last_unstale":"2024-06-06T03:35:48.988385+0000","last_undegraded":"2024-06-06T03:35:48.988385+0000","last_fullsized":"2024-06-06T03:35:48.988385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:47:24.646067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984826+0000","last_change":"2024-06-06T03:35:13.836724+0000","last_active":"2024-06-06T03:35:48.984826+0000","last_peered":"2024-06-06T03:35:48.984826+0000","last_clean":"2024-06-06T03:35:48.984826+0000","last_became_active":"2024-06-06T03:35:13.836180+0000","last_became_peered":"2024-06-06T03:35:13.836180+0000","last_unstale":"2024-06-06T03:35:48.984826+0000","last_undegraded":"2024-06-06T03:35:48.984826+0000","last_fullsized":"2024-06-06T03:35:48.984826+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:08.868462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983408+0000","last_change":"2024-06-06T03:35:09.823047+0000","last_active":"2024-06-06T03:35:48.983408+0000","last_peered":"2024-06-06T03:35:48.983408+0000","last_clean":"2024-06-06T03:35:48.983408+0000","last_became_active":"2024-06-06T03:35:09.822744+0000","last_became_peered":"2024-06-06T03:35:09.822744+0000","last_unstale":"2024-06-06T03:35:48.983408+0000","last_undegraded":"2024-06-06T03:35:48.983408+0000","last_fullsized":"2024-06-06T03:35:48.983408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:20:20.165682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.989848+0000","last_change":"2024-06-06T03:35:13.857222+0000","last_active":"2024-06-06T03:35:48.989848+0000","last_peered":"2024-06-06T03:35:48.989848+0000","last_clean":"2024-06-06T03:35:48.989848+0000","last_became_active":"2024-06-06T03:35:13.856873+0000","last_became_peered":"2024-06-06T03:35:13.856873+0000","last_unstale":"2024-06-06T03:35:48.989848+0000","last_undegraded":"2024-06-06T03:35:48.989848+0000","last_fullsized":"2024-06-06T03:35:48.989848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:32:16.646275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984083+0000","last_change":"2024-06-06T03:35:09.826036+0000","last_active":"2024-06-06T03:35:48.984083+0000","last_peered":"2024-06-06T03:35:48.984083+0000","last_clean":"2024-06-06T03:35:48.984083+0000","last_became_active":"2024-06-06T03:35:09.825690+0000","last_became_peered":"2024-06-06T03:35:09.825690+0000","last_unstale":"2024-06-06T03:35:48.984083+0000","last_undegraded":"2024-06-06T03:35:48.984083+0000","last_fullsized":"2024-06-06T03:35:48.984083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:03:35.858828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.986086+0000","last_change":"2024-06-06T03:35:13.849016+0000","last_active":"2024-06-06T03:35:48.986086+0000","last_peered":"2024-06-06T03:35:48.986086+0000","last_clean":"2024-06-06T03:35:48.986086+0000","last_became_active":"2024-06-06T03:35:13.848709+0000","last_became_peered":"2024-06-06T03:35:13.848709+0000","last_unstale":"2024-06-06T03:35:48.986086+0000","last_undegraded":"2024-06-06T03:35:48.986086+0000","last_fullsized":"2024-06-06T03:35:48.986086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:42.778612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":224,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:11.086611+0000","last_change":"2024-06-06T03:35:09.827072+0000","last_active":"2024-06-06T03:37:11.086611+0000","last_peered":"2024-06-06T03:37:11.086611+0000","last_clean":"2024-06-06T03:37:11.086611+0000","last_became_active":"2024-06-06T03:35:09.826734+0000","last_became_peered":"2024-06-06T03:35:09.826734+0000","last_unstale":"2024-06-06T03:37:11.086611+0000","last_undegraded":"2024-06-06T03:37:11.086611+0000","last_fullsized":"2024-06-06T03:37:11.086611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:07:13.388398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985601+0000","last_change":"2024-06-06T03:35:13.856883+0000","last_active":"2024-06-06T03:35:48.985601+0000","last_peered":"2024-06-06T03:35:48.985601+0000","last_clean":"2024-06-06T03:35:48.985601+0000","last_became_active":"2024-06-06T03:35:13.856521+0000","last_became_peered":"2024-06-06T03:35:13.856521+0000","last_unstale":"2024-06-06T03:35:48.985601+0000","last_undegraded":"2024-06-06T03:35:48.985601+0000","last_fullsized":"2024-06-06T03:35:48.985601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:19.466218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985292+0000","last_change":"2024-06-06T03:35:09.829153+0000","last_active":"2024-06-06T03:35:48.985292+0000","last_peered":"2024-06-06T03:35:48.985292+0000","last_clean":"2024-06-06T03:35:48.985292+0000","last_became_active":"2024-06-06T03:35:09.828797+0000","last_became_peered":"2024-06-06T03:35:09.828797+0000","last_unstale":"2024-06-06T03:35:48.985292+0000","last_undegraded":"2024-06-06T03:35:48.985292+0000","last_fullsized":"2024-06-06T03:35:48.985292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:06:02.986396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990268+0000","last_change":"2024-06-06T03:35:13.857094+0000","last_active":"2024-06-06T03:35:48.990268+0000","last_peered":"2024-06-06T03:35:48.990268+0000","last_clean":"2024-06-06T03:35:48.990268+0000","last_became_active":"2024-06-06T03:35:13.856778+0000","last_became_peered":"2024-06-06T03:35:13.856778+0000","last_unstale":"2024-06-06T03:35:48.990268+0000","last_undegraded":"2024-06-06T03:35:48.990268+0000","last_fullsized":"2024-06-06T03:35:48.990268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:50:30.097762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984506+0000","last_change":"2024-06-06T03:35:10.277802+0000","last_active":"2024-06-06T03:35:48.984506+0000","last_peered":"2024-06-06T03:35:48.984506+0000","last_clean":"2024-06-06T03:35:48.984506+0000","last_became_active":"2024-06-06T03:35:10.277417+0000","last_became_peered":"2024-06-06T03:35:10.277417+0000","last_unstale":"2024-06-06T03:35:48.984506+0000","last_undegraded":"2024-06-06T03:35:48.984506+0000","last_fullsized":"2024-06-06T03:35:48.984506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:44:35.178184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984699+0000","last_change":"2024-06-06T03:35:13.858160+0000","last_active":"2024-06-06T03:35:48.984699+0000","last_peered":"2024-06-06T03:35:48.984699+0000","last_clean":"2024-06-06T03:35:48.984699+0000","last_became_active":"2024-06-06T03:35:13.853149+0000","last_became_peered":"2024-06-06T03:35:13.853149+0000","last_unstale":"2024-06-06T03:35:48.984699+0000","last_undegraded":"2024-06-06T03:35:48.984699+0000","last_fullsized":"2024-06-06T03:35:48.984699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:10:06.697274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'31","reported_seq":260,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:11.086580+0000","last_change":"2024-06-06T03:35:09.829443+0000","last_active":"2024-06-06T03:37:11.086580+0000","last_peered":"2024-06-06T03:37:11.086580+0000","last_clean":"2024-06-06T03:37:11.086580+0000","last_became_active":"2024-06-06T03:35:09.828093+0000","last_became_peered":"2024-06-06T03:35:09.828093+0000","last_unstale":"2024-06-06T03:37:11.086580+0000","last_undegraded":"2024-06-06T03:37:11.086580+0000","last_fullsized":"2024-06-06T03:37:11.086580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:05:19.395177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62334,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":203,"num_read_kb":162,"num_write":33,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991968+0000","last_change":"2024-06-06T03:35:13.850869+0000","last_active":"2024-06-06T03:35:48.991968+0000","last_peered":"2024-06-06T03:35:48.991968+0000","last_clean":"2024-06-06T03:35:48.991968+0000","last_became_active":"2024-06-06T03:35:13.849489+0000","last_became_peered":"2024-06-06T03:35:13.849489+0000","last_unstale":"2024-06-06T03:35:48.991968+0000","last_undegraded":"2024-06-06T03:35:48.991968+0000","last_fullsized":"2024-06-06T03:35:48.991968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:27:45.052033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991491+0000","last_change":"2024-06-06T03:35:09.823117+0000","last_active":"2024-06-06T03:35:48.991491+0000","last_peered":"2024-06-06T03:35:48.991491+0000","last_clean":"2024-06-06T03:35:48.991491+0000","last_became_active":"2024-06-06T03:35:09.822624+0000","last_became_peered":"2024-06-06T03:35:09.822624+0000","last_unstale":"2024-06-06T03:35:48.991491+0000","last_undegraded":"2024-06-06T03:35:48.991491+0000","last_fullsized":"2024-06-06T03:35:48.991491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:59.171343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983437+0000","last_change":"2024-06-06T03:35:13.861043+0000","last_active":"2024-06-06T03:35:48.983437+0000","last_peered":"2024-06-06T03:35:48.983437+0000","last_clean":"2024-06-06T03:35:48.983437+0000","last_became_active":"2024-06-06T03:35:13.860718+0000","last_became_peered":"2024-06-06T03:35:13.860718+0000","last_unstale":"2024-06-06T03:35:48.983437+0000","last_undegraded":"2024-06-06T03:35:48.983437+0000","last_fullsized":"2024-06-06T03:35:48.983437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:34:04.595780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984355+0000","last_change":"2024-06-06T03:35:09.825505+0000","last_active":"2024-06-06T03:35:48.984355+0000","last_peered":"2024-06-06T03:35:48.984355+0000","last_clean":"2024-06-06T03:35:48.984355+0000","last_became_active":"2024-06-06T03:35:09.825127+0000","last_became_peered":"2024-06-06T03:35:09.825127+0000","last_unstale":"2024-06-06T03:35:48.984355+0000","last_undegraded":"2024-06-06T03:35:48.984355+0000","last_fullsized":"2024-06-06T03:35:48.984355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:07:27.664589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987834+0000","last_change":"2024-06-06T03:35:48.987834+0000","last_active":"2024-06-06T03:35:48.987834+0000","last_peered":"2024-06-06T03:35:48.987834+0000","last_clean":"2024-06-06T03:35:48.987834+0000","last_became_active":"2024-06-06T03:35:48.987515+0000","last_became_peered":"2024-06-06T03:35:48.987515+0000","last_unstale":"2024-06-06T03:35:48.987834+0000","last_undegraded":"2024-06-06T03:35:48.987834+0000","last_fullsized":"2024-06-06T03:35:48.987834+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:05:19.342289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":132,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:12.088111+0000","last_change":"2024-06-06T03:35:09.826192+0000","last_active":"2024-06-06T03:37:12.088111+0000","last_peered":"2024-06-06T03:37:12.088111+0000","last_clean":"2024-06-06T03:37:12.088111+0000","last_became_active":"2024-06-06T03:35:09.825481+0000","last_became_peered":"2024-06-06T03:35:09.825481+0000","last_unstale":"2024-06-06T03:37:12.088111+0000","last_undegraded":"2024-06-06T03:37:12.088111+0000","last_fullsized":"2024-06-06T03:37:12.088111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:18:30.772991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985035+0000","last_change":"2024-06-06T03:35:13.853113+0000","last_active":"2024-06-06T03:35:48.985035+0000","last_peered":"2024-06-06T03:35:48.985035+0000","last_clean":"2024-06-06T03:35:48.985035+0000","last_became_active":"2024-06-06T03:35:13.852751+0000","last_became_peered":"2024-06-06T03:35:13.852751+0000","last_unstale":"2024-06-06T03:35:48.985035+0000","last_undegraded":"2024-06-06T03:35:48.985035+0000","last_fullsized":"2024-06-06T03:35:48.985035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:53.536685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.006465+0000","last_change":"2024-06-06T03:35:49.005958+0000","last_active":"2024-06-06T03:35:49.006465+0000","last_peered":"2024-06-06T03:35:49.006465+0000","last_clean":"2024-06-06T03:35:49.006465+0000","last_became_active":"2024-06-06T03:35:48.993266+0000","last_became_peered":"2024-06-06T03:35:48.993266+0000","last_unstale":"2024-06-06T03:35:49.006465+0000","last_undegraded":"2024-06-06T03:35:49.006465+0000","last_fullsized":"2024-06-06T03:35:49.006465+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:47.782824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984158+0000","last_change":"2024-06-06T03:35:13.853614+0000","last_active":"2024-06-06T03:35:48.984158+0000","last_peered":"2024-06-06T03:35:48.984158+0000","last_clean":"2024-06-06T03:35:48.984158+0000","last_became_active":"2024-06-06T03:35:13.852327+0000","last_became_peered":"2024-06-06T03:35:13.852327+0000","last_unstale":"2024-06-06T03:35:48.984158+0000","last_undegraded":"2024-06-06T03:35:48.984158+0000","last_fullsized":"2024-06-06T03:35:48.984158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:02:32.463028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991648+0000","last_change":"2024-06-06T03:35:09.826220+0000","last_active":"2024-06-06T03:35:48.991648+0000","last_peered":"2024-06-06T03:35:48.991648+0000","last_clean":"2024-06-06T03:35:48.991648+0000","last_became_active":"2024-06-06T03:35:09.825887+0000","last_became_peered":"2024-06-06T03:35:09.825887+0000","last_unstale":"2024-06-06T03:35:48.991648+0000","last_undegraded":"2024-06-06T03:35:48.991648+0000","last_fullsized":"2024-06-06T03:35:48.991648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:23:33.704822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987180+0000","last_change":"2024-06-06T03:35:13.852455+0000","last_active":"2024-06-06T03:35:48.987180+0000","last_peered":"2024-06-06T03:35:48.987180+0000","last_clean":"2024-06-06T03:35:48.987180+0000","last_became_active":"2024-06-06T03:35:13.852134+0000","last_became_peered":"2024-06-06T03:35:13.852134+0000","last_unstale":"2024-06-06T03:35:48.987180+0000","last_undegraded":"2024-06-06T03:35:48.987180+0000","last_fullsized":"2024-06-06T03:35:48.987180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:00:16.418479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.986584+0000","last_change":"2024-06-06T03:35:10.277767+0000","last_active":"2024-06-06T03:35:48.986584+0000","last_peered":"2024-06-06T03:35:48.986584+0000","last_clean":"2024-06-06T03:35:48.986584+0000","last_became_active":"2024-06-06T03:35:10.277381+0000","last_became_peered":"2024-06-06T03:35:10.277381+0000","last_unstale":"2024-06-06T03:35:48.986584+0000","last_undegraded":"2024-06-06T03:35:48.986584+0000","last_fullsized":"2024-06-06T03:35:48.986584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:12:13.425284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988580+0000","last_change":"2024-06-06T03:35:13.859594+0000","last_active":"2024-06-06T03:35:48.988580+0000","last_peered":"2024-06-06T03:35:48.988580+0000","last_clean":"2024-06-06T03:35:48.988580+0000","last_became_active":"2024-06-06T03:35:13.859240+0000","last_became_peered":"2024-06-06T03:35:13.859240+0000","last_unstale":"2024-06-06T03:35:48.988580+0000","last_undegraded":"2024-06-06T03:35:48.988580+0000","last_fullsized":"2024-06-06T03:35:48.988580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:39:34.557581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":130,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:09.083940+0000","last_change":"2024-06-06T03:35:09.826204+0000","last_active":"2024-06-06T03:37:09.083940+0000","last_peered":"2024-06-06T03:37:09.083940+0000","last_clean":"2024-06-06T03:37:09.083940+0000","last_became_active":"2024-06-06T03:35:09.825815+0000","last_became_peered":"2024-06-06T03:35:09.825815+0000","last_unstale":"2024-06-06T03:37:09.083940+0000","last_undegraded":"2024-06-06T03:37:09.083940+0000","last_fullsized":"2024-06-06T03:37:09.083940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:02:04.169227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992786+0000","last_change":"2024-06-06T03:35:48.992786+0000","last_active":"2024-06-06T03:35:48.992786+0000","last_peered":"2024-06-06T03:35:48.992786+0000","last_clean":"2024-06-06T03:35:48.992786+0000","last_became_active":"2024-06-06T03:35:48.992456+0000","last_became_peered":"2024-06-06T03:35:48.992456+0000","last_unstale":"2024-06-06T03:35:48.992786+0000","last_undegraded":"2024-06-06T03:35:48.992786+0000","last_fullsized":"2024-06-06T03:35:48.992786+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:06:58.563975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.005564+0000","last_change":"2024-06-06T03:35:49.005119+0000","last_active":"2024-06-06T03:35:49.005564+0000","last_peered":"2024-06-06T03:35:49.005564+0000","last_clean":"2024-06-06T03:35:49.005564+0000","last_became_active":"2024-06-06T03:35:49.004788+0000","last_became_peered":"2024-06-06T03:35:49.004788+0000","last_unstale":"2024-06-06T03:35:49.005564+0000","last_undegraded":"2024-06-06T03:35:49.005564+0000","last_fullsized":"2024-06-06T03:35:49.005564+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:20:06.961454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":128,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:09.083523+0000","last_change":"2024-06-06T03:35:10.278559+0000","last_active":"2024-06-06T03:37:09.083523+0000","last_peered":"2024-06-06T03:37:09.083523+0000","last_clean":"2024-06-06T03:37:09.083523+0000","last_became_active":"2024-06-06T03:35:10.278104+0000","last_became_peered":"2024-06-06T03:35:10.278104+0000","last_unstale":"2024-06-06T03:37:09.083523+0000","last_undegraded":"2024-06-06T03:37:09.083523+0000","last_fullsized":"2024-06-06T03:37:09.083523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:23:47.131731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990973+0000","last_change":"2024-06-06T03:35:13.862303+0000","last_active":"2024-06-06T03:35:48.990973+0000","last_peered":"2024-06-06T03:35:48.990973+0000","last_clean":"2024-06-06T03:35:48.990973+0000","last_became_active":"2024-06-06T03:35:13.861978+0000","last_became_peered":"2024-06-06T03:35:13.861978+0000","last_unstale":"2024-06-06T03:35:48.990973+0000","last_undegraded":"2024-06-06T03:35:48.990973+0000","last_fullsized":"2024-06-06T03:35:48.990973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:24:00.824646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.004144+0000","last_change":"2024-06-06T03:35:09.825365+0000","last_active":"2024-06-06T03:35:49.004144+0000","last_peered":"2024-06-06T03:35:49.004144+0000","last_clean":"2024-06-06T03:35:49.004144+0000","last_became_active":"2024-06-06T03:35:09.824948+0000","last_became_peered":"2024-06-06T03:35:09.824948+0000","last_unstale":"2024-06-06T03:35:49.004144+0000","last_undegraded":"2024-06-06T03:35:49.004144+0000","last_fullsized":"2024-06-06T03:35:49.004144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:23:48.189292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":359,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985318+0000","last_change":"2024-06-06T03:33:44.124626+0000","last_active":"2024-06-06T03:35:48.985318+0000","last_peered":"2024-06-06T03:35:48.985318+0000","last_clean":"2024-06-06T03:35:48.985318+0000","last_became_active":"2024-06-06T03:33:44.124046+0000","last_became_peered":"2024-06-06T03:33:44.124046+0000","last_unstale":"2024-06-06T03:35:48.985318+0000","last_undegraded":"2024-06-06T03:35:48.985318+0000","last_fullsized":"2024-06-06T03:35:48.985318+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:30:14.586719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:30:14.586719+0000","last_clean_scrub_stamp":"2024-06-06T03:30:14.586719+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:17.604180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983301+0000","last_change":"2024-06-06T03:35:13.852044+0000","last_active":"2024-06-06T03:35:48.983301+0000","last_peered":"2024-06-06T03:35:48.983301+0000","last_clean":"2024-06-06T03:35:48.983301+0000","last_became_active":"2024-06-06T03:35:13.851715+0000","last_became_peered":"2024-06-06T03:35:13.851715+0000","last_unstale":"2024-06-06T03:35:48.983301+0000","last_undegraded":"2024-06-06T03:35:48.983301+0000","last_fullsized":"2024-06-06T03:35:48.983301+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:19:23.778367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985500+0000","last_change":"2024-06-06T03:35:09.828463+0000","last_active":"2024-06-06T03:35:48.985500+0000","last_peered":"2024-06-06T03:35:48.985500+0000","last_clean":"2024-06-06T03:35:48.985500+0000","last_became_active":"2024-06-06T03:35:09.828121+0000","last_became_peered":"2024-06-06T03:35:09.828121+0000","last_unstale":"2024-06-06T03:35:48.985500+0000","last_undegraded":"2024-06-06T03:35:48.985500+0000","last_fullsized":"2024-06-06T03:35:48.985500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:54:25.095741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992104+0000","last_change":"2024-06-06T03:35:10.277901+0000","last_active":"2024-06-06T03:35:48.992104+0000","last_peered":"2024-06-06T03:35:48.992104+0000","last_clean":"2024-06-06T03:35:48.992104+0000","last_became_active":"2024-06-06T03:35:10.277447+0000","last_became_peered":"2024-06-06T03:35:10.277447+0000","last_unstale":"2024-06-06T03:35:48.992104+0000","last_undegraded":"2024-06-06T03:35:48.992104+0000","last_fullsized":"2024-06-06T03:35:48.992104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:22:50.550287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983583+0000","last_change":"2024-06-06T03:35:13.864323+0000","last_active":"2024-06-06T03:35:48.983583+0000","last_peered":"2024-06-06T03:35:48.983583+0000","last_clean":"2024-06-06T03:35:48.983583+0000","last_became_active":"2024-06-06T03:35:13.863101+0000","last_became_peered":"2024-06-06T03:35:13.863101+0000","last_unstale":"2024-06-06T03:35:48.983583+0000","last_undegraded":"2024-06-06T03:35:48.983583+0000","last_fullsized":"2024-06-06T03:35:48.983583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:32.399325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:57.869408+0000","last_change":"2024-06-06T03:35:09.822280+0000","last_active":"2024-06-06T03:35:57.869408+0000","last_peered":"2024-06-06T03:35:57.869408+0000","last_clean":"2024-06-06T03:35:57.869408+0000","last_became_active":"2024-06-06T03:35:09.821922+0000","last_became_peered":"2024-06-06T03:35:09.821922+0000","last_unstale":"2024-06-06T03:35:57.869408+0000","last_undegraded":"2024-06-06T03:35:57.869408+0000","last_fullsized":"2024-06-06T03:35:57.869408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:00:03.328747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991350+0000","last_change":"2024-06-06T03:35:13.853538+0000","last_active":"2024-06-06T03:35:48.991350+0000","last_peered":"2024-06-06T03:35:48.991350+0000","last_clean":"2024-06-06T03:35:48.991350+0000","last_became_active":"2024-06-06T03:35:13.853200+0000","last_became_peered":"2024-06-06T03:35:13.853200+0000","last_unstale":"2024-06-06T03:35:48.991350+0000","last_undegraded":"2024-06-06T03:35:48.991350+0000","last_fullsized":"2024-06-06T03:35:48.991350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:13:06.574272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.004336+0000","last_change":"2024-06-06T03:35:10.761675+0000","last_active":"2024-06-06T03:35:49.004336+0000","last_peered":"2024-06-06T03:35:49.004336+0000","last_clean":"2024-06-06T03:35:49.004336+0000","last_became_active":"2024-06-06T03:35:10.761287+0000","last_became_peered":"2024-06-06T03:35:10.761287+0000","last_unstale":"2024-06-06T03:35:49.004336+0000","last_undegraded":"2024-06-06T03:35:49.004336+0000","last_fullsized":"2024-06-06T03:35:49.004336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:58:18.524273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990632+0000","last_change":"2024-06-06T03:35:13.847680+0000","last_active":"2024-06-06T03:35:48.990632+0000","last_peered":"2024-06-06T03:35:48.990632+0000","last_clean":"2024-06-06T03:35:48.990632+0000","last_became_active":"2024-06-06T03:35:13.847359+0000","last_became_peered":"2024-06-06T03:35:13.847359+0000","last_unstale":"2024-06-06T03:35:48.990632+0000","last_undegraded":"2024-06-06T03:35:48.990632+0000","last_fullsized":"2024-06-06T03:35:48.990632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:29:18.041078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984544+0000","last_change":"2024-06-06T03:35:10.277491+0000","last_active":"2024-06-06T03:35:48.984544+0000","last_peered":"2024-06-06T03:35:48.984544+0000","last_clean":"2024-06-06T03:35:48.984544+0000","last_became_active":"2024-06-06T03:35:10.277135+0000","last_became_peered":"2024-06-06T03:35:10.277135+0000","last_unstale":"2024-06-06T03:35:48.984544+0000","last_undegraded":"2024-06-06T03:35:48.984544+0000","last_fullsized":"2024-06-06T03:35:48.984544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:55.198789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991400+0000","last_change":"2024-06-06T03:35:13.846013+0000","last_active":"2024-06-06T03:35:48.991400+0000","last_peered":"2024-06-06T03:35:48.991400+0000","last_clean":"2024-06-06T03:35:48.991400+0000","last_became_active":"2024-06-06T03:35:13.845650+0000","last_became_peered":"2024-06-06T03:35:13.845650+0000","last_unstale":"2024-06-06T03:35:48.991400+0000","last_undegraded":"2024-06-06T03:35:48.991400+0000","last_fullsized":"2024-06-06T03:35:48.991400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:51.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990583+0000","last_change":"2024-06-06T03:35:09.826380+0000","last_active":"2024-06-06T03:35:48.990583+0000","last_peered":"2024-06-06T03:35:48.990583+0000","last_clean":"2024-06-06T03:35:48.990583+0000","last_became_active":"2024-06-06T03:35:09.825813+0000","last_became_peered":"2024-06-06T03:35:09.825813+0000","last_unstale":"2024-06-06T03:35:48.990583+0000","last_undegraded":"2024-06-06T03:35:48.990583+0000","last_fullsized":"2024-06-06T03:35:48.990583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:24.792398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987048+0000","last_change":"2024-06-06T03:35:13.855051+0000","last_active":"2024-06-06T03:35:48.987048+0000","last_peered":"2024-06-06T03:35:48.987048+0000","last_clean":"2024-06-06T03:35:48.987048+0000","last_became_active":"2024-06-06T03:35:13.853781+0000","last_became_peered":"2024-06-06T03:35:13.853781+0000","last_unstale":"2024-06-06T03:35:48.987048+0000","last_undegraded":"2024-06-06T03:35:48.987048+0000","last_fullsized":"2024-06-06T03:35:48.987048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:17:56.174924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.009988+0000","last_change":"2024-06-06T03:35:49.009583+0000","last_active":"2024-06-06T03:35:49.009988+0000","last_peered":"2024-06-06T03:35:49.009988+0000","last_clean":"2024-06-06T03:35:49.009988+0000","last_became_active":"2024-06-06T03:35:48.994275+0000","last_became_peered":"2024-06-06T03:35:48.994275+0000","last_unstale":"2024-06-06T03:35:49.009988+0000","last_undegraded":"2024-06-06T03:35:49.009988+0000","last_fullsized":"2024-06-06T03:35:49.009988+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:24:54.985854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991216+0000","last_change":"2024-06-06T03:35:13.855503+0000","last_active":"2024-06-06T03:35:48.991216+0000","last_peered":"2024-06-06T03:35:48.991216+0000","last_clean":"2024-06-06T03:35:48.991216+0000","last_became_active":"2024-06-06T03:35:13.855069+0000","last_became_peered":"2024-06-06T03:35:13.855069+0000","last_unstale":"2024-06-06T03:35:48.991216+0000","last_undegraded":"2024-06-06T03:35:48.991216+0000","last_fullsized":"2024-06-06T03:35:48.991216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:43.778258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991692+0000","last_change":"2024-06-06T03:35:09.827116+0000","last_active":"2024-06-06T03:35:48.991692+0000","last_peered":"2024-06-06T03:35:48.991692+0000","last_clean":"2024-06-06T03:35:48.991692+0000","last_became_active":"2024-06-06T03:35:09.826756+0000","last_became_peered":"2024-06-06T03:35:09.826756+0000","last_unstale":"2024-06-06T03:35:48.991692+0000","last_undegraded":"2024-06-06T03:35:48.991692+0000","last_fullsized":"2024-06-06T03:35:48.991692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:31.501411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985532+0000","last_change":"2024-06-06T03:35:13.861189+0000","last_active":"2024-06-06T03:35:48.985532+0000","last_peered":"2024-06-06T03:35:48.985532+0000","last_clean":"2024-06-06T03:35:48.985532+0000","last_became_active":"2024-06-06T03:35:13.860751+0000","last_became_peered":"2024-06-06T03:35:13.860751+0000","last_unstale":"2024-06-06T03:35:48.985532+0000","last_undegraded":"2024-06-06T03:35:48.985532+0000","last_fullsized":"2024-06-06T03:35:48.985532+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:16:34.036420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":132,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:12.099804+0000","last_change":"2024-06-06T03:35:10.761606+0000","last_active":"2024-06-06T03:37:12.099804+0000","last_peered":"2024-06-06T03:37:12.099804+0000","last_clean":"2024-06-06T03:37:12.099804+0000","last_became_active":"2024-06-06T03:35:10.761104+0000","last_became_peered":"2024-06-06T03:35:10.761104+0000","last_unstale":"2024-06-06T03:37:12.099804+0000","last_undegraded":"2024-06-06T03:37:12.099804+0000","last_fullsized":"2024-06-06T03:37:12.099804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:51:39.867088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985751+0000","last_change":"2024-06-06T03:35:13.856047+0000","last_active":"2024-06-06T03:35:48.985751+0000","last_peered":"2024-06-06T03:35:48.985751+0000","last_clean":"2024-06-06T03:35:48.985751+0000","last_became_active":"2024-06-06T03:35:13.855718+0000","last_became_peered":"2024-06-06T03:35:13.855718+0000","last_unstale":"2024-06-06T03:35:48.985751+0000","last_undegraded":"2024-06-06T03:35:48.985751+0000","last_fullsized":"2024-06-06T03:35:48.985751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:18:40.926872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984381+0000","last_change":"2024-06-06T03:35:09.827283+0000","last_active":"2024-06-06T03:35:48.984381+0000","last_peered":"2024-06-06T03:35:48.984381+0000","last_clean":"2024-06-06T03:35:48.984381+0000","last_became_active":"2024-06-06T03:35:09.826595+0000","last_became_peered":"2024-06-06T03:35:09.826595+0000","last_unstale":"2024-06-06T03:35:48.984381+0000","last_undegraded":"2024-06-06T03:35:48.984381+0000","last_fullsized":"2024-06-06T03:35:48.984381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:34:16.398847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985446+0000","last_change":"2024-06-06T03:35:13.851900+0000","last_active":"2024-06-06T03:35:48.985446+0000","last_peered":"2024-06-06T03:35:48.985446+0000","last_clean":"2024-06-06T03:35:48.985446+0000","last_became_active":"2024-06-06T03:35:13.851477+0000","last_became_peered":"2024-06-06T03:35:13.851477+0000","last_unstale":"2024-06-06T03:35:48.985446+0000","last_undegraded":"2024-06-06T03:35:48.985446+0000","last_fullsized":"2024-06-06T03:35:48.985446+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:31:54.001226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983666+0000","last_change":"2024-06-06T03:35:09.824675+0000","last_active":"2024-06-06T03:35:48.983666+0000","last_peered":"2024-06-06T03:35:48.983666+0000","last_clean":"2024-06-06T03:35:48.983666+0000","last_became_active":"2024-06-06T03:35:09.824347+0000","last_became_peered":"2024-06-06T03:35:09.824347+0000","last_unstale":"2024-06-06T03:35:48.983666+0000","last_undegraded":"2024-06-06T03:35:48.983666+0000","last_fullsized":"2024-06-06T03:35:48.983666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:01:43.080552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985522+0000","last_change":"2024-06-06T03:35:13.854706+0000","last_active":"2024-06-06T03:35:48.985522+0000","last_peered":"2024-06-06T03:35:48.985522+0000","last_clean":"2024-06-06T03:35:48.985522+0000","last_became_active":"2024-06-06T03:35:13.854392+0000","last_became_peered":"2024-06-06T03:35:13.854392+0000","last_unstale":"2024-06-06T03:35:48.985522+0000","last_undegraded":"2024-06-06T03:35:48.985522+0000","last_fullsized":"2024-06-06T03:35:48.985522+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:58:10.102367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987223+0000","last_change":"2024-06-06T03:35:13.853594+0000","last_active":"2024-06-06T03:35:48.987223+0000","last_peered":"2024-06-06T03:35:48.987223+0000","last_clean":"2024-06-06T03:35:48.987223+0000","last_became_active":"2024-06-06T03:35:13.852962+0000","last_became_peered":"2024-06-06T03:35:13.852962+0000","last_unstale":"2024-06-06T03:35:48.987223+0000","last_undegraded":"2024-06-06T03:35:48.987223+0000","last_fullsized":"2024-06-06T03:35:48.987223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:02:44.233868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991198+0000","last_change":"2024-06-06T03:35:13.848790+0000","last_active":"2024-06-06T03:35:48.991198+0000","last_peered":"2024-06-06T03:35:48.991198+0000","last_clean":"2024-06-06T03:35:48.991198+0000","last_became_active":"2024-06-06T03:35:13.848442+0000","last_became_peered":"2024-06-06T03:35:13.848442+0000","last_unstale":"2024-06-06T03:35:48.991198+0000","last_undegraded":"2024-06-06T03:35:48.991198+0000","last_fullsized":"2024-06-06T03:35:48.991198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:51.382210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991331+0000","last_change":"2024-06-06T03:35:13.866999+0000","last_active":"2024-06-06T03:35:48.991331+0000","last_peered":"2024-06-06T03:35:48.991331+0000","last_clean":"2024-06-06T03:35:48.991331+0000","last_became_active":"2024-06-06T03:35:13.866664+0000","last_became_peered":"2024-06-06T03:35:13.866664+0000","last_unstale":"2024-06-06T03:35:48.991331+0000","last_undegraded":"2024-06-06T03:35:48.991331+0000","last_fullsized":"2024-06-06T03:35:48.991331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:31:14.166122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990482+0000","last_change":"2024-06-06T03:35:13.849062+0000","last_active":"2024-06-06T03:35:48.990482+0000","last_peered":"2024-06-06T03:35:48.990482+0000","last_clean":"2024-06-06T03:35:48.990482+0000","last_became_active":"2024-06-06T03:35:13.848750+0000","last_became_peered":"2024-06-06T03:35:13.848750+0000","last_unstale":"2024-06-06T03:35:48.990482+0000","last_undegraded":"2024-06-06T03:35:48.990482+0000","last_fullsized":"2024-06-06T03:35:48.990482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:55:58.084908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983429+0000","last_change":"2024-06-06T03:35:13.857601+0000","last_active":"2024-06-06T03:35:48.983429+0000","last_peered":"2024-06-06T03:35:48.983429+0000","last_clean":"2024-06-06T03:35:48.983429+0000","last_became_active":"2024-06-06T03:35:13.857270+0000","last_became_peered":"2024-06-06T03:35:13.857270+0000","last_unstale":"2024-06-06T03:35:48.983429+0000","last_undegraded":"2024-06-06T03:35:48.983429+0000","last_fullsized":"2024-06-06T03:35:48.983429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:56:29.970210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987336+0000","last_change":"2024-06-06T03:35:13.846699+0000","last_active":"2024-06-06T03:35:48.987336+0000","last_peered":"2024-06-06T03:35:48.987336+0000","last_clean":"2024-06-06T03:35:48.987336+0000","last_became_active":"2024-06-06T03:35:13.846355+0000","last_became_peered":"2024-06-06T03:35:13.846355+0000","last_unstale":"2024-06-06T03:35:48.987336+0000","last_undegraded":"2024-06-06T03:35:48.987336+0000","last_fullsized":"2024-06-06T03:35:48.987336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:50:22.537339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991821+0000","last_change":"2024-06-06T03:35:13.854013+0000","last_active":"2024-06-06T03:35:48.991821+0000","last_peered":"2024-06-06T03:35:48.991821+0000","last_clean":"2024-06-06T03:35:48.991821+0000","last_became_active":"2024-06-06T03:35:13.853739+0000","last_became_peered":"2024-06-06T03:35:13.853739+0000","last_unstale":"2024-06-06T03:35:48.991821+0000","last_undegraded":"2024-06-06T03:35:48.991821+0000","last_fullsized":"2024-06-06T03:35:48.991821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:27:12.330943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991256+0000","last_change":"2024-06-06T03:35:13.854251+0000","last_active":"2024-06-06T03:35:48.991256+0000","last_peered":"2024-06-06T03:35:48.991256+0000","last_clean":"2024-06-06T03:35:48.991256+0000","last_became_active":"2024-06-06T03:35:13.853915+0000","last_became_peered":"2024-06-06T03:35:13.853915+0000","last_unstale":"2024-06-06T03:35:48.991256+0000","last_undegraded":"2024-06-06T03:35:48.991256+0000","last_fullsized":"2024-06-06T03:35:48.991256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:08.813316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984146+0000","last_change":"2024-06-06T03:35:13.849998+0000","last_active":"2024-06-06T03:35:48.984146+0000","last_peered":"2024-06-06T03:35:48.984146+0000","last_clean":"2024-06-06T03:35:48.984146+0000","last_became_active":"2024-06-06T03:35:13.848789+0000","last_became_peered":"2024-06-06T03:35:13.848789+0000","last_unstale":"2024-06-06T03:35:48.984146+0000","last_undegraded":"2024-06-06T03:35:48.984146+0000","last_fullsized":"2024-06-06T03:35:48.984146+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:53:39.043356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990461+0000","last_change":"2024-06-06T03:35:13.858020+0000","last_active":"2024-06-06T03:35:48.990461+0000","last_peered":"2024-06-06T03:35:48.990461+0000","last_clean":"2024-06-06T03:35:48.990461+0000","last_became_active":"2024-06-06T03:35:13.857723+0000","last_became_peered":"2024-06-06T03:35:13.857723+0000","last_unstale":"2024-06-06T03:35:48.990461+0000","last_undegraded":"2024-06-06T03:35:48.990461+0000","last_fullsized":"2024-06-06T03:35:48.990461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:08:08.651213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984730+0000","last_change":"2024-06-06T03:35:13.867546+0000","last_active":"2024-06-06T03:35:48.984730+0000","last_peered":"2024-06-06T03:35:48.984730+0000","last_clean":"2024-06-06T03:35:48.984730+0000","last_became_active":"2024-06-06T03:35:13.867182+0000","last_became_peered":"2024-06-06T03:35:13.867182+0000","last_unstale":"2024-06-06T03:35:48.984730+0000","last_undegraded":"2024-06-06T03:35:48.984730+0000","last_fullsized":"2024-06-06T03:35:48.984730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:25:55.831943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983770+0000","last_change":"2024-06-06T03:35:13.849080+0000","last_active":"2024-06-06T03:35:48.983770+0000","last_peered":"2024-06-06T03:35:48.983770+0000","last_clean":"2024-06-06T03:35:48.983770+0000","last_became_active":"2024-06-06T03:35:13.848476+0000","last_became_peered":"2024-06-06T03:35:13.848476+0000","last_unstale":"2024-06-06T03:35:48.983770+0000","last_undegraded":"2024-06-06T03:35:48.983770+0000","last_fullsized":"2024-06-06T03:35:48.983770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:48:37.306528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987065+0000","last_change":"2024-06-06T03:35:13.848442+0000","last_active":"2024-06-06T03:35:48.987065+0000","last_peered":"2024-06-06T03:35:48.987065+0000","last_clean":"2024-06-06T03:35:48.987065+0000","last_became_active":"2024-06-06T03:35:13.848074+0000","last_became_peered":"2024-06-06T03:35:13.848074+0000","last_unstale":"2024-06-06T03:35:48.987065+0000","last_undegraded":"2024-06-06T03:35:48.987065+0000","last_fullsized":"2024-06-06T03:35:48.987065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:07:00.197671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280416,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69536,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1007,"num_read_kb":578,"num_write":245,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":208608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":480334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":1133585,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808882,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":480418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1073437,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":480259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":543506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":481208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":543726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":479694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":479730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757319,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":1072725,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34027,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34027,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34027,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T03:37:16.065 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T03:37:16.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:15 smithi104 ceph-mon[36569]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:37:16.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:15 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:37:16.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:15 smithi131 ceph-mon[36687]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:37:16.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:15 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:37:16.338 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:16.857 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:16 smithi131 ceph-mon[36687]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:37:16.886 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:16 smithi104 ceph-mon[36569]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:37:17.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:16 smithi040 ceph-mon[32220]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:37:18.624 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:18 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:18.624 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:18 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:18.625 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:18 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:18.625 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:18 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.730+0000 7f9cc3317700 1 -- 172.21.15.40:0/3616768566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cbc107870 msgr2=0x7f9cbc10f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.730+0000 7f9cc3317700 1 --2- 172.21.15.40:0/3616768566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cbc107870 0x7f9cbc10f3f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400b0d0 tx=0x7f9cb400b3e0 comp rx=0 tx=0).stop 2024-06-06T03:37:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.730+0000 7f9cc3317700 1 -- 172.21.15.40:0/3616768566 shutdown_connections 2024-06-06T03:37:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.730+0000 7f9cc3317700 1 --2- 172.21.15.40:0/3616768566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cbc1027a0 0x7f9cbc102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.730+0000 7f9cc3317700 1 --2- 172.21.15.40:0/3616768566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cbc107870 0x7f9cbc10f3f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.730+0000 7f9cc3317700 1 --2- 172.21.15.40:0/3616768566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc103150 0x7f9cbc107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.730+0000 7f9cc3317700 1 -- 172.21.15.40:0/3616768566 >> 172.21.15.40:0/3616768566 conn(0x7f9cbc079570 msgr2=0x7f9cbc079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.731+0000 7f9cc3317700 1 -- 172.21.15.40:0/3616768566 shutdown_connections 2024-06-06T03:37:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.731+0000 7f9cc3317700 1 -- 172.21.15.40:0/3616768566 wait complete. 2024-06-06T03:37:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.731+0000 7f9cc3317700 1 Processor -- start 2024-06-06T03:37:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.731+0000 7f9cc3317700 1 -- start start 2024-06-06T03:37:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.731+0000 7f9cc3317700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cbc1027a0 0x7f9cbc19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.731+0000 7f9cc3317700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cbc103150 0x7f9cbc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.731+0000 7f9cc3317700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc107870 0x7f9cbc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.731+0000 7f9cc3317700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc112790 con 0x7f9cbc1027a0 2024-06-06T03:37:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.731+0000 7f9cc3317700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc112610 con 0x7f9cbc107870 2024-06-06T03:37:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.731+0000 7f9cc3317700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc112910 con 0x7f9cbc103150 2024-06-06T03:37:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cc10b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cbc1027a0 0x7f9cbc19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cc10b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cbc1027a0 0x7f9cbc19e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36086/0 (socket says 172.21.15.40:36086) 2024-06-06T03:37:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cc10b3700 1 -- 172.21.15.40:0/656356373 learned_addr learned my addr 172.21.15.40:0/656356373 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cc18b4700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc107870 0x7f9cbc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cc08b2700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cbc103150 0x7f9cbc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cc10b3700 1 -- 172.21.15.40:0/656356373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cbc103150 msgr2=0x7f9cbc19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cc10b3700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cbc103150 0x7f9cbc19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cc10b3700 1 -- 172.21.15.40:0/656356373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc107870 msgr2=0x7f9cbc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cc10b3700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc107870 0x7f9cbc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cc10b3700 1 -- 172.21.15.40:0/656356373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cbc199110 con 0x7f9cbc1027a0 2024-06-06T03:37:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cc08b2700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cbc103150 0x7f9cbc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:37:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cc10b3700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cbc1027a0 0x7f9cbc19e7f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800c920 tx=0x7f9cb800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cae7fc700 1 -- 172.21.15.40:0/656356373 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8007b00 con 0x7f9cbc1027a0 2024-06-06T03:37:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cae7fc700 1 -- 172.21.15.40:0/656356373 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb8004d10 con 0x7f9cbc1027a0 2024-06-06T03:37:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cc3317700 1 -- 172.21.15.40:0/656356373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cbc199400 con 0x7f9cbc1027a0 2024-06-06T03:37:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cae7fc700 1 -- 172.21.15.40:0/656356373 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800f450 con 0x7f9cbc1027a0 2024-06-06T03:37:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.732+0000 7f9cc3317700 1 -- 172.21.15.40:0/656356373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cbc1a56c0 con 0x7f9cbc1027a0 2024-06-06T03:37:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.733+0000 7f9cc3317700 1 -- 172.21.15.40:0/656356373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cbc069180 con 0x7f9cbc1027a0 2024-06-06T03:37:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.734+0000 7f9cae7fc700 1 -- 172.21.15.40:0/656356373 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9cb8004750 con 0x7f9cbc1027a0 2024-06-06T03:37:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.735+0000 7f9cae7fc700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9ca80a2c20 0x7f9ca80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.735+0000 7f9cc08b2700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9ca80a2c20 0x7f9ca80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.735+0000 7f9cae7fc700 1 -- 172.21.15.40:0/656356373 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f9cb80cf0a0 con 0x7f9cbc1027a0 2024-06-06T03:37:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.736+0000 7f9cc08b2700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9ca80a2c20 0x7f9ca80a50e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc108c20 tx=0x7f9cb0009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.740+0000 7f9cae7fc700 1 -- 172.21.15.40:0/656356373 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9cb8014030 con 0x7f9cbc1027a0 2024-06-06T03:37:18.781 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:18 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:18.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:18 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:18.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:18 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:18.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:18 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:18.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:18 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:18.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:18 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:18.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:18 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:18.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:18 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:18.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.968+0000 7f9cc3317700 1 -- 172.21.15.40:0/656356373 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9cbc0628d0 con 0x7f9ca80a2c20 2024-06-06T03:37:18.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.976+0000 7f9cae7fc700 1 -- 172.21.15.40:0/656356373 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258952 (secure 0 0 0) 0x7f9cbc0628d0 con 0x7f9ca80a2c20 2024-06-06T03:37:18.981 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:37:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.980+0000 7f9cc3317700 1 -- 172.21.15.40:0/656356373 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9ca80a2c20 msgr2=0x7f9ca80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.980+0000 7f9cc3317700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9ca80a2c20 0x7f9ca80a50e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc108c20 tx=0x7f9cb0009550 comp rx=0 tx=0).stop 2024-06-06T03:37:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.980+0000 7f9cc3317700 1 -- 172.21.15.40:0/656356373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cbc1027a0 msgr2=0x7f9cbc19e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.980+0000 7f9cc3317700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cbc1027a0 0x7f9cbc19e7f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800c920 tx=0x7f9cb800cce0 comp rx=0 tx=0).stop 2024-06-06T03:37:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.980+0000 7f9cc3317700 1 -- 172.21.15.40:0/656356373 shutdown_connections 2024-06-06T03:37:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.980+0000 7f9cc3317700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f9ca80a2c20 0x7f9ca80a50e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.980+0000 7f9cc3317700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9cbc107870 0x7f9cbc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.980+0000 7f9cc3317700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cbc1027a0 0x7f9cbc19e7f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.980+0000 7f9cc3317700 1 --2- 172.21.15.40:0/656356373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cbc103150 0x7f9cbc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.980+0000 7f9cc3317700 1 -- 172.21.15.40:0/656356373 >> 172.21.15.40:0/656356373 conn(0x7f9cbc079570 msgr2=0x7f9cbc100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.980+0000 7f9cc3317700 1 -- 172.21.15.40:0/656356373 shutdown_connections 2024-06-06T03:37:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:18.980+0000 7f9cc3317700 1 -- 172.21.15.40:0/656356373 wait complete. 2024-06-06T03:37:18.984 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-06-06T03:37:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:19 smithi040 ceph-mon[32220]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:37:19.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:19 smithi040 ceph-mon[32220]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-06-06T03:37:19.670 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-06-06T03:37:17.709593+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660138,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1183,"num_read_kb":722,"num_write":428,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365492,"kb_used_data":12156,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576332,"statfs":{"total":1151940427776,"available":1151566163968,"internally_reserved":0,"allocated":12447744,"data_stored":7728774,"data_compressed":107001,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002954"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987328+0000","last_change":"2024-06-06T03:35:13.854938+0000","last_active":"2024-06-06T03:35:48.987328+0000","last_peered":"2024-06-06T03:35:48.987328+0000","last_clean":"2024-06-06T03:35:48.987328+0000","last_became_active":"2024-06-06T03:35:13.853574+0000","last_became_peered":"2024-06-06T03:35:13.853574+0000","last_unstale":"2024-06-06T03:35:48.987328+0000","last_undegraded":"2024-06-06T03:35:48.987328+0000","last_fullsized":"2024-06-06T03:35:48.987328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:21:09.256002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990557+0000","last_change":"2024-06-06T03:35:13.858299+0000","last_active":"2024-06-06T03:35:48.990557+0000","last_peered":"2024-06-06T03:35:48.990557+0000","last_clean":"2024-06-06T03:35:48.990557+0000","last_became_active":"2024-06-06T03:35:13.857993+0000","last_became_peered":"2024-06-06T03:35:13.857993+0000","last_unstale":"2024-06-06T03:35:48.990557+0000","last_undegraded":"2024-06-06T03:35:48.990557+0000","last_fullsized":"2024-06-06T03:35:48.990557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.811056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990502+0000","last_change":"2024-06-06T03:35:13.852783+0000","last_active":"2024-06-06T03:35:48.990502+0000","last_peered":"2024-06-06T03:35:48.990502+0000","last_clean":"2024-06-06T03:35:48.990502+0000","last_became_active":"2024-06-06T03:35:13.852468+0000","last_became_peered":"2024-06-06T03:35:13.852468+0000","last_unstale":"2024-06-06T03:35:48.990502+0000","last_undegraded":"2024-06-06T03:35:48.990502+0000","last_fullsized":"2024-06-06T03:35:48.990502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:04:59.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.004048+0000","last_change":"2024-06-06T03:35:13.845223+0000","last_active":"2024-06-06T03:35:49.004048+0000","last_peered":"2024-06-06T03:35:49.004048+0000","last_clean":"2024-06-06T03:35:49.004048+0000","last_became_active":"2024-06-06T03:35:13.844854+0000","last_became_peered":"2024-06-06T03:35:13.844854+0000","last_unstale":"2024-06-06T03:35:49.004048+0000","last_undegraded":"2024-06-06T03:35:49.004048+0000","last_fullsized":"2024-06-06T03:35:49.004048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:35:20.058188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991571+0000","last_change":"2024-06-06T03:35:13.853703+0000","last_active":"2024-06-06T03:35:48.991571+0000","last_peered":"2024-06-06T03:35:48.991571+0000","last_clean":"2024-06-06T03:35:48.991571+0000","last_became_active":"2024-06-06T03:35:13.853358+0000","last_became_peered":"2024-06-06T03:35:13.853358+0000","last_unstale":"2024-06-06T03:35:48.991571+0000","last_undegraded":"2024-06-06T03:35:48.991571+0000","last_fullsized":"2024-06-06T03:35:48.991571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:23:39.092306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988483+0000","last_change":"2024-06-06T03:35:13.859414+0000","last_active":"2024-06-06T03:35:48.988483+0000","last_peered":"2024-06-06T03:35:48.988483+0000","last_clean":"2024-06-06T03:35:48.988483+0000","last_became_active":"2024-06-06T03:35:13.859038+0000","last_became_peered":"2024-06-06T03:35:13.859038+0000","last_unstale":"2024-06-06T03:35:48.988483+0000","last_undegraded":"2024-06-06T03:35:48.988483+0000","last_fullsized":"2024-06-06T03:35:48.988483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:14:31.780064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985258+0000","last_change":"2024-06-06T03:35:13.859245+0000","last_active":"2024-06-06T03:35:48.985258+0000","last_peered":"2024-06-06T03:35:48.985258+0000","last_clean":"2024-06-06T03:35:48.985258+0000","last_became_active":"2024-06-06T03:35:13.858857+0000","last_became_peered":"2024-06-06T03:35:13.858857+0000","last_unstale":"2024-06-06T03:35:48.985258+0000","last_undegraded":"2024-06-06T03:35:48.985258+0000","last_fullsized":"2024-06-06T03:35:48.985258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:03:23.240007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990694+0000","last_change":"2024-06-06T03:35:13.851249+0000","last_active":"2024-06-06T03:35:48.990694+0000","last_peered":"2024-06-06T03:35:48.990694+0000","last_clean":"2024-06-06T03:35:48.990694+0000","last_became_active":"2024-06-06T03:35:13.850777+0000","last_became_peered":"2024-06-06T03:35:13.850777+0000","last_unstale":"2024-06-06T03:35:48.990694+0000","last_undegraded":"2024-06-06T03:35:48.990694+0000","last_fullsized":"2024-06-06T03:35:48.990694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:53:48.463672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985036+0000","last_change":"2024-06-06T03:35:13.852950+0000","last_active":"2024-06-06T03:35:48.985036+0000","last_peered":"2024-06-06T03:35:48.985036+0000","last_clean":"2024-06-06T03:35:48.985036+0000","last_became_active":"2024-06-06T03:35:13.852160+0000","last_became_peered":"2024-06-06T03:35:13.852160+0000","last_unstale":"2024-06-06T03:35:48.985036+0000","last_undegraded":"2024-06-06T03:35:48.985036+0000","last_fullsized":"2024-06-06T03:35:48.985036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:00.498440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988970+0000","last_change":"2024-06-06T03:35:13.856081+0000","last_active":"2024-06-06T03:35:48.988970+0000","last_peered":"2024-06-06T03:35:48.988970+0000","last_clean":"2024-06-06T03:35:48.988970+0000","last_became_active":"2024-06-06T03:35:13.855718+0000","last_became_peered":"2024-06-06T03:35:13.855718+0000","last_unstale":"2024-06-06T03:35:48.988970+0000","last_undegraded":"2024-06-06T03:35:48.988970+0000","last_fullsized":"2024-06-06T03:35:48.988970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:33:38.171996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985594+0000","last_change":"2024-06-06T03:35:13.861850+0000","last_active":"2024-06-06T03:35:48.985594+0000","last_peered":"2024-06-06T03:35:48.985594+0000","last_clean":"2024-06-06T03:35:48.985594+0000","last_became_active":"2024-06-06T03:35:13.861513+0000","last_became_peered":"2024-06-06T03:35:13.861513+0000","last_unstale":"2024-06-06T03:35:48.985594+0000","last_undegraded":"2024-06-06T03:35:48.985594+0000","last_fullsized":"2024-06-06T03:35:48.985594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:32:45.938302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.003896+0000","last_change":"2024-06-06T03:35:13.847742+0000","last_active":"2024-06-06T03:35:49.003896+0000","last_peered":"2024-06-06T03:35:49.003896+0000","last_clean":"2024-06-06T03:35:49.003896+0000","last_became_active":"2024-06-06T03:35:13.847409+0000","last_became_peered":"2024-06-06T03:35:13.847409+0000","last_unstale":"2024-06-06T03:35:49.003896+0000","last_undegraded":"2024-06-06T03:35:49.003896+0000","last_fullsized":"2024-06-06T03:35:49.003896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:20:31.353919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983746+0000","last_change":"2024-06-06T03:35:13.848785+0000","last_active":"2024-06-06T03:35:48.983746+0000","last_peered":"2024-06-06T03:35:48.983746+0000","last_clean":"2024-06-06T03:35:48.983746+0000","last_became_active":"2024-06-06T03:35:13.848048+0000","last_became_peered":"2024-06-06T03:35:13.848048+0000","last_unstale":"2024-06-06T03:35:48.983746+0000","last_undegraded":"2024-06-06T03:35:48.983746+0000","last_fullsized":"2024-06-06T03:35:48.983746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:43.657761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990820+0000","last_change":"2024-06-06T03:35:13.856083+0000","last_active":"2024-06-06T03:35:48.990820+0000","last_peered":"2024-06-06T03:35:48.990820+0000","last_clean":"2024-06-06T03:35:48.990820+0000","last_became_active":"2024-06-06T03:35:13.854390+0000","last_became_peered":"2024-06-06T03:35:13.854390+0000","last_unstale":"2024-06-06T03:35:48.990820+0000","last_undegraded":"2024-06-06T03:35:48.990820+0000","last_fullsized":"2024-06-06T03:35:48.990820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:51:37.163814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990153+0000","last_change":"2024-06-06T03:35:13.852611+0000","last_active":"2024-06-06T03:35:48.990153+0000","last_peered":"2024-06-06T03:35:48.990153+0000","last_clean":"2024-06-06T03:35:48.990153+0000","last_became_active":"2024-06-06T03:35:13.852244+0000","last_became_peered":"2024-06-06T03:35:13.852244+0000","last_unstale":"2024-06-06T03:35:48.990153+0000","last_undegraded":"2024-06-06T03:35:48.990153+0000","last_fullsized":"2024-06-06T03:35:48.990153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:59:53.513150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984532+0000","last_change":"2024-06-06T03:35:13.857840+0000","last_active":"2024-06-06T03:35:48.984532+0000","last_peered":"2024-06-06T03:35:48.984532+0000","last_clean":"2024-06-06T03:35:48.984532+0000","last_became_active":"2024-06-06T03:35:13.857137+0000","last_became_peered":"2024-06-06T03:35:13.857137+0000","last_unstale":"2024-06-06T03:35:48.984532+0000","last_undegraded":"2024-06-06T03:35:48.984532+0000","last_fullsized":"2024-06-06T03:35:48.984532+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:42:07.924735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987452+0000","last_change":"2024-06-06T03:35:13.857033+0000","last_active":"2024-06-06T03:35:48.987452+0000","last_peered":"2024-06-06T03:35:48.987452+0000","last_clean":"2024-06-06T03:35:48.987452+0000","last_became_active":"2024-06-06T03:35:13.856519+0000","last_became_peered":"2024-06-06T03:35:13.856519+0000","last_unstale":"2024-06-06T03:35:48.987452+0000","last_undegraded":"2024-06-06T03:35:48.987452+0000","last_fullsized":"2024-06-06T03:35:48.987452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:05:25.668344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.986761+0000","last_change":"2024-06-06T03:35:13.851356+0000","last_active":"2024-06-06T03:35:48.986761+0000","last_peered":"2024-06-06T03:35:48.986761+0000","last_clean":"2024-06-06T03:35:48.986761+0000","last_became_active":"2024-06-06T03:35:13.851030+0000","last_became_peered":"2024-06-06T03:35:13.851030+0000","last_unstale":"2024-06-06T03:35:48.986761+0000","last_undegraded":"2024-06-06T03:35:48.986761+0000","last_fullsized":"2024-06-06T03:35:48.986761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:49:38.833022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991734+0000","last_change":"2024-06-06T03:35:13.858613+0000","last_active":"2024-06-06T03:35:48.991734+0000","last_peered":"2024-06-06T03:35:48.991734+0000","last_clean":"2024-06-06T03:35:48.991734+0000","last_became_active":"2024-06-06T03:35:13.858182+0000","last_became_peered":"2024-06-06T03:35:13.858182+0000","last_unstale":"2024-06-06T03:35:48.991734+0000","last_undegraded":"2024-06-06T03:35:48.991734+0000","last_fullsized":"2024-06-06T03:35:48.991734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:23:08.216431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":141,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:14.090544+0000","last_change":"2024-06-06T03:35:09.821896+0000","last_active":"2024-06-06T03:37:14.090544+0000","last_peered":"2024-06-06T03:37:14.090544+0000","last_clean":"2024-06-06T03:37:14.090544+0000","last_became_active":"2024-06-06T03:35:09.821567+0000","last_became_peered":"2024-06-06T03:35:09.821567+0000","last_unstale":"2024-06-06T03:37:14.090544+0000","last_undegraded":"2024-06-06T03:37:14.090544+0000","last_fullsized":"2024-06-06T03:37:14.090544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:04:51.918393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991407+0000","last_change":"2024-06-06T03:35:13.857357+0000","last_active":"2024-06-06T03:35:48.991407+0000","last_peered":"2024-06-06T03:35:48.991407+0000","last_clean":"2024-06-06T03:35:48.991407+0000","last_became_active":"2024-06-06T03:35:13.857016+0000","last_became_peered":"2024-06-06T03:35:13.857016+0000","last_unstale":"2024-06-06T03:35:48.991407+0000","last_undegraded":"2024-06-06T03:35:48.991407+0000","last_fullsized":"2024-06-06T03:35:48.991407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:42.566425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991487+0000","last_change":"2024-06-06T03:35:10.277316+0000","last_active":"2024-06-06T03:35:48.991487+0000","last_peered":"2024-06-06T03:35:48.991487+0000","last_clean":"2024-06-06T03:35:48.991487+0000","last_became_active":"2024-06-06T03:35:10.276986+0000","last_became_peered":"2024-06-06T03:35:10.276986+0000","last_unstale":"2024-06-06T03:35:48.991487+0000","last_undegraded":"2024-06-06T03:35:48.991487+0000","last_fullsized":"2024-06-06T03:35:48.991487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:17:29.034224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990383+0000","last_change":"2024-06-06T03:35:13.863069+0000","last_active":"2024-06-06T03:35:48.990383+0000","last_peered":"2024-06-06T03:35:48.990383+0000","last_clean":"2024-06-06T03:35:48.990383+0000","last_became_active":"2024-06-06T03:35:13.862647+0000","last_became_peered":"2024-06-06T03:35:13.862647+0000","last_unstale":"2024-06-06T03:35:48.990383+0000","last_undegraded":"2024-06-06T03:35:48.990383+0000","last_fullsized":"2024-06-06T03:35:48.990383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:48.252606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992306+0000","last_change":"2024-06-06T03:35:09.824977+0000","last_active":"2024-06-06T03:35:48.992306+0000","last_peered":"2024-06-06T03:35:48.992306+0000","last_clean":"2024-06-06T03:35:48.992306+0000","last_became_active":"2024-06-06T03:35:09.824643+0000","last_became_peered":"2024-06-06T03:35:09.824643+0000","last_unstale":"2024-06-06T03:35:48.992306+0000","last_undegraded":"2024-06-06T03:35:48.992306+0000","last_fullsized":"2024-06-06T03:35:48.992306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:35:37.208023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984591+0000","last_change":"2024-06-06T03:35:13.860572+0000","last_active":"2024-06-06T03:35:48.984591+0000","last_peered":"2024-06-06T03:35:48.984591+0000","last_clean":"2024-06-06T03:35:48.984591+0000","last_became_active":"2024-06-06T03:35:13.858398+0000","last_became_peered":"2024-06-06T03:35:13.858398+0000","last_unstale":"2024-06-06T03:35:48.984591+0000","last_undegraded":"2024-06-06T03:35:48.984591+0000","last_fullsized":"2024-06-06T03:35:48.984591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:02:12.683096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991795+0000","last_change":"2024-06-06T03:35:09.827016+0000","last_active":"2024-06-06T03:35:48.991795+0000","last_peered":"2024-06-06T03:35:48.991795+0000","last_clean":"2024-06-06T03:35:48.991795+0000","last_became_active":"2024-06-06T03:35:09.826576+0000","last_became_peered":"2024-06-06T03:35:09.826576+0000","last_unstale":"2024-06-06T03:35:48.991795+0000","last_undegraded":"2024-06-06T03:35:48.991795+0000","last_fullsized":"2024-06-06T03:35:48.991795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:24.120549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984420+0000","last_change":"2024-06-06T03:35:13.853988+0000","last_active":"2024-06-06T03:35:48.984420+0000","last_peered":"2024-06-06T03:35:48.984420+0000","last_clean":"2024-06-06T03:35:48.984420+0000","last_became_active":"2024-06-06T03:35:13.852343+0000","last_became_peered":"2024-06-06T03:35:13.852343+0000","last_unstale":"2024-06-06T03:35:48.984420+0000","last_undegraded":"2024-06-06T03:35:48.984420+0000","last_fullsized":"2024-06-06T03:35:48.984420+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:39:03.054045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":134,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:12.088800+0000","last_change":"2024-06-06T03:35:09.829109+0000","last_active":"2024-06-06T03:37:12.088800+0000","last_peered":"2024-06-06T03:37:12.088800+0000","last_clean":"2024-06-06T03:37:12.088800+0000","last_became_active":"2024-06-06T03:35:09.828734+0000","last_became_peered":"2024-06-06T03:35:09.828734+0000","last_unstale":"2024-06-06T03:37:12.088800+0000","last_undegraded":"2024-06-06T03:37:12.088800+0000","last_fullsized":"2024-06-06T03:37:12.088800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:50.475600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992478+0000","last_change":"2024-06-06T03:35:13.859198+0000","last_active":"2024-06-06T03:35:48.992478+0000","last_peered":"2024-06-06T03:35:48.992478+0000","last_clean":"2024-06-06T03:35:48.992478+0000","last_became_active":"2024-06-06T03:35:13.858863+0000","last_became_peered":"2024-06-06T03:35:13.858863+0000","last_unstale":"2024-06-06T03:35:48.992478+0000","last_undegraded":"2024-06-06T03:35:48.992478+0000","last_fullsized":"2024-06-06T03:35:48.992478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:57.153174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988385+0000","last_change":"2024-06-06T03:35:09.824352+0000","last_active":"2024-06-06T03:35:48.988385+0000","last_peered":"2024-06-06T03:35:48.988385+0000","last_clean":"2024-06-06T03:35:48.988385+0000","last_became_active":"2024-06-06T03:35:09.823998+0000","last_became_peered":"2024-06-06T03:35:09.823998+0000","last_unstale":"2024-06-06T03:35:48.988385+0000","last_undegraded":"2024-06-06T03:35:48.988385+0000","last_fullsized":"2024-06-06T03:35:48.988385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:47:24.646067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984826+0000","last_change":"2024-06-06T03:35:13.836724+0000","last_active":"2024-06-06T03:35:48.984826+0000","last_peered":"2024-06-06T03:35:48.984826+0000","last_clean":"2024-06-06T03:35:48.984826+0000","last_became_active":"2024-06-06T03:35:13.836180+0000","last_became_peered":"2024-06-06T03:35:13.836180+0000","last_unstale":"2024-06-06T03:35:48.984826+0000","last_undegraded":"2024-06-06T03:35:48.984826+0000","last_fullsized":"2024-06-06T03:35:48.984826+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:08.868462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983408+0000","last_change":"2024-06-06T03:35:09.823047+0000","last_active":"2024-06-06T03:35:48.983408+0000","last_peered":"2024-06-06T03:35:48.983408+0000","last_clean":"2024-06-06T03:35:48.983408+0000","last_became_active":"2024-06-06T03:35:09.822744+0000","last_became_peered":"2024-06-06T03:35:09.822744+0000","last_unstale":"2024-06-06T03:35:48.983408+0000","last_undegraded":"2024-06-06T03:35:48.983408+0000","last_fullsized":"2024-06-06T03:35:48.983408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:20:20.165682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.989848+0000","last_change":"2024-06-06T03:35:13.857222+0000","last_active":"2024-06-06T03:35:48.989848+0000","last_peered":"2024-06-06T03:35:48.989848+0000","last_clean":"2024-06-06T03:35:48.989848+0000","last_became_active":"2024-06-06T03:35:13.856873+0000","last_became_peered":"2024-06-06T03:35:13.856873+0000","last_unstale":"2024-06-06T03:35:48.989848+0000","last_undegraded":"2024-06-06T03:35:48.989848+0000","last_fullsized":"2024-06-06T03:35:48.989848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:32:16.646275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984083+0000","last_change":"2024-06-06T03:35:09.826036+0000","last_active":"2024-06-06T03:35:48.984083+0000","last_peered":"2024-06-06T03:35:48.984083+0000","last_clean":"2024-06-06T03:35:48.984083+0000","last_became_active":"2024-06-06T03:35:09.825690+0000","last_became_peered":"2024-06-06T03:35:09.825690+0000","last_unstale":"2024-06-06T03:35:48.984083+0000","last_undegraded":"2024-06-06T03:35:48.984083+0000","last_fullsized":"2024-06-06T03:35:48.984083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:03:35.858828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.986086+0000","last_change":"2024-06-06T03:35:13.849016+0000","last_active":"2024-06-06T03:35:48.986086+0000","last_peered":"2024-06-06T03:35:48.986086+0000","last_clean":"2024-06-06T03:35:48.986086+0000","last_became_active":"2024-06-06T03:35:13.848709+0000","last_became_peered":"2024-06-06T03:35:13.848709+0000","last_unstale":"2024-06-06T03:35:48.986086+0000","last_undegraded":"2024-06-06T03:35:48.986086+0000","last_fullsized":"2024-06-06T03:35:48.986086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:42.778612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":234,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:16.094394+0000","last_change":"2024-06-06T03:35:09.827072+0000","last_active":"2024-06-06T03:37:16.094394+0000","last_peered":"2024-06-06T03:37:16.094394+0000","last_clean":"2024-06-06T03:37:16.094394+0000","last_became_active":"2024-06-06T03:35:09.826734+0000","last_became_peered":"2024-06-06T03:35:09.826734+0000","last_unstale":"2024-06-06T03:37:16.094394+0000","last_undegraded":"2024-06-06T03:37:16.094394+0000","last_fullsized":"2024-06-06T03:37:16.094394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:07:13.388398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985601+0000","last_change":"2024-06-06T03:35:13.856883+0000","last_active":"2024-06-06T03:35:48.985601+0000","last_peered":"2024-06-06T03:35:48.985601+0000","last_clean":"2024-06-06T03:35:48.985601+0000","last_became_active":"2024-06-06T03:35:13.856521+0000","last_became_peered":"2024-06-06T03:35:13.856521+0000","last_unstale":"2024-06-06T03:35:48.985601+0000","last_undegraded":"2024-06-06T03:35:48.985601+0000","last_fullsized":"2024-06-06T03:35:48.985601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:19.466218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985292+0000","last_change":"2024-06-06T03:35:09.829153+0000","last_active":"2024-06-06T03:35:48.985292+0000","last_peered":"2024-06-06T03:35:48.985292+0000","last_clean":"2024-06-06T03:35:48.985292+0000","last_became_active":"2024-06-06T03:35:09.828797+0000","last_became_peered":"2024-06-06T03:35:09.828797+0000","last_unstale":"2024-06-06T03:35:48.985292+0000","last_undegraded":"2024-06-06T03:35:48.985292+0000","last_fullsized":"2024-06-06T03:35:48.985292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:06:02.986396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990268+0000","last_change":"2024-06-06T03:35:13.857094+0000","last_active":"2024-06-06T03:35:48.990268+0000","last_peered":"2024-06-06T03:35:48.990268+0000","last_clean":"2024-06-06T03:35:48.990268+0000","last_became_active":"2024-06-06T03:35:13.856778+0000","last_became_peered":"2024-06-06T03:35:13.856778+0000","last_unstale":"2024-06-06T03:35:48.990268+0000","last_undegraded":"2024-06-06T03:35:48.990268+0000","last_fullsized":"2024-06-06T03:35:48.990268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:50:30.097762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984506+0000","last_change":"2024-06-06T03:35:10.277802+0000","last_active":"2024-06-06T03:35:48.984506+0000","last_peered":"2024-06-06T03:35:48.984506+0000","last_clean":"2024-06-06T03:35:48.984506+0000","last_became_active":"2024-06-06T03:35:10.277417+0000","last_became_peered":"2024-06-06T03:35:10.277417+0000","last_unstale":"2024-06-06T03:35:48.984506+0000","last_undegraded":"2024-06-06T03:35:48.984506+0000","last_fullsized":"2024-06-06T03:35:48.984506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:44:35.178184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984699+0000","last_change":"2024-06-06T03:35:13.858160+0000","last_active":"2024-06-06T03:35:48.984699+0000","last_peered":"2024-06-06T03:35:48.984699+0000","last_clean":"2024-06-06T03:35:48.984699+0000","last_became_active":"2024-06-06T03:35:13.853149+0000","last_became_peered":"2024-06-06T03:35:13.853149+0000","last_unstale":"2024-06-06T03:35:48.984699+0000","last_undegraded":"2024-06-06T03:35:48.984699+0000","last_fullsized":"2024-06-06T03:35:48.984699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:10:06.697274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'31","reported_seq":270,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:16.094357+0000","last_change":"2024-06-06T03:35:09.829443+0000","last_active":"2024-06-06T03:37:16.094357+0000","last_peered":"2024-06-06T03:37:16.094357+0000","last_clean":"2024-06-06T03:37:16.094357+0000","last_became_active":"2024-06-06T03:35:09.828093+0000","last_became_peered":"2024-06-06T03:35:09.828093+0000","last_unstale":"2024-06-06T03:37:16.094357+0000","last_undegraded":"2024-06-06T03:37:16.094357+0000","last_fullsized":"2024-06-06T03:37:16.094357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:05:19.395177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62334,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":213,"num_read_kb":167,"num_write":33,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991968+0000","last_change":"2024-06-06T03:35:13.850869+0000","last_active":"2024-06-06T03:35:48.991968+0000","last_peered":"2024-06-06T03:35:48.991968+0000","last_clean":"2024-06-06T03:35:48.991968+0000","last_became_active":"2024-06-06T03:35:13.849489+0000","last_became_peered":"2024-06-06T03:35:13.849489+0000","last_unstale":"2024-06-06T03:35:48.991968+0000","last_undegraded":"2024-06-06T03:35:48.991968+0000","last_fullsized":"2024-06-06T03:35:48.991968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:27:45.052033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991491+0000","last_change":"2024-06-06T03:35:09.823117+0000","last_active":"2024-06-06T03:35:48.991491+0000","last_peered":"2024-06-06T03:35:48.991491+0000","last_clean":"2024-06-06T03:35:48.991491+0000","last_became_active":"2024-06-06T03:35:09.822624+0000","last_became_peered":"2024-06-06T03:35:09.822624+0000","last_unstale":"2024-06-06T03:35:48.991491+0000","last_undegraded":"2024-06-06T03:35:48.991491+0000","last_fullsized":"2024-06-06T03:35:48.991491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:59.171343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983437+0000","last_change":"2024-06-06T03:35:13.861043+0000","last_active":"2024-06-06T03:35:48.983437+0000","last_peered":"2024-06-06T03:35:48.983437+0000","last_clean":"2024-06-06T03:35:48.983437+0000","last_became_active":"2024-06-06T03:35:13.860718+0000","last_became_peered":"2024-06-06T03:35:13.860718+0000","last_unstale":"2024-06-06T03:35:48.983437+0000","last_undegraded":"2024-06-06T03:35:48.983437+0000","last_fullsized":"2024-06-06T03:35:48.983437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:34:04.595780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984355+0000","last_change":"2024-06-06T03:35:09.825505+0000","last_active":"2024-06-06T03:35:48.984355+0000","last_peered":"2024-06-06T03:35:48.984355+0000","last_clean":"2024-06-06T03:35:48.984355+0000","last_became_active":"2024-06-06T03:35:09.825127+0000","last_became_peered":"2024-06-06T03:35:09.825127+0000","last_unstale":"2024-06-06T03:35:48.984355+0000","last_undegraded":"2024-06-06T03:35:48.984355+0000","last_fullsized":"2024-06-06T03:35:48.984355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:07:27.664589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987834+0000","last_change":"2024-06-06T03:35:48.987834+0000","last_active":"2024-06-06T03:35:48.987834+0000","last_peered":"2024-06-06T03:35:48.987834+0000","last_clean":"2024-06-06T03:35:48.987834+0000","last_became_active":"2024-06-06T03:35:48.987515+0000","last_became_peered":"2024-06-06T03:35:48.987515+0000","last_unstale":"2024-06-06T03:35:48.987834+0000","last_undegraded":"2024-06-06T03:35:48.987834+0000","last_fullsized":"2024-06-06T03:35:48.987834+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:05:19.342289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":132,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:12.088111+0000","last_change":"2024-06-06T03:35:09.826192+0000","last_active":"2024-06-06T03:37:12.088111+0000","last_peered":"2024-06-06T03:37:12.088111+0000","last_clean":"2024-06-06T03:37:12.088111+0000","last_became_active":"2024-06-06T03:35:09.825481+0000","last_became_peered":"2024-06-06T03:35:09.825481+0000","last_unstale":"2024-06-06T03:37:12.088111+0000","last_undegraded":"2024-06-06T03:37:12.088111+0000","last_fullsized":"2024-06-06T03:37:12.088111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:18:30.772991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985035+0000","last_change":"2024-06-06T03:35:13.853113+0000","last_active":"2024-06-06T03:35:48.985035+0000","last_peered":"2024-06-06T03:35:48.985035+0000","last_clean":"2024-06-06T03:35:48.985035+0000","last_became_active":"2024-06-06T03:35:13.852751+0000","last_became_peered":"2024-06-06T03:35:13.852751+0000","last_unstale":"2024-06-06T03:35:48.985035+0000","last_undegraded":"2024-06-06T03:35:48.985035+0000","last_fullsized":"2024-06-06T03:35:48.985035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:53.536685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.006465+0000","last_change":"2024-06-06T03:35:49.005958+0000","last_active":"2024-06-06T03:35:49.006465+0000","last_peered":"2024-06-06T03:35:49.006465+0000","last_clean":"2024-06-06T03:35:49.006465+0000","last_became_active":"2024-06-06T03:35:48.993266+0000","last_became_peered":"2024-06-06T03:35:48.993266+0000","last_unstale":"2024-06-06T03:35:49.006465+0000","last_undegraded":"2024-06-06T03:35:49.006465+0000","last_fullsized":"2024-06-06T03:35:49.006465+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:47.782824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984158+0000","last_change":"2024-06-06T03:35:13.853614+0000","last_active":"2024-06-06T03:35:48.984158+0000","last_peered":"2024-06-06T03:35:48.984158+0000","last_clean":"2024-06-06T03:35:48.984158+0000","last_became_active":"2024-06-06T03:35:13.852327+0000","last_became_peered":"2024-06-06T03:35:13.852327+0000","last_unstale":"2024-06-06T03:35:48.984158+0000","last_undegraded":"2024-06-06T03:35:48.984158+0000","last_fullsized":"2024-06-06T03:35:48.984158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:02:32.463028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991648+0000","last_change":"2024-06-06T03:35:09.826220+0000","last_active":"2024-06-06T03:35:48.991648+0000","last_peered":"2024-06-06T03:35:48.991648+0000","last_clean":"2024-06-06T03:35:48.991648+0000","last_became_active":"2024-06-06T03:35:09.825887+0000","last_became_peered":"2024-06-06T03:35:09.825887+0000","last_unstale":"2024-06-06T03:35:48.991648+0000","last_undegraded":"2024-06-06T03:35:48.991648+0000","last_fullsized":"2024-06-06T03:35:48.991648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:23:33.704822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987180+0000","last_change":"2024-06-06T03:35:13.852455+0000","last_active":"2024-06-06T03:35:48.987180+0000","last_peered":"2024-06-06T03:35:48.987180+0000","last_clean":"2024-06-06T03:35:48.987180+0000","last_became_active":"2024-06-06T03:35:13.852134+0000","last_became_peered":"2024-06-06T03:35:13.852134+0000","last_unstale":"2024-06-06T03:35:48.987180+0000","last_undegraded":"2024-06-06T03:35:48.987180+0000","last_fullsized":"2024-06-06T03:35:48.987180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:00:16.418479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.986714+0000","last_change":"2024-06-06T03:35:10.277767+0000","last_active":"2024-06-06T03:35:48.986714+0000","last_peered":"2024-06-06T03:35:48.986714+0000","last_clean":"2024-06-06T03:35:48.986714+0000","last_became_active":"2024-06-06T03:35:10.277381+0000","last_became_peered":"2024-06-06T03:35:10.277381+0000","last_unstale":"2024-06-06T03:35:48.986714+0000","last_undegraded":"2024-06-06T03:35:48.986714+0000","last_fullsized":"2024-06-06T03:35:48.986714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:12:13.425284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.988580+0000","last_change":"2024-06-06T03:35:13.859594+0000","last_active":"2024-06-06T03:35:48.988580+0000","last_peered":"2024-06-06T03:35:48.988580+0000","last_clean":"2024-06-06T03:35:48.988580+0000","last_became_active":"2024-06-06T03:35:13.859240+0000","last_became_peered":"2024-06-06T03:35:13.859240+0000","last_unstale":"2024-06-06T03:35:48.988580+0000","last_undegraded":"2024-06-06T03:35:48.988580+0000","last_fullsized":"2024-06-06T03:35:48.988580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:39:34.557581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":136,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:15.093009+0000","last_change":"2024-06-06T03:35:09.826204+0000","last_active":"2024-06-06T03:37:15.093009+0000","last_peered":"2024-06-06T03:37:15.093009+0000","last_clean":"2024-06-06T03:37:15.093009+0000","last_became_active":"2024-06-06T03:35:09.825815+0000","last_became_peered":"2024-06-06T03:35:09.825815+0000","last_unstale":"2024-06-06T03:37:15.093009+0000","last_undegraded":"2024-06-06T03:37:15.093009+0000","last_fullsized":"2024-06-06T03:37:15.093009+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:02:04.169227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992786+0000","last_change":"2024-06-06T03:35:48.992786+0000","last_active":"2024-06-06T03:35:48.992786+0000","last_peered":"2024-06-06T03:35:48.992786+0000","last_clean":"2024-06-06T03:35:48.992786+0000","last_became_active":"2024-06-06T03:35:48.992456+0000","last_became_peered":"2024-06-06T03:35:48.992456+0000","last_unstale":"2024-06-06T03:35:48.992786+0000","last_undegraded":"2024-06-06T03:35:48.992786+0000","last_fullsized":"2024-06-06T03:35:48.992786+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:06:58.563975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.005564+0000","last_change":"2024-06-06T03:35:49.005119+0000","last_active":"2024-06-06T03:35:49.005564+0000","last_peered":"2024-06-06T03:35:49.005564+0000","last_clean":"2024-06-06T03:35:49.005564+0000","last_became_active":"2024-06-06T03:35:49.004788+0000","last_became_peered":"2024-06-06T03:35:49.004788+0000","last_unstale":"2024-06-06T03:35:49.005564+0000","last_undegraded":"2024-06-06T03:35:49.005564+0000","last_fullsized":"2024-06-06T03:35:49.005564+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:20:06.961454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":134,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:15.092381+0000","last_change":"2024-06-06T03:35:10.278559+0000","last_active":"2024-06-06T03:37:15.092381+0000","last_peered":"2024-06-06T03:37:15.092381+0000","last_clean":"2024-06-06T03:37:15.092381+0000","last_became_active":"2024-06-06T03:35:10.278104+0000","last_became_peered":"2024-06-06T03:35:10.278104+0000","last_unstale":"2024-06-06T03:37:15.092381+0000","last_undegraded":"2024-06-06T03:37:15.092381+0000","last_fullsized":"2024-06-06T03:37:15.092381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:23:47.131731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990973+0000","last_change":"2024-06-06T03:35:13.862303+0000","last_active":"2024-06-06T03:35:48.990973+0000","last_peered":"2024-06-06T03:35:48.990973+0000","last_clean":"2024-06-06T03:35:48.990973+0000","last_became_active":"2024-06-06T03:35:13.861978+0000","last_became_peered":"2024-06-06T03:35:13.861978+0000","last_unstale":"2024-06-06T03:35:48.990973+0000","last_undegraded":"2024-06-06T03:35:48.990973+0000","last_fullsized":"2024-06-06T03:35:48.990973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:24:00.824646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.004144+0000","last_change":"2024-06-06T03:35:09.825365+0000","last_active":"2024-06-06T03:35:49.004144+0000","last_peered":"2024-06-06T03:35:49.004144+0000","last_clean":"2024-06-06T03:35:49.004144+0000","last_became_active":"2024-06-06T03:35:09.824948+0000","last_became_peered":"2024-06-06T03:35:09.824948+0000","last_unstale":"2024-06-06T03:35:49.004144+0000","last_undegraded":"2024-06-06T03:35:49.004144+0000","last_fullsized":"2024-06-06T03:35:49.004144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:23:48.189292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":359,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985318+0000","last_change":"2024-06-06T03:33:44.124626+0000","last_active":"2024-06-06T03:35:48.985318+0000","last_peered":"2024-06-06T03:35:48.985318+0000","last_clean":"2024-06-06T03:35:48.985318+0000","last_became_active":"2024-06-06T03:33:44.124046+0000","last_became_peered":"2024-06-06T03:33:44.124046+0000","last_unstale":"2024-06-06T03:35:48.985318+0000","last_undegraded":"2024-06-06T03:35:48.985318+0000","last_fullsized":"2024-06-06T03:35:48.985318+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:30:14.586719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:30:14.586719+0000","last_clean_scrub_stamp":"2024-06-06T03:30:14.586719+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:17.604180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983301+0000","last_change":"2024-06-06T03:35:13.852044+0000","last_active":"2024-06-06T03:35:48.983301+0000","last_peered":"2024-06-06T03:35:48.983301+0000","last_clean":"2024-06-06T03:35:48.983301+0000","last_became_active":"2024-06-06T03:35:13.851715+0000","last_became_peered":"2024-06-06T03:35:13.851715+0000","last_unstale":"2024-06-06T03:35:48.983301+0000","last_undegraded":"2024-06-06T03:35:48.983301+0000","last_fullsized":"2024-06-06T03:35:48.983301+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:19:23.778367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985500+0000","last_change":"2024-06-06T03:35:09.828463+0000","last_active":"2024-06-06T03:35:48.985500+0000","last_peered":"2024-06-06T03:35:48.985500+0000","last_clean":"2024-06-06T03:35:48.985500+0000","last_became_active":"2024-06-06T03:35:09.828121+0000","last_became_peered":"2024-06-06T03:35:09.828121+0000","last_unstale":"2024-06-06T03:35:48.985500+0000","last_undegraded":"2024-06-06T03:35:48.985500+0000","last_fullsized":"2024-06-06T03:35:48.985500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:54:25.095741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.992160+0000","last_change":"2024-06-06T03:35:10.277901+0000","last_active":"2024-06-06T03:35:48.992160+0000","last_peered":"2024-06-06T03:35:48.992160+0000","last_clean":"2024-06-06T03:35:48.992160+0000","last_became_active":"2024-06-06T03:35:10.277447+0000","last_became_peered":"2024-06-06T03:35:10.277447+0000","last_unstale":"2024-06-06T03:35:48.992160+0000","last_undegraded":"2024-06-06T03:35:48.992160+0000","last_fullsized":"2024-06-06T03:35:48.992160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:22:50.550287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983583+0000","last_change":"2024-06-06T03:35:13.864323+0000","last_active":"2024-06-06T03:35:48.983583+0000","last_peered":"2024-06-06T03:35:48.983583+0000","last_clean":"2024-06-06T03:35:48.983583+0000","last_became_active":"2024-06-06T03:35:13.863101+0000","last_became_peered":"2024-06-06T03:35:13.863101+0000","last_unstale":"2024-06-06T03:35:48.983583+0000","last_undegraded":"2024-06-06T03:35:48.983583+0000","last_fullsized":"2024-06-06T03:35:48.983583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:32.399325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:57.869408+0000","last_change":"2024-06-06T03:35:09.822280+0000","last_active":"2024-06-06T03:35:57.869408+0000","last_peered":"2024-06-06T03:35:57.869408+0000","last_clean":"2024-06-06T03:35:57.869408+0000","last_became_active":"2024-06-06T03:35:09.821922+0000","last_became_peered":"2024-06-06T03:35:09.821922+0000","last_unstale":"2024-06-06T03:35:57.869408+0000","last_undegraded":"2024-06-06T03:35:57.869408+0000","last_fullsized":"2024-06-06T03:35:57.869408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:00:03.328747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991350+0000","last_change":"2024-06-06T03:35:13.853538+0000","last_active":"2024-06-06T03:35:48.991350+0000","last_peered":"2024-06-06T03:35:48.991350+0000","last_clean":"2024-06-06T03:35:48.991350+0000","last_became_active":"2024-06-06T03:35:13.853200+0000","last_became_peered":"2024-06-06T03:35:13.853200+0000","last_unstale":"2024-06-06T03:35:48.991350+0000","last_undegraded":"2024-06-06T03:35:48.991350+0000","last_fullsized":"2024-06-06T03:35:48.991350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:13:06.574272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.004336+0000","last_change":"2024-06-06T03:35:10.761675+0000","last_active":"2024-06-06T03:35:49.004336+0000","last_peered":"2024-06-06T03:35:49.004336+0000","last_clean":"2024-06-06T03:35:49.004336+0000","last_became_active":"2024-06-06T03:35:10.761287+0000","last_became_peered":"2024-06-06T03:35:10.761287+0000","last_unstale":"2024-06-06T03:35:49.004336+0000","last_undegraded":"2024-06-06T03:35:49.004336+0000","last_fullsized":"2024-06-06T03:35:49.004336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:58:18.524273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990632+0000","last_change":"2024-06-06T03:35:13.847680+0000","last_active":"2024-06-06T03:35:48.990632+0000","last_peered":"2024-06-06T03:35:48.990632+0000","last_clean":"2024-06-06T03:35:48.990632+0000","last_became_active":"2024-06-06T03:35:13.847359+0000","last_became_peered":"2024-06-06T03:35:13.847359+0000","last_unstale":"2024-06-06T03:35:48.990632+0000","last_undegraded":"2024-06-06T03:35:48.990632+0000","last_fullsized":"2024-06-06T03:35:48.990632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:29:18.041078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984544+0000","last_change":"2024-06-06T03:35:10.277491+0000","last_active":"2024-06-06T03:35:48.984544+0000","last_peered":"2024-06-06T03:35:48.984544+0000","last_clean":"2024-06-06T03:35:48.984544+0000","last_became_active":"2024-06-06T03:35:10.277135+0000","last_became_peered":"2024-06-06T03:35:10.277135+0000","last_unstale":"2024-06-06T03:35:48.984544+0000","last_undegraded":"2024-06-06T03:35:48.984544+0000","last_fullsized":"2024-06-06T03:35:48.984544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:55.198789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991400+0000","last_change":"2024-06-06T03:35:13.846013+0000","last_active":"2024-06-06T03:35:48.991400+0000","last_peered":"2024-06-06T03:35:48.991400+0000","last_clean":"2024-06-06T03:35:48.991400+0000","last_became_active":"2024-06-06T03:35:13.845650+0000","last_became_peered":"2024-06-06T03:35:13.845650+0000","last_unstale":"2024-06-06T03:35:48.991400+0000","last_undegraded":"2024-06-06T03:35:48.991400+0000","last_fullsized":"2024-06-06T03:35:48.991400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:51.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990583+0000","last_change":"2024-06-06T03:35:09.826380+0000","last_active":"2024-06-06T03:35:48.990583+0000","last_peered":"2024-06-06T03:35:48.990583+0000","last_clean":"2024-06-06T03:35:48.990583+0000","last_became_active":"2024-06-06T03:35:09.825813+0000","last_became_peered":"2024-06-06T03:35:09.825813+0000","last_unstale":"2024-06-06T03:35:48.990583+0000","last_undegraded":"2024-06-06T03:35:48.990583+0000","last_fullsized":"2024-06-06T03:35:48.990583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:24.792398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987048+0000","last_change":"2024-06-06T03:35:13.855051+0000","last_active":"2024-06-06T03:35:48.987048+0000","last_peered":"2024-06-06T03:35:48.987048+0000","last_clean":"2024-06-06T03:35:48.987048+0000","last_became_active":"2024-06-06T03:35:13.853781+0000","last_became_peered":"2024-06-06T03:35:13.853781+0000","last_unstale":"2024-06-06T03:35:48.987048+0000","last_undegraded":"2024-06-06T03:35:48.987048+0000","last_fullsized":"2024-06-06T03:35:48.987048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:17:56.174924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:49.009988+0000","last_change":"2024-06-06T03:35:49.009583+0000","last_active":"2024-06-06T03:35:49.009988+0000","last_peered":"2024-06-06T03:35:49.009988+0000","last_clean":"2024-06-06T03:35:49.009988+0000","last_became_active":"2024-06-06T03:35:48.994275+0000","last_became_peered":"2024-06-06T03:35:48.994275+0000","last_unstale":"2024-06-06T03:35:49.009988+0000","last_undegraded":"2024-06-06T03:35:49.009988+0000","last_fullsized":"2024-06-06T03:35:49.009988+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:24:54.985854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991216+0000","last_change":"2024-06-06T03:35:13.855503+0000","last_active":"2024-06-06T03:35:48.991216+0000","last_peered":"2024-06-06T03:35:48.991216+0000","last_clean":"2024-06-06T03:35:48.991216+0000","last_became_active":"2024-06-06T03:35:13.855069+0000","last_became_peered":"2024-06-06T03:35:13.855069+0000","last_unstale":"2024-06-06T03:35:48.991216+0000","last_undegraded":"2024-06-06T03:35:48.991216+0000","last_fullsized":"2024-06-06T03:35:48.991216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:43.778258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991692+0000","last_change":"2024-06-06T03:35:09.827116+0000","last_active":"2024-06-06T03:35:48.991692+0000","last_peered":"2024-06-06T03:35:48.991692+0000","last_clean":"2024-06-06T03:35:48.991692+0000","last_became_active":"2024-06-06T03:35:09.826756+0000","last_became_peered":"2024-06-06T03:35:09.826756+0000","last_unstale":"2024-06-06T03:35:48.991692+0000","last_undegraded":"2024-06-06T03:35:48.991692+0000","last_fullsized":"2024-06-06T03:35:48.991692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:31.501411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985532+0000","last_change":"2024-06-06T03:35:13.861189+0000","last_active":"2024-06-06T03:35:48.985532+0000","last_peered":"2024-06-06T03:35:48.985532+0000","last_clean":"2024-06-06T03:35:48.985532+0000","last_became_active":"2024-06-06T03:35:13.860751+0000","last_became_peered":"2024-06-06T03:35:13.860751+0000","last_unstale":"2024-06-06T03:35:48.985532+0000","last_undegraded":"2024-06-06T03:35:48.985532+0000","last_fullsized":"2024-06-06T03:35:48.985532+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:16:34.036420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":132,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:37:12.099804+0000","last_change":"2024-06-06T03:35:10.761606+0000","last_active":"2024-06-06T03:37:12.099804+0000","last_peered":"2024-06-06T03:37:12.099804+0000","last_clean":"2024-06-06T03:37:12.099804+0000","last_became_active":"2024-06-06T03:35:10.761104+0000","last_became_peered":"2024-06-06T03:35:10.761104+0000","last_unstale":"2024-06-06T03:37:12.099804+0000","last_undegraded":"2024-06-06T03:37:12.099804+0000","last_fullsized":"2024-06-06T03:37:12.099804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:51:39.867088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985751+0000","last_change":"2024-06-06T03:35:13.856047+0000","last_active":"2024-06-06T03:35:48.985751+0000","last_peered":"2024-06-06T03:35:48.985751+0000","last_clean":"2024-06-06T03:35:48.985751+0000","last_became_active":"2024-06-06T03:35:13.855718+0000","last_became_peered":"2024-06-06T03:35:13.855718+0000","last_unstale":"2024-06-06T03:35:48.985751+0000","last_undegraded":"2024-06-06T03:35:48.985751+0000","last_fullsized":"2024-06-06T03:35:48.985751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:18:40.926872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984381+0000","last_change":"2024-06-06T03:35:09.827283+0000","last_active":"2024-06-06T03:35:48.984381+0000","last_peered":"2024-06-06T03:35:48.984381+0000","last_clean":"2024-06-06T03:35:48.984381+0000","last_became_active":"2024-06-06T03:35:09.826595+0000","last_became_peered":"2024-06-06T03:35:09.826595+0000","last_unstale":"2024-06-06T03:35:48.984381+0000","last_undegraded":"2024-06-06T03:35:48.984381+0000","last_fullsized":"2024-06-06T03:35:48.984381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:34:16.398847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985446+0000","last_change":"2024-06-06T03:35:13.851900+0000","last_active":"2024-06-06T03:35:48.985446+0000","last_peered":"2024-06-06T03:35:48.985446+0000","last_clean":"2024-06-06T03:35:48.985446+0000","last_became_active":"2024-06-06T03:35:13.851477+0000","last_became_peered":"2024-06-06T03:35:13.851477+0000","last_unstale":"2024-06-06T03:35:48.985446+0000","last_undegraded":"2024-06-06T03:35:48.985446+0000","last_fullsized":"2024-06-06T03:35:48.985446+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:31:54.001226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983666+0000","last_change":"2024-06-06T03:35:09.824675+0000","last_active":"2024-06-06T03:35:48.983666+0000","last_peered":"2024-06-06T03:35:48.983666+0000","last_clean":"2024-06-06T03:35:48.983666+0000","last_became_active":"2024-06-06T03:35:09.824347+0000","last_became_peered":"2024-06-06T03:35:09.824347+0000","last_unstale":"2024-06-06T03:35:48.983666+0000","last_undegraded":"2024-06-06T03:35:48.983666+0000","last_fullsized":"2024-06-06T03:35:48.983666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:01:43.080552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.985522+0000","last_change":"2024-06-06T03:35:13.854706+0000","last_active":"2024-06-06T03:35:48.985522+0000","last_peered":"2024-06-06T03:35:48.985522+0000","last_clean":"2024-06-06T03:35:48.985522+0000","last_became_active":"2024-06-06T03:35:13.854392+0000","last_became_peered":"2024-06-06T03:35:13.854392+0000","last_unstale":"2024-06-06T03:35:48.985522+0000","last_undegraded":"2024-06-06T03:35:48.985522+0000","last_fullsized":"2024-06-06T03:35:48.985522+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:58:10.102367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987223+0000","last_change":"2024-06-06T03:35:13.853594+0000","last_active":"2024-06-06T03:35:48.987223+0000","last_peered":"2024-06-06T03:35:48.987223+0000","last_clean":"2024-06-06T03:35:48.987223+0000","last_became_active":"2024-06-06T03:35:13.852962+0000","last_became_peered":"2024-06-06T03:35:13.852962+0000","last_unstale":"2024-06-06T03:35:48.987223+0000","last_undegraded":"2024-06-06T03:35:48.987223+0000","last_fullsized":"2024-06-06T03:35:48.987223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:02:44.233868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991198+0000","last_change":"2024-06-06T03:35:13.848790+0000","last_active":"2024-06-06T03:35:48.991198+0000","last_peered":"2024-06-06T03:35:48.991198+0000","last_clean":"2024-06-06T03:35:48.991198+0000","last_became_active":"2024-06-06T03:35:13.848442+0000","last_became_peered":"2024-06-06T03:35:13.848442+0000","last_unstale":"2024-06-06T03:35:48.991198+0000","last_undegraded":"2024-06-06T03:35:48.991198+0000","last_fullsized":"2024-06-06T03:35:48.991198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:51.382210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991331+0000","last_change":"2024-06-06T03:35:13.866999+0000","last_active":"2024-06-06T03:35:48.991331+0000","last_peered":"2024-06-06T03:35:48.991331+0000","last_clean":"2024-06-06T03:35:48.991331+0000","last_became_active":"2024-06-06T03:35:13.866664+0000","last_became_peered":"2024-06-06T03:35:13.866664+0000","last_unstale":"2024-06-06T03:35:48.991331+0000","last_undegraded":"2024-06-06T03:35:48.991331+0000","last_fullsized":"2024-06-06T03:35:48.991331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:31:14.166122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990482+0000","last_change":"2024-06-06T03:35:13.849062+0000","last_active":"2024-06-06T03:35:48.990482+0000","last_peered":"2024-06-06T03:35:48.990482+0000","last_clean":"2024-06-06T03:35:48.990482+0000","last_became_active":"2024-06-06T03:35:13.848750+0000","last_became_peered":"2024-06-06T03:35:13.848750+0000","last_unstale":"2024-06-06T03:35:48.990482+0000","last_undegraded":"2024-06-06T03:35:48.990482+0000","last_fullsized":"2024-06-06T03:35:48.990482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:55:58.084908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983429+0000","last_change":"2024-06-06T03:35:13.857601+0000","last_active":"2024-06-06T03:35:48.983429+0000","last_peered":"2024-06-06T03:35:48.983429+0000","last_clean":"2024-06-06T03:35:48.983429+0000","last_became_active":"2024-06-06T03:35:13.857270+0000","last_became_peered":"2024-06-06T03:35:13.857270+0000","last_unstale":"2024-06-06T03:35:48.983429+0000","last_undegraded":"2024-06-06T03:35:48.983429+0000","last_fullsized":"2024-06-06T03:35:48.983429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:56:29.970210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987336+0000","last_change":"2024-06-06T03:35:13.846699+0000","last_active":"2024-06-06T03:35:48.987336+0000","last_peered":"2024-06-06T03:35:48.987336+0000","last_clean":"2024-06-06T03:35:48.987336+0000","last_became_active":"2024-06-06T03:35:13.846355+0000","last_became_peered":"2024-06-06T03:35:13.846355+0000","last_unstale":"2024-06-06T03:35:48.987336+0000","last_undegraded":"2024-06-06T03:35:48.987336+0000","last_fullsized":"2024-06-06T03:35:48.987336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:50:22.537339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991821+0000","last_change":"2024-06-06T03:35:13.854013+0000","last_active":"2024-06-06T03:35:48.991821+0000","last_peered":"2024-06-06T03:35:48.991821+0000","last_clean":"2024-06-06T03:35:48.991821+0000","last_became_active":"2024-06-06T03:35:13.853739+0000","last_became_peered":"2024-06-06T03:35:13.853739+0000","last_unstale":"2024-06-06T03:35:48.991821+0000","last_undegraded":"2024-06-06T03:35:48.991821+0000","last_fullsized":"2024-06-06T03:35:48.991821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:27:12.330943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.991256+0000","last_change":"2024-06-06T03:35:13.854251+0000","last_active":"2024-06-06T03:35:48.991256+0000","last_peered":"2024-06-06T03:35:48.991256+0000","last_clean":"2024-06-06T03:35:48.991256+0000","last_became_active":"2024-06-06T03:35:13.853915+0000","last_became_peered":"2024-06-06T03:35:13.853915+0000","last_unstale":"2024-06-06T03:35:48.991256+0000","last_undegraded":"2024-06-06T03:35:48.991256+0000","last_fullsized":"2024-06-06T03:35:48.991256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:08.813316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984146+0000","last_change":"2024-06-06T03:35:13.849998+0000","last_active":"2024-06-06T03:35:48.984146+0000","last_peered":"2024-06-06T03:35:48.984146+0000","last_clean":"2024-06-06T03:35:48.984146+0000","last_became_active":"2024-06-06T03:35:13.848789+0000","last_became_peered":"2024-06-06T03:35:13.848789+0000","last_unstale":"2024-06-06T03:35:48.984146+0000","last_undegraded":"2024-06-06T03:35:48.984146+0000","last_fullsized":"2024-06-06T03:35:48.984146+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:53:39.043356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.990461+0000","last_change":"2024-06-06T03:35:13.858020+0000","last_active":"2024-06-06T03:35:48.990461+0000","last_peered":"2024-06-06T03:35:48.990461+0000","last_clean":"2024-06-06T03:35:48.990461+0000","last_became_active":"2024-06-06T03:35:13.857723+0000","last_became_peered":"2024-06-06T03:35:13.857723+0000","last_unstale":"2024-06-06T03:35:48.990461+0000","last_undegraded":"2024-06-06T03:35:48.990461+0000","last_fullsized":"2024-06-06T03:35:48.990461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:08:08.651213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.984730+0000","last_change":"2024-06-06T03:35:13.867546+0000","last_active":"2024-06-06T03:35:48.984730+0000","last_peered":"2024-06-06T03:35:48.984730+0000","last_clean":"2024-06-06T03:35:48.984730+0000","last_became_active":"2024-06-06T03:35:13.867182+0000","last_became_peered":"2024-06-06T03:35:13.867182+0000","last_unstale":"2024-06-06T03:35:48.984730+0000","last_undegraded":"2024-06-06T03:35:48.984730+0000","last_fullsized":"2024-06-06T03:35:48.984730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:25:55.831943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.983770+0000","last_change":"2024-06-06T03:35:13.849080+0000","last_active":"2024-06-06T03:35:48.983770+0000","last_peered":"2024-06-06T03:35:48.983770+0000","last_clean":"2024-06-06T03:35:48.983770+0000","last_became_active":"2024-06-06T03:35:13.848476+0000","last_became_peered":"2024-06-06T03:35:13.848476+0000","last_unstale":"2024-06-06T03:35:48.983770+0000","last_undegraded":"2024-06-06T03:35:48.983770+0000","last_fullsized":"2024-06-06T03:35:48.983770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:48:37.306528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-06T03:35:48.987065+0000","last_change":"2024-06-06T03:35:13.848442+0000","last_active":"2024-06-06T03:35:48.987065+0000","last_peered":"2024-06-06T03:35:48.987065+0000","last_clean":"2024-06-06T03:35:48.987065+0000","last_became_active":"2024-06-06T03:35:13.848074+0000","last_became_peered":"2024-06-06T03:35:13.848074+0000","last_unstale":"2024-06-06T03:35:48.987065+0000","last_undegraded":"2024-06-06T03:35:48.987065+0000","last_fullsized":"2024-06-06T03:35:48.987065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:07:00.197671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280416,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69536,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1047,"num_read_kb":603,"num_write":245,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":208608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":480334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":1133585,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808883,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":480418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1073437,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":480259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":543506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":481208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":543726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":479694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":479730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":1072725,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34027,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34027,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34027,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T03:37:19.674 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-06T03:37:19.674 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-06T03:37:19.674 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-06T03:37:19.674 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-06T03:37:19.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:19 smithi131 ceph-mon[36687]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:37:19.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:19 smithi131 ceph-mon[36687]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-06-06T03:37:19.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:19 smithi104 ceph-mon[36569]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:37:19.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:19 smithi104 ceph-mon[36569]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-06-06T03:37:19.937 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:20.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:20 smithi040 ceph-mon[32220]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:37:20.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:20.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:20.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:37:20.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:37:20.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:20.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:20 smithi131 ceph-mon[36687]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:37:20.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:20.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:20.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:37:20.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:37:20.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:20.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:20 smithi104 ceph-mon[36569]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:37:20.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:20.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:20.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:37:20.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:37:20.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:37:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.620+0000 7ff0925c5700 1 -- 172.21.15.40:0/1763600609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff08c107400 msgr2=0x7ff08c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.620+0000 7ff0925c5700 1 --2- 172.21.15.40:0/1763600609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff08c107400 0x7ff08c1077e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009990 tx=0x7ff07c009ca0 comp rx=0 tx=0).stop 2024-06-06T03:37:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.620+0000 7ff0925c5700 1 -- 172.21.15.40:0/1763600609 shutdown_connections 2024-06-06T03:37:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.620+0000 7ff0925c5700 1 --2- 172.21.15.40:0/1763600609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c101d50 0x7ff08c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.620+0000 7ff0925c5700 1 --2- 172.21.15.40:0/1763600609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff08c107400 0x7ff08c1077e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.620+0000 7ff0925c5700 1 --2- 172.21.15.40:0/1763600609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff08c101390 0x7ff08c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.620+0000 7ff0925c5700 1 -- 172.21.15.40:0/1763600609 >> 172.21.15.40:0/1763600609 conn(0x7ff08c078340 msgr2=0x7ff08c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.620+0000 7ff0925c5700 1 -- 172.21.15.40:0/1763600609 shutdown_connections 2024-06-06T03:37:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.620+0000 7ff0925c5700 1 -- 172.21.15.40:0/1763600609 wait complete. 2024-06-06T03:37:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.621+0000 7ff0925c5700 1 Processor -- start 2024-06-06T03:37:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.621+0000 7ff0925c5700 1 -- start start 2024-06-06T03:37:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.621+0000 7ff0925c5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c101390 0x7ff08c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.621+0000 7ff0925c5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff08c101d50 0x7ff08c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.621+0000 7ff0925c5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff08c107400 0x7ff08c1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.621+0000 7ff0925c5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c1116f0 con 0x7ff08c107400 2024-06-06T03:37:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.621+0000 7ff0925c5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c111570 con 0x7ff08c101d50 2024-06-06T03:37:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.621+0000 7ff0925c5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c111870 con 0x7ff08c101390 2024-06-06T03:37:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.621+0000 7ff090b62700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff08c107400 0x7ff08c1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.621+0000 7ff090b62700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff08c107400 0x7ff08c1a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57920/0 (socket says 172.21.15.40:57920) 2024-06-06T03:37:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.621+0000 7ff090b62700 1 -- 172.21.15.40:0/1784019032 learned_addr learned my addr 172.21.15.40:0/1784019032 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.621+0000 7ff083fff700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff08c101d50 0x7ff08c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.622+0000 7ff08bfff700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c101390 0x7ff08c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.622+0000 7ff083fff700 1 -- 172.21.15.40:0/1784019032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c101390 msgr2=0x7ff08c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.622+0000 7ff083fff700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c101390 0x7ff08c199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.622+0000 7ff083fff700 1 -- 172.21.15.40:0/1784019032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff08c107400 msgr2=0x7ff08c1a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.622+0000 7ff083fff700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff08c107400 0x7ff08c1a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.622+0000 7ff083fff700 1 -- 172.21.15.40:0/1784019032 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c1a50b0 con 0x7ff08c101d50 2024-06-06T03:37:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.622+0000 7ff08bfff700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c101390 0x7ff08c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:37:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.622+0000 7ff083fff700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff08c101d50 0x7ff08c19a030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff07400b450 tx=0x7ff07400b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.622+0000 7ff089ffb700 1 -- 172.21.15.40:0/1784019032 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074015020 con 0x7ff08c101d50 2024-06-06T03:37:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.622+0000 7ff0925c5700 1 -- 172.21.15.40:0/1784019032 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c1a52e0 con 0x7ff08c101d50 2024-06-06T03:37:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.622+0000 7ff089ffb700 1 -- 172.21.15.40:0/1784019032 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff074011d10 con 0x7ff08c101d50 2024-06-06T03:37:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.622+0000 7ff0925c5700 1 -- 172.21.15.40:0/1784019032 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c1a5870 con 0x7ff08c101d50 2024-06-06T03:37:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.622+0000 7ff089ffb700 1 -- 172.21.15.40:0/1784019032 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074010e20 con 0x7ff08c101d50 2024-06-06T03:37:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.623+0000 7ff0925c5700 1 -- 172.21.15.40:0/1784019032 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08c069180 con 0x7ff08c101d50 2024-06-06T03:37:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.624+0000 7ff089ffb700 1 -- 172.21.15.40:0/1784019032 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff074018020 con 0x7ff08c101d50 2024-06-06T03:37:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.625+0000 7ff089ffb700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff0780a2c20 0x7ff0780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.625+0000 7ff08bfff700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff0780a2c20 0x7ff0780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.625+0000 7ff089ffb700 1 -- 172.21.15.40:0/1784019032 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7ff0740ce910 con 0x7ff08c101d50 2024-06-06T03:37:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.626+0000 7ff08bfff700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff0780a2c20 0x7ff0780a50e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff07c006010 tx=0x7ff07c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.629+0000 7ff089ffb700 1 -- 172.21.15.40:0/1784019032 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff07401f020 con 0x7ff08c101d50 2024-06-06T03:37:21.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:21 smithi131 ceph-mon[36687]: pgmap v366: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-06T03:37:21.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:21 smithi040 ceph-mon[32220]: pgmap v366: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-06T03:37:21.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:21 smithi104 ceph-mon[36569]: pgmap v366: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-06T03:37:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.883+0000 7ff0925c5700 1 -- 172.21.15.40:0/1784019032 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff08c02dca0 con 0x7ff08c101d50 2024-06-06T03:37:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.884+0000 7ff089ffb700 1 -- 172.21.15.40:0/1784019032 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff07409b930 con 0x7ff08c101d50 2024-06-06T03:37:21.883 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:37:21.884 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-06T03:37:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.886+0000 7ff0925c5700 1 -- 172.21.15.40:0/1784019032 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff0780a2c20 msgr2=0x7ff0780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.886+0000 7ff0925c5700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff0780a2c20 0x7ff0780a50e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff07c006010 tx=0x7ff07c00b540 comp rx=0 tx=0).stop 2024-06-06T03:37:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.886+0000 7ff0925c5700 1 -- 172.21.15.40:0/1784019032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff08c101d50 msgr2=0x7ff08c19a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.886+0000 7ff0925c5700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff08c101d50 0x7ff08c19a030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff07400b450 tx=0x7ff07400b760 comp rx=0 tx=0).stop 2024-06-06T03:37:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.886+0000 7ff0925c5700 1 -- 172.21.15.40:0/1784019032 shutdown_connections 2024-06-06T03:37:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.886+0000 7ff0925c5700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff0780a2c20 0x7ff0780a50e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.886+0000 7ff0925c5700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c101390 0x7ff08c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.886+0000 7ff0925c5700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff08c107400 0x7ff08c1a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.886+0000 7ff0925c5700 1 --2- 172.21.15.40:0/1784019032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff08c101d50 0x7ff08c19a030 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.886+0000 7ff0925c5700 1 -- 172.21.15.40:0/1784019032 >> 172.21.15.40:0/1784019032 conn(0x7ff08c078340 msgr2=0x7ff08c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.886+0000 7ff0925c5700 1 -- 172.21.15.40:0/1784019032 shutdown_connections 2024-06-06T03:37:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:21.886+0000 7ff0925c5700 1 -- 172.21.15.40:0/1784019032 wait complete. 2024-06-06T03:37:22.557 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:22 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1784019032' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T03:37:22.558 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-06T03:37:22.558 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:37:22.812 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:22.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:22 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1784019032' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T03:37:23.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:22 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1784019032' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T03:37:23.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:23 smithi040 ceph-mon[32220]: pgmap v367: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-06-06T03:37:23.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:23 smithi104 ceph-mon[36569]: pgmap v367: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-06-06T03:37:24.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:23 smithi131 ceph-mon[36687]: pgmap v367: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-06-06T03:37:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.401+0000 7fee9015b700 1 -- 172.21.15.40:0/4291542361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee88107840 msgr2=0x7fee8810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.401+0000 7fee9015b700 1 --2- 172.21.15.40:0/4291542361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee88107840 0x7fee8810f3c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fee8000b0d0 tx=0x7fee8000b3e0 comp rx=0 tx=0).stop 2024-06-06T03:37:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.401+0000 7fee9015b700 1 -- 172.21.15.40:0/4291542361 shutdown_connections 2024-06-06T03:37:24.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.401+0000 7fee9015b700 1 --2- 172.21.15.40:0/4291542361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee88102740 0x7fee88102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:24.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.401+0000 7fee9015b700 1 --2- 172.21.15.40:0/4291542361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee88107840 0x7fee8810f3c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:24.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.401+0000 7fee9015b700 1 --2- 172.21.15.40:0/4291542361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881030f0 0x7fee88107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:24.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.401+0000 7fee9015b700 1 -- 172.21.15.40:0/4291542361 >> 172.21.15.40:0/4291542361 conn(0x7fee880795f0 msgr2=0x7fee88079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:24.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.401+0000 7fee9015b700 1 -- 172.21.15.40:0/4291542361 shutdown_connections 2024-06-06T03:37:24.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.401+0000 7fee9015b700 1 -- 172.21.15.40:0/4291542361 wait complete. 2024-06-06T03:37:24.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee9015b700 1 Processor -- start 2024-06-06T03:37:24.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee9015b700 1 -- start start 2024-06-06T03:37:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee9015b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee88102740 0x7fee8819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee9015b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881030f0 0x7fee8819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee9015b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee88107840 0x7fee88198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee9015b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fee88112760 con 0x7fee88107840 2024-06-06T03:37:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee9015b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee881125e0 con 0x7fee881030f0 2024-06-06T03:37:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee9015b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee881128e0 con 0x7fee88102740 2024-06-06T03:37:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee8e6f8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee88107840 0x7fee88198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:24.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee8e6f8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee88107840 0x7fee88198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57942/0 (socket says 172.21.15.40:57942) 2024-06-06T03:37:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee8e6f8700 1 -- 172.21.15.40:0/822469982 learned_addr learned my addr 172.21.15.40:0/822469982 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee8def7700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee88102740 0x7fee8819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee8d6f6700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881030f0 0x7fee8819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee8e6f8700 1 -- 172.21.15.40:0/822469982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee88102740 msgr2=0x7fee8819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee8e6f8700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee88102740 0x7fee8819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee8e6f8700 1 -- 172.21.15.40:0/822469982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881030f0 msgr2=0x7fee8819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee8e6f8700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881030f0 0x7fee8819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee8e6f8700 1 -- 172.21.15.40:0/822469982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee881990c0 con 0x7fee88107840 2024-06-06T03:37:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee8def7700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee88102740 0x7fee8819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:37:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.402+0000 7fee8d6f6700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881030f0 0x7fee8819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:37:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.403+0000 7fee8e6f8700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee88107840 0x7fee88198960 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fee80006070 tx=0x7fee80004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.403+0000 7fee7effd700 1 -- 172.21.15.40:0/822469982 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80021070 con 0x7fee88107840 2024-06-06T03:37:24.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.403+0000 7fee7effd700 1 -- 172.21.15.40:0/822469982 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee80007c80 con 0x7fee88107840 2024-06-06T03:37:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.403+0000 7fee9015b700 1 -- 172.21.15.40:0/822469982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee88199350 con 0x7fee88107840 2024-06-06T03:37:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.403+0000 7fee7effd700 1 -- 172.21.15.40:0/822469982 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee800053c0 con 0x7fee88107840 2024-06-06T03:37:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.403+0000 7fee9015b700 1 -- 172.21.15.40:0/822469982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee881a5670 con 0x7fee88107840 2024-06-06T03:37:24.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.404+0000 7fee9015b700 1 -- 172.21.15.40:0/822469982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee88069180 con 0x7fee88107840 2024-06-06T03:37:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.405+0000 7fee7effd700 1 -- 172.21.15.40:0/822469982 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fee80024070 con 0x7fee88107840 2024-06-06T03:37:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.406+0000 7fee7effd700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fee740a2ca0 0x7fee740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.406+0000 7fee8def7700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fee740a2ca0 0x7fee740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.406+0000 7fee7effd700 1 -- 172.21.15.40:0/822469982 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fee8000f020 con 0x7fee88107840 2024-06-06T03:37:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.406+0000 7fee8def7700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fee740a2ca0 0x7fee740a5160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fee84005950 tx=0x7fee840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:24.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.411+0000 7fee7effd700 1 -- 172.21.15.40:0/822469982 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fee8002b020 con 0x7fee88107840 2024-06-06T03:37:24.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.664+0000 7fee9015b700 1 -- 172.21.15.40:0/822469982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee8802dca0 con 0x7fee88107840 2024-06-06T03:37:24.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.665+0000 7fee7effd700 1 -- 172.21.15.40:0/822469982 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11933 (secure 0 0 0) 0x7fee8009d640 con 0x7fee88107840 2024-06-06T03:37:24.665 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:37:24.665 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:35:38.053505+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24398,24403,24425,24443,24470]},"id":1}]} 2024-06-06T03:37:24.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.667+0000 7fee9015b700 1 -- 172.21.15.40:0/822469982 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fee740a2ca0 msgr2=0x7fee740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:24.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.667+0000 7fee9015b700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fee740a2ca0 0x7fee740a5160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fee84005950 tx=0x7fee840058e0 comp rx=0 tx=0).stop 2024-06-06T03:37:24.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.667+0000 7fee9015b700 1 -- 172.21.15.40:0/822469982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee88107840 msgr2=0x7fee88198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:24.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.667+0000 7fee9015b700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee88107840 0x7fee88198960 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fee80006070 tx=0x7fee80004db0 comp rx=0 tx=0).stop 2024-06-06T03:37:24.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.667+0000 7fee9015b700 1 -- 172.21.15.40:0/822469982 shutdown_connections 2024-06-06T03:37:24.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.667+0000 7fee9015b700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fee740a2ca0 0x7fee740a5160 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:24.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.667+0000 7fee9015b700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee88102740 0x7fee8819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:24.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.667+0000 7fee9015b700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee88107840 0x7fee88198960 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:24.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.667+0000 7fee9015b700 1 --2- 172.21.15.40:0/822469982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881030f0 0x7fee8819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:24.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.667+0000 7fee9015b700 1 -- 172.21.15.40:0/822469982 >> 172.21.15.40:0/822469982 conn(0x7fee880795f0 msgr2=0x7fee88100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:24.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.668+0000 7fee9015b700 1 -- 172.21.15.40:0/822469982 shutdown_connections 2024-06-06T03:37:24.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:24.668+0000 7fee9015b700 1 -- 172.21.15.40:0/822469982 wait complete. 2024-06-06T03:37:24.668 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 28 2024-06-06T03:37:25.383 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:37:25.638 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:25.760 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:25 smithi040 ceph-mon[32220]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T03:37:25.761 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:25 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/822469982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:37:25.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:25 smithi104 ceph-mon[36569]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T03:37:25.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:25 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/822469982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:37:26.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:25 smithi131 ceph-mon[36687]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T03:37:26.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:25 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/822469982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:37:27.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.430+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1946966525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b04107840 msgr2=0x7f5b0410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:27.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.430+0000 7f5b09cc4700 1 --2- 172.21.15.40:0/1946966525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b04107840 0x7f5b0410f3c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0099c0 tx=0x7f5afc009cd0 comp rx=0 tx=0).stop 2024-06-06T03:37:27.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.430+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1946966525 shutdown_connections 2024-06-06T03:37:27.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.430+0000 7f5b09cc4700 1 --2- 172.21.15.40:0/1946966525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b04102740 0x7f5b04102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:27.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.430+0000 7f5b09cc4700 1 --2- 172.21.15.40:0/1946966525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b04107840 0x7f5b0410f3c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:27.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.430+0000 7f5b09cc4700 1 --2- 172.21.15.40:0/1946966525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b041030f0 0x7f5b04107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:27.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.430+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1946966525 >> 172.21.15.40:0/1946966525 conn(0x7f5b040795f0 msgr2=0x7f5b04079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:27.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.430+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1946966525 shutdown_connections 2024-06-06T03:37:27.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.430+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1946966525 wait complete. 2024-06-06T03:37:27.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.430+0000 7f5b09cc4700 1 Processor -- start 2024-06-06T03:37:27.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b09cc4700 1 -- start start 2024-06-06T03:37:27.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b09cc4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b04102740 0x7f5b0419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:27.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b09cc4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b041030f0 0x7f5b0419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:27.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b09cc4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b04107840 0x7f5b0419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:27.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b09cc4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b04112830 con 0x7f5b04102740 2024-06-06T03:37:27.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b09cc4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b041126b0 con 0x7f5b04107840 2024-06-06T03:37:27.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b09cc4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b041129b0 con 0x7f5b041030f0 2024-06-06T03:37:27.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b037fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b04102740 0x7f5b0419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:27.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b037fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b04102740 0x7f5b0419ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57954/0 (socket says 172.21.15.40:57954) 2024-06-06T03:37:27.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b037fe700 1 -- 172.21.15.40:0/1915420741 learned_addr learned my addr 172.21.15.40:0/1915420741 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:27.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b03fff700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b04107840 0x7f5b0419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:27.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b02ffd700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b041030f0 0x7f5b0419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:27.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b037fe700 1 -- 172.21.15.40:0/1915420741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b041030f0 msgr2=0x7f5b0419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:27.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b037fe700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b041030f0 0x7f5b0419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:27.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b037fe700 1 -- 172.21.15.40:0/1915420741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b04107840 msgr2=0x7f5b0419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:27.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b037fe700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b04107840 0x7f5b0419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:27.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b037fe700 1 -- 172.21.15.40:0/1915420741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0419fdf0 con 0x7f5b04102740 2024-06-06T03:37:27.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.431+0000 7f5b03fff700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b04107840 0x7f5b0419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:37:27.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.432+0000 7f5b037fe700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b04102740 0x7f5b0419ae30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5af400ba50 tx=0x7f5af400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:27.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.432+0000 7f5b00ff9700 1 -- 172.21.15.40:0/1915420741 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af4019070 con 0x7f5b04102740 2024-06-06T03:37:27.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.432+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1915420741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b041a00e0 con 0x7f5b04102740 2024-06-06T03:37:27.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.432+0000 7f5b00ff9700 1 -- 172.21.15.40:0/1915420741 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5af4014440 con 0x7f5b04102740 2024-06-06T03:37:27.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.432+0000 7f5b00ff9700 1 -- 172.21.15.40:0/1915420741 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af40075d0 con 0x7f5b04102740 2024-06-06T03:37:27.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.432+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1915420741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b041a79e0 con 0x7f5b04102740 2024-06-06T03:37:27.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.433+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1915420741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b04069180 con 0x7f5b04102740 2024-06-06T03:37:27.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.434+0000 7f5b00ff9700 1 -- 172.21.15.40:0/1915420741 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5af401c070 con 0x7f5b04102740 2024-06-06T03:37:27.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.435+0000 7f5b00ff9700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5af00a2c20 0x7f5af00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:27.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.435+0000 7f5b02ffd700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5af00a2c20 0x7f5af00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:27.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.435+0000 7f5b00ff9700 1 -- 172.21.15.40:0/1915420741 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f5af40ce430 con 0x7f5b04102740 2024-06-06T03:37:27.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.435+0000 7f5b02ffd700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5af00a2c20 0x7f5af00a50e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5aec005fd0 tx=0x7f5aec00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:27.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.439+0000 7f5b00ff9700 1 -- 172.21.15.40:0/1915420741 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5af409b450 con 0x7f5b04102740 2024-06-06T03:37:27.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:27 smithi040 ceph-mon[32220]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T03:37:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.695+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1915420741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b041a7c70 con 0x7f5b04102740 2024-06-06T03:37:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.696+0000 7f5b00ff9700 1 -- 172.21.15.40:0/1915420741 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11933 (secure 0 0 0) 0x7f5af409ad50 con 0x7f5b04102740 2024-06-06T03:37:27.696 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:37:27.696 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:35:38.053505+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24398,24403,24425,24443,24470]},"id":1}]} 2024-06-06T03:37:27.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.698+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1915420741 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5af00a2c20 msgr2=0x7f5af00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:27.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.698+0000 7f5b09cc4700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5af00a2c20 0x7f5af00a50e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5aec005fd0 tx=0x7f5aec00a560 comp rx=0 tx=0).stop 2024-06-06T03:37:27.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.698+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1915420741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b04102740 msgr2=0x7f5b0419ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:27.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.698+0000 7f5b09cc4700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b04102740 0x7f5b0419ae30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5af400ba50 tx=0x7f5af400be10 comp rx=0 tx=0).stop 2024-06-06T03:37:27.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.698+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1915420741 shutdown_connections 2024-06-06T03:37:27.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.698+0000 7f5b09cc4700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f5af00a2c20 0x7f5af00a50e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:27.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.698+0000 7f5b09cc4700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b04107840 0x7f5b0419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:27.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.698+0000 7f5b09cc4700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b04102740 0x7f5b0419ae30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:27.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.698+0000 7f5b09cc4700 1 --2- 172.21.15.40:0/1915420741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b041030f0 0x7f5b0419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:27.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.698+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1915420741 >> 172.21.15.40:0/1915420741 conn(0x7f5b040795f0 msgr2=0x7f5b04100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:27.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.699+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1915420741 shutdown_connections 2024-06-06T03:37:27.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:27.699+0000 7f5b09cc4700 1 -- 172.21.15.40:0/1915420741 wait complete. 2024-06-06T03:37:27.700 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 28 2024-06-06T03:37:27.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:27 smithi104 ceph-mon[36569]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T03:37:28.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:27 smithi131 ceph-mon[36687]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T03:37:28.396 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T03:37:28.653 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:28.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:28 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1915420741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:37:28.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:28 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1915420741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:37:29.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:28 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1915420741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:37:29.721 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:29 smithi040 ceph-mon[32220]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T03:37:29.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:29 smithi104 ceph-mon[36569]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T03:37:30.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:29 smithi131 ceph-mon[36687]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T03:37:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.235+0000 7fbca4e45700 1 -- 172.21.15.40:0/386385580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca0107400 msgr2=0x7fbca01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.235+0000 7fbca4e45700 1 --2- 172.21.15.40:0/386385580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca0107400 0x7fbca01077e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009990 tx=0x7fbc90009ca0 comp rx=0 tx=0).stop 2024-06-06T03:37:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.235+0000 7fbca4e45700 1 -- 172.21.15.40:0/386385580 shutdown_connections 2024-06-06T03:37:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.235+0000 7fbca4e45700 1 --2- 172.21.15.40:0/386385580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbca0101d50 0x7fbca010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.235+0000 7fbca4e45700 1 --2- 172.21.15.40:0/386385580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca0107400 0x7fbca01077e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.235+0000 7fbca4e45700 1 --2- 172.21.15.40:0/386385580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca0101390 0x7fbca0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.235+0000 7fbca4e45700 1 -- 172.21.15.40:0/386385580 >> 172.21.15.40:0/386385580 conn(0x7fbca0078340 msgr2=0x7fbca0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.235+0000 7fbca4e45700 1 -- 172.21.15.40:0/386385580 shutdown_connections 2024-06-06T03:37:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.235+0000 7fbca4e45700 1 -- 172.21.15.40:0/386385580 wait complete. 2024-06-06T03:37:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.236+0000 7fbca4e45700 1 Processor -- start 2024-06-06T03:37:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.236+0000 7fbca4e45700 1 -- start start 2024-06-06T03:37:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.236+0000 7fbca4e45700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca0101390 0x7fbca019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.236+0000 7fbca4e45700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbca0101d50 0x7fbca019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.236+0000 7fbca4e45700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca0107400 0x7fbca01978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.236+0000 7fbca4e45700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01116f0 con 0x7fbca0107400 2024-06-06T03:37:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.236+0000 7fbca4e45700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0111570 con 0x7fbca0101d50 2024-06-06T03:37:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.236+0000 7fbca4e45700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0111870 con 0x7fbca0101390 2024-06-06T03:37:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.236+0000 7fbc9ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca0107400 0x7fbca01978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.236+0000 7fbc9ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca0107400 0x7fbca01978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40408/0 (socket says 172.21.15.40:40408) 2024-06-06T03:37:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.236+0000 7fbc9ed9d700 1 -- 172.21.15.40:0/239147140 learned_addr learned my addr 172.21.15.40:0/239147140 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.237+0000 7fbc9dd9b700 1 --2- 172.21.15.40:0/239147140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbca0101d50 0x7fbca019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.237+0000 7fbc9e59c700 1 --2- 172.21.15.40:0/239147140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca0101390 0x7fbca019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.237+0000 7fbc9ed9d700 1 -- 172.21.15.40:0/239147140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca0101390 msgr2=0x7fbca019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.237+0000 7fbc9ed9d700 1 --2- 172.21.15.40:0/239147140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca0101390 0x7fbca019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.237+0000 7fbc9ed9d700 1 -- 172.21.15.40:0/239147140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbca0101d50 msgr2=0x7fbca019dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.237+0000 7fbc9ed9d700 1 --2- 172.21.15.40:0/239147140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbca0101d50 0x7fbca019dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.237+0000 7fbc9ed9d700 1 -- 172.21.15.40:0/239147140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca0198030 con 0x7fbca0107400 2024-06-06T03:37:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.237+0000 7fbc9ed9d700 1 --2- 172.21.15.40:0/239147140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca0107400 0x7fbca01978d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800b970 tx=0x7fbc9800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.237+0000 7fbc8f7fe700 1 -- 172.21.15.40:0/239147140 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98019070 con 0x7fbca0107400 2024-06-06T03:37:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.237+0000 7fbc8f7fe700 1 -- 172.21.15.40:0/239147140 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9800ce10 con 0x7fbca0107400 2024-06-06T03:37:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.237+0000 7fbca4e45700 1 -- 172.21.15.40:0/239147140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca0198320 con 0x7fbca0107400 2024-06-06T03:37:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.237+0000 7fbc8f7fe700 1 -- 172.21.15.40:0/239147140 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98007370 con 0x7fbca0107400 2024-06-06T03:37:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.237+0000 7fbca4e45700 1 -- 172.21.15.40:0/239147140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca01a45e0 con 0x7fbca0107400 2024-06-06T03:37:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.238+0000 7fbca4e45700 1 -- 172.21.15.40:0/239147140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca0069180 con 0x7fbca0107400 2024-06-06T03:37:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.239+0000 7fbc8f7fe700 1 -- 172.21.15.40:0/239147140 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbc9801c070 con 0x7fbca0107400 2024-06-06T03:37:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.240+0000 7fbc8f7fe700 1 --2- 172.21.15.40:0/239147140 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbc880a2c20 0x7fbc880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.240+0000 7fbc8f7fe700 1 -- 172.21.15.40:0/239147140 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fbc980ce800 con 0x7fbca0107400 2024-06-06T03:37:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.240+0000 7fbc9e59c700 1 --2- 172.21.15.40:0/239147140 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbc880a2c20 0x7fbc880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.240+0000 7fbc9e59c700 1 --2- 172.21.15.40:0/239147140 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbc880a2c20 0x7fbc880a50e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc90006010 tx=0x7fbc9000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:30.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.244+0000 7fbc8f7fe700 1 -- 172.21.15.40:0/239147140 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbc9809b820 con 0x7fbca0107400 2024-06-06T03:37:30.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.448+0000 7fbca4e45700 1 -- 172.21.15.40:0/239147140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbca01a4810 con 0x7fbca0107400 2024-06-06T03:37:30.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.450+0000 7fbc8f7fe700 1 -- 172.21.15.40:0/239147140 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22534 (secure 0 0 0) 0x7fbc9809b120 con 0x7fbca0107400 2024-06-06T03:37:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:37:30.449 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"fsid":"77c25d68-23b4-11ef-bc9b-c7b262605968","created":"2024-06-06T03:26:53.165489+0000","modified":"2024-06-06T03:35:48.980461+0000","last_up_change":"2024-06-06T03:34:07.114448+0000","last_in_change":"2024-06-06T03:33:46.152948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T03:30:14.102785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T03:35:08.521025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T03:35:12.369035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63c0a320-a2f2-4b84-87e7-68501d335285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6803","nonce":3453526793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6805","nonce":3453526793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6809","nonce":3453526793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6807","nonce":3453526793}]},"public_addr":"172.21.15.40:6803/3453526793","cluster_addr":"172.21.15.40:6805/3453526793","heartbeat_back_addr":"172.21.15.40:6809/3453526793","heartbeat_front_addr":"172.21.15.40:6807/3453526793","state":["exists","up"]},{"osd":1,"uuid":"70a31534-06a1-4ffa-a54d-577d45bccf98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6801","nonce":405854252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6803","nonce":405854252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6807","nonce":405854252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6805","nonce":405854252}]},"public_addr":"172.21.15.104:6801/405854252","cluster_addr":"172.21.15.104:6803/405854252","heartbeat_back_addr":"172.21.15.104:6807/405854252","heartbeat_front_addr":"172.21.15.104:6805/405854252","state":["exists","up"]},{"osd":2,"uuid":"f5a89f43-e533-45f0-a777-bd3e9bfa8072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6801","nonce":3919221141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6803","nonce":3919221141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6807","nonce":3919221141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6805","nonce":3919221141}]},"public_addr":"172.21.15.131:6801/3919221141","cluster_addr":"172.21.15.131:6803/3919221141","heartbeat_back_addr":"172.21.15.131:6807/3919221141","heartbeat_front_addr":"172.21.15.131:6805/3919221141","state":["exists","up"]},{"osd":3,"uuid":"4df19390-6378-4346-a2be-29bce1dd2380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6811","nonce":2008541522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6813","nonce":2008541522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6817","nonce":2008541522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6815","nonce":2008541522}]},"public_addr":"172.21.15.40:6811/2008541522","cluster_addr":"172.21.15.40:6813/2008541522","heartbeat_back_addr":"172.21.15.40:6817/2008541522","heartbeat_front_addr":"172.21.15.40:6815/2008541522","state":["exists","up"]},{"osd":4,"uuid":"508de408-1cb1-4423-8199-db6bde9034f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6809","nonce":1153394847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6811","nonce":1153394847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6815","nonce":1153394847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6813","nonce":1153394847}]},"public_addr":"172.21.15.104:6809/1153394847","cluster_addr":"172.21.15.104:6811/1153394847","heartbeat_back_addr":"172.21.15.104:6815/1153394847","heartbeat_front_addr":"172.21.15.104:6813/1153394847","state":["exists","up"]},{"osd":5,"uuid":"a8c0cec9-a8ad-4a5d-800b-8b891ca6d836","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6809","nonce":1051933303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6811","nonce":1051933303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6815","nonce":1051933303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6813","nonce":1051933303}]},"public_addr":"172.21.15.131:6809/1051933303","cluster_addr":"172.21.15.131:6811/1051933303","heartbeat_back_addr":"172.21.15.131:6815/1051933303","heartbeat_front_addr":"172.21.15.131:6813/1051933303","state":["exists","up"]},{"osd":6,"uuid":"5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6819","nonce":2670922011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6821","nonce":2670922011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6825","nonce":2670922011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6823","nonce":2670922011}]},"public_addr":"172.21.15.40:6819/2670922011","cluster_addr":"172.21.15.40:6821/2670922011","heartbeat_back_addr":"172.21.15.40:6825/2670922011","heartbeat_front_addr":"172.21.15.40:6823/2670922011","state":["exists","up"]},{"osd":7,"uuid":"bfc536bb-6ae4-459e-afd9-8197c7420bd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6817","nonce":2353571051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6819","nonce":2353571051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6823","nonce":2353571051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6821","nonce":2353571051}]},"public_addr":"172.21.15.104:6817/2353571051","cluster_addr":"172.21.15.104:6819/2353571051","heartbeat_back_addr":"172.21.15.104:6823/2353571051","heartbeat_front_addr":"172.21.15.104:6821/2353571051","state":["exists","up"]},{"osd":8,"uuid":"1fe214f2-5ea3-4fc5-83bf-d9bccafe1529","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6817","nonce":1364533794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6819","nonce":1364533794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6823","nonce":1364533794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6821","nonce":1364533794}]},"public_addr":"172.21.15.131:6817/1364533794","cluster_addr":"172.21.15.131:6819/1364533794","heartbeat_back_addr":"172.21.15.131:6823/1364533794","heartbeat_front_addr":"172.21.15.131:6821/1364533794","state":["exists","up"]},{"osd":9,"uuid":"4b2b3555-6870-4ead-be50-b635799265a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6827","nonce":2604133190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6829","nonce":2604133190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6833","nonce":2604133190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6831","nonce":2604133190}]},"public_addr":"172.21.15.40:6827/2604133190","cluster_addr":"172.21.15.40:6829/2604133190","heartbeat_back_addr":"172.21.15.40:6833/2604133190","heartbeat_front_addr":"172.21.15.40:6831/2604133190","state":["exists","up"]},{"osd":10,"uuid":"70049e24-94ce-458e-97e3-dbcf9de05444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6825","nonce":2231164832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6827","nonce":2231164832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6831","nonce":2231164832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6829","nonce":2231164832}]},"public_addr":"172.21.15.104:6825/2231164832","cluster_addr":"172.21.15.104:6827/2231164832","heartbeat_back_addr":"172.21.15.104:6831/2231164832","heartbeat_front_addr":"172.21.15.104:6829/2231164832","state":["exists","up"]},{"osd":11,"uuid":"80447548-1fbe-4892-ac13-eb4bb6acf10d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6825","nonce":4128185670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6827","nonce":4128185670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6831","nonce":4128185670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6829","nonce":4128185670}]},"public_addr":"172.21.15.131:6825/4128185670","cluster_addr":"172.21.15.131:6827/4128185670","heartbeat_back_addr":"172.21.15.131:6831/4128185670","heartbeat_front_addr":"172.21.15.131:6829/4128185670","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:23.289784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:47.271677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:10.866792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:36.781924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:01.995848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:26.904606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:52.997438+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:19.221567+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:44.543977+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:11.948545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:38.402943+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:34:04.712200+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/23942578":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2234980855":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6801/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6800/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2301695357":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/1676880727":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6801/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6800/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/1597665640":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/513375601":"2024-06-07T03:27:19.799499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T03:37:30.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.452+0000 7fbca4e45700 1 -- 172.21.15.40:0/239147140 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbc880a2c20 msgr2=0x7fbc880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:30.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.452+0000 7fbca4e45700 1 --2- 172.21.15.40:0/239147140 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbc880a2c20 0x7fbc880a50e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc90006010 tx=0x7fbc9000b540 comp rx=0 tx=0).stop 2024-06-06T03:37:30.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.452+0000 7fbca4e45700 1 -- 172.21.15.40:0/239147140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca0107400 msgr2=0x7fbca01978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:30.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.452+0000 7fbca4e45700 1 --2- 172.21.15.40:0/239147140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca0107400 0x7fbca01978d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800b970 tx=0x7fbc9800bc80 comp rx=0 tx=0).stop 2024-06-06T03:37:30.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.452+0000 7fbca4e45700 1 -- 172.21.15.40:0/239147140 shutdown_connections 2024-06-06T03:37:30.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.452+0000 7fbca4e45700 1 --2- 172.21.15.40:0/239147140 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbc880a2c20 0x7fbc880a50e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:30.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.452+0000 7fbca4e45700 1 --2- 172.21.15.40:0/239147140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca0101390 0x7fbca019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:30.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.452+0000 7fbca4e45700 1 --2- 172.21.15.40:0/239147140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca0107400 0x7fbca01978d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:30.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.452+0000 7fbca4e45700 1 --2- 172.21.15.40:0/239147140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbca0101d50 0x7fbca019dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:30.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.452+0000 7fbca4e45700 1 -- 172.21.15.40:0/239147140 >> 172.21.15.40:0/239147140 conn(0x7fbca0078340 msgr2=0x7fbca00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:30.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.452+0000 7fbca4e45700 1 -- 172.21.15.40:0/239147140 shutdown_connections 2024-06-06T03:37:30.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:30.452+0000 7fbca4e45700 1 -- 172.21.15.40:0/239147140 wait complete. 2024-06-06T03:37:30.754 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:30 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/239147140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:37:30.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:30 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/239147140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:37:31.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:30 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/239147140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:37:31.076 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:37:31.326 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:31.700 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:31 smithi040 ceph-mon[32220]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T03:37:31.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:31 smithi104 ceph-mon[36569]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T03:37:31.933 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:31 smithi131 ceph-mon[36687]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T03:37:33.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.077+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1423994306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b406ac10 msgr2=0x7f22b406b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:33.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.077+0000 7f22bc2a7700 1 --2- 172.21.15.40:0/1423994306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b406ac10 0x7f22b406b090 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f22b00099c0 tx=0x7f22b0009cd0 comp rx=0 tx=0).stop 2024-06-06T03:37:33.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.077+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1423994306 shutdown_connections 2024-06-06T03:37:33.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.077+0000 7f22bc2a7700 1 --2- 172.21.15.40:0/1423994306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22b406b5d0 0x7f22b410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:33.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.077+0000 7f22bc2a7700 1 --2- 172.21.15.40:0/1423994306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b4107390 0x7f22b4107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:33.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.077+0000 7f22bc2a7700 1 --2- 172.21.15.40:0/1423994306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b406ac10 0x7f22b406b090 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:33.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.077+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1423994306 >> 172.21.15.40:0/1423994306 conn(0x7f22b40783a0 msgr2=0x7f22b40787b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:33.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.077+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1423994306 shutdown_connections 2024-06-06T03:37:33.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.077+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1423994306 wait complete. 2024-06-06T03:37:33.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22bc2a7700 1 Processor -- start 2024-06-06T03:37:33.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22bc2a7700 1 -- start start 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22bc2a7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b406ac10 0x7f22b4101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22bc2a7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b406b5d0 0x7f22b4101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22bc2a7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22b4107390 0x7f22b4101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22bc2a7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f22b4112770 con 0x7f22b406ac10 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22bc2a7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41125f0 con 0x7f22b406b5d0 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22bc2a7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41128f0 con 0x7f22b4107390 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22ba043700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b406ac10 0x7f22b4101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22ba043700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b406ac10 0x7f22b4101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40424/0 (socket says 172.21.15.40:40424) 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22ba043700 1 -- 172.21.15.40:0/1740639067 learned_addr learned my addr 172.21.15.40:0/1740639067 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22b9842700 1 --2- 172.21.15.40:0/1740639067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b406b5d0 0x7f22b4101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22ba844700 1 --2- 172.21.15.40:0/1740639067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22b4107390 0x7f22b4101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22ba043700 1 -- 172.21.15.40:0/1740639067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22b4107390 msgr2=0x7f22b4101ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22ba043700 1 --2- 172.21.15.40:0/1740639067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22b4107390 0x7f22b4101ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22ba043700 1 -- 172.21.15.40:0/1740639067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b406b5d0 msgr2=0x7f22b4101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22ba043700 1 --2- 172.21.15.40:0/1740639067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b406b5d0 0x7f22b4101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:33.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.078+0000 7f22ba043700 1 -- 172.21.15.40:0/1740639067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b41a7110 con 0x7f22b406ac10 2024-06-06T03:37:33.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.079+0000 7f22ba043700 1 --2- 172.21.15.40:0/1740639067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b406ac10 0x7f22b4101440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f22a800d950 tx=0x7f22a800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:33.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.079+0000 7f22a77fe700 1 -- 172.21.15.40:0/1740639067 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8016070 con 0x7f22b406ac10 2024-06-06T03:37:33.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.079+0000 7f22a77fe700 1 -- 172.21.15.40:0/1740639067 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a800fd70 con 0x7f22b406ac10 2024-06-06T03:37:33.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.079+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1740639067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b41a7400 con 0x7f22b406ac10 2024-06-06T03:37:33.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.079+0000 7f22a77fe700 1 -- 172.21.15.40:0/1740639067 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a800e610 con 0x7f22b406ac10 2024-06-06T03:37:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.079+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1740639067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b41a7910 con 0x7f22b406ac10 2024-06-06T03:37:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.080+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1740639067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b4069180 con 0x7f22b406ac10 2024-06-06T03:37:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.081+0000 7f22a77fe700 1 -- 172.21.15.40:0/1740639067 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f22a8019070 con 0x7f22b406ac10 2024-06-06T03:37:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.082+0000 7f22a77fe700 1 --2- 172.21.15.40:0/1740639067 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f22a00a2c20 0x7f22a00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.082+0000 7f22b9842700 1 --2- 172.21.15.40:0/1740639067 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f22a00a2c20 0x7f22a00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.082+0000 7f22a77fe700 1 -- 172.21.15.40:0/1740639067 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f22a80ce990 con 0x7f22b406ac10 2024-06-06T03:37:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.082+0000 7f22b9842700 1 --2- 172.21.15.40:0/1740639067 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f22a00a2c20 0x7f22a00a50e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f22b4102ed0 tx=0x7f22b0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.086+0000 7f22a77fe700 1 -- 172.21.15.40:0/1740639067 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f22a809b9b0 con 0x7f22b406ac10 2024-06-06T03:37:33.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.337+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1740639067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22b402dca0 con 0x7f22b406ac10 2024-06-06T03:37:33.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.338+0000 7f22a77fe700 1 -- 172.21.15.40:0/1740639067 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11933 (secure 0 0 0) 0x7f22a8095e20 con 0x7f22b406ac10 2024-06-06T03:37:33.337 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:37:33.337 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:35:38.053505+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24398,24403,24425,24443,24470]},"id":1}]} 2024-06-06T03:37:33.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.340+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1740639067 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f22a00a2c20 msgr2=0x7f22a00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:33.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.340+0000 7f22bc2a7700 1 --2- 172.21.15.40:0/1740639067 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f22a00a2c20 0x7f22a00a50e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f22b4102ed0 tx=0x7f22b0005fb0 comp rx=0 tx=0).stop 2024-06-06T03:37:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.340+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1740639067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b406ac10 msgr2=0x7f22b4101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.340+0000 7f22bc2a7700 1 --2- 172.21.15.40:0/1740639067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b406ac10 0x7f22b4101440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f22a800d950 tx=0x7f22a800dc60 comp rx=0 tx=0).stop 2024-06-06T03:37:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.340+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1740639067 shutdown_connections 2024-06-06T03:37:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.340+0000 7f22bc2a7700 1 --2- 172.21.15.40:0/1740639067 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f22a00a2c20 0x7f22a00a50e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.340+0000 7f22bc2a7700 1 --2- 172.21.15.40:0/1740639067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22b4107390 0x7f22b4101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.340+0000 7f22bc2a7700 1 --2- 172.21.15.40:0/1740639067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b406ac10 0x7f22b4101440 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.340+0000 7f22bc2a7700 1 --2- 172.21.15.40:0/1740639067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b406b5d0 0x7f22b4101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.340+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1740639067 >> 172.21.15.40:0/1740639067 conn(0x7f22b40783a0 msgr2=0x7f22b40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.340+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1740639067 shutdown_connections 2024-06-06T03:37:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:33.340+0000 7f22bc2a7700 1 -- 172.21.15.40:0/1740639067 wait complete. 2024-06-06T03:37:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 28 2024-06-06T03:37:33.720 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:33 smithi040 ceph-mon[32220]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-06T03:37:33.721 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:33 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1740639067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:37:33.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:33 smithi104 ceph-mon[36569]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-06T03:37:33.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:33 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1740639067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:37:33.977 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-06T03:35:15.961853+0000', 'modified': '2024-06-06T03:35:38.053505+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24470, 'mds_1': 24443, 'mds_2': 24425, 'mds_3': 24403, 'mds_4': 24398}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14580': {'gid': 14580, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6839/1930069210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6838', 'nonce': 1930069210}, {'type': 'v1', 'addr': '172.21.15.40:6839', 'nonce': 1930069210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14586': {'gid': 14586, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6841/2460301066', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6840', 'nonce': 2460301066}, {'type': 'v1', 'addr': '172.21.15.40:6841', 'nonce': 2460301066}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24358': {'gid': 24358, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6833/1833802668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 1833802668}, {'type': 'v1', 'addr': '172.21.15.131:6833', 'nonce': 1833802668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24376': {'gid': 24376, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/1221528906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 1221528906}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1221528906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24388': {'gid': 24388, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6837/526159320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6836', 'nonce': 526159320}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 526159320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24398': {'gid': 24398, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.104:6833/1414464971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6832', 'nonce': 1414464971}, {'type': 'v1', 'addr': '172.21.15.104:6833', 'nonce': 1414464971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.131:6839/782268782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 782268782}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 782268782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.104:6835/75462915', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 75462915}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 75462915}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.104:6837/2498466212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 2498466212}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 2498466212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.104:6839/489480648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 489480648}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 489480648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24470, 'qdb_cluster': [24398, 24403, 24425, 24443, 24470]} 2024-06-06T03:37:33.978 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-06T03:37:33.978 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.k status 2024-06-06T03:37:34.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:33 smithi131 ceph-mon[36687]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-06T03:37:34.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:33 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1740639067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:37:34.227 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:34.677 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:34 smithi040 ceph-mon[32220]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-06-06T03:37:34.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:34 smithi104 ceph-mon[36569]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-06-06T03:37:35.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:34 smithi131 ceph-mon[36687]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-06-06T03:37:35.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.862+0000 7f17aa3c5700 1 -- 172.21.15.40:0/3119607702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17a41030f0 msgr2=0x7f17a4107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:35.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.862+0000 7f17aa3c5700 1 --2- 172.21.15.40:0/3119607702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17a41030f0 0x7f17a4107110 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f178c0099c0 tx=0x7f178c009cd0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.862+0000 7f17aa3c5700 1 -- 172.21.15.40:0/3119607702 shutdown_connections 2024-06-06T03:37:35.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.862+0000 7f17aa3c5700 1 --2- 172.21.15.40:0/3119607702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a4102740 0x7f17a4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.862+0000 7f17aa3c5700 1 --2- 172.21.15.40:0/3119607702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17a4107840 0x7f17a410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.862+0000 7f17aa3c5700 1 --2- 172.21.15.40:0/3119607702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17a41030f0 0x7f17a4107110 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.862+0000 7f17aa3c5700 1 -- 172.21.15.40:0/3119607702 >> 172.21.15.40:0/3119607702 conn(0x7f17a40795f0 msgr2=0x7f17a4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:35.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.862+0000 7f17aa3c5700 1 -- 172.21.15.40:0/3119607702 shutdown_connections 2024-06-06T03:37:35.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.862+0000 7f17aa3c5700 1 -- 172.21.15.40:0/3119607702 wait complete. 2024-06-06T03:37:35.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.863+0000 7f17aa3c5700 1 Processor -- start 2024-06-06T03:37:35.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.863+0000 7f17aa3c5700 1 -- start start 2024-06-06T03:37:35.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.863+0000 7f17aa3c5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17a4102740 0x7f17a4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.863+0000 7f17aa3c5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17a41030f0 0x7f17a4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.863+0000 7f17aa3c5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a4107840 0x7f17a4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.863+0000 7f17aa3c5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4112770 con 0x7f17a41030f0 2024-06-06T03:37:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.863+0000 7f17aa3c5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41125f0 con 0x7f17a4102740 2024-06-06T03:37:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.863+0000 7f17aa3c5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41128f0 con 0x7f17a4107840 2024-06-06T03:37:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f179bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17a41030f0 0x7f17a4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f179bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17a41030f0 0x7f17a4074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40436/0 (socket says 172.21.15.40:40436) 2024-06-06T03:37:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f179bfff700 1 -- 172.21.15.40:0/1652265143 learned_addr learned my addr 172.21.15.40:0/1652265143 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f17a3fff700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17a4102740 0x7f17a4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f17a8962700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a4107840 0x7f17a4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f179bfff700 1 -- 172.21.15.40:0/1652265143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a4107840 msgr2=0x7f17a4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f179bfff700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a4107840 0x7f17a4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f179bfff700 1 -- 172.21.15.40:0/1652265143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17a4102740 msgr2=0x7f17a4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f179bfff700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17a4102740 0x7f17a4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f179bfff700 1 -- 172.21.15.40:0/1652265143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a41a70b0 con 0x7f17a41030f0 2024-06-06T03:37:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f17a3fff700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17a4102740 0x7f17a4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:37:35.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f179bfff700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17a41030f0 0x7f17a4074cd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f178c006010 tx=0x7f178c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f17a1ffb700 1 -- 172.21.15.40:0/1652265143 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c027070 con 0x7f17a41030f0 2024-06-06T03:37:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f17aa3c5700 1 -- 172.21.15.40:0/1652265143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a41a7340 con 0x7f17a41030f0 2024-06-06T03:37:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f17a1ffb700 1 -- 172.21.15.40:0/1652265143 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178c003780 con 0x7f17a41030f0 2024-06-06T03:37:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f17a1ffb700 1 -- 172.21.15.40:0/1652265143 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c022da0 con 0x7f17a41030f0 2024-06-06T03:37:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.864+0000 7f17aa3c5700 1 -- 172.21.15.40:0/1652265143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a41a77a0 con 0x7f17a41030f0 2024-06-06T03:37:35.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.867+0000 7f17a1ffb700 1 -- 172.21.15.40:0/1652265143 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f178c02a070 con 0x7f17a41030f0 2024-06-06T03:37:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.868+0000 7f17a1ffb700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f17900a2c20 0x7f17900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.868+0000 7f17a3fff700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f17900a2c20 0x7f17900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.868+0000 7f17a1ffb700 1 -- 172.21.15.40:0/1652265143 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f178c0cfe30 con 0x7f17a41030f0 2024-06-06T03:37:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f17b4c48b80 1 Processor -- start 2024-06-06T03:37:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f17a3fff700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f17900a2c20 0x7f17900a50e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f179400bcf0 tx=0x7f179400b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:35.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f17b4c48b80 1 -- start start 2024-06-06T03:37:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f17b4c48b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732def10 0x55df732df2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f17b4c48b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732d9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f17b4c48b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732d9750 0x55df732d9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f17b4c48b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55df731cbf10 con 0x55df732df830 2024-06-06T03:37:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f17b4c48b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55df7322a800 con 0x55df732def10 2024-06-06T03:37:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f17b4c48b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55df73192410 con 0x55df732d9750 2024-06-06T03:37:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f179bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732d9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f179bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732d9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40448/0 (socket says 172.21.15.40:40448) 2024-06-06T03:37:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f179bfff700 1 -- 172.21.15.40:0/497923084 learned_addr learned my addr 172.21.15.40:0/497923084 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f17a8962700 1 --2- 172.21.15.40:0/497923084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732d9750 0x55df732d9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f17a3fff700 1 --2- 172.21.15.40:0/497923084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732def10 0x55df732df2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f179bfff700 1 -- 172.21.15.40:0/497923084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732d9750 msgr2=0x55df732d9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f179bfff700 1 --2- 172.21.15.40:0/497923084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732d9750 0x55df732d9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f179bfff700 1 -- 172.21.15.40:0/497923084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732def10 msgr2=0x55df732df2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:35.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f179bfff700 1 --2- 172.21.15.40:0/497923084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732def10 0x55df732df2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f179bfff700 1 -- 172.21.15.40:0/497923084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df73179880 con 0x55df732df830 2024-06-06T03:37:35.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f17a3fff700 1 --2- 172.21.15.40:0/497923084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732def10 0x55df732df2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:37:35.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.869+0000 7f179bfff700 1 --2- 172.21.15.40:0/497923084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732d9210 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f178c098ba0 tx=0x7f178c0227a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ec932c9bb3ae12 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:35.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.870+0000 7f179a7fc700 1 -- 172.21.15.40:0/497923084 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c09ce40 con 0x55df732df830 2024-06-06T03:37:35.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.870+0000 7f179a7fc700 1 -- 172.21.15.40:0/497923084 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178c09c740 con 0x55df732df830 2024-06-06T03:37:35.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.870+0000 7f179a7fc700 1 -- 172.21.15.40:0/497923084 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c09cc00 con 0x55df732df830 2024-06-06T03:37:35.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.870+0000 7f17b4c48b80 1 -- 172.21.15.40:0/497923084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 msgr2=0x55df732d9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:35.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.870+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/497923084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732d9210 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f178c098ba0 tx=0x7f178c0227a0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.870+0000 7f17b4c48b80 1 -- 172.21.15.40:0/497923084 shutdown_connections 2024-06-06T03:37:35.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.870+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/497923084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732d9750 0x55df732d9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.870+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/497923084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732def10 0x55df732df2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.870+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/497923084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732d9210 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.870+0000 7f17b4c48b80 1 -- 172.21.15.40:0/497923084 >> 172.21.15.40:0/497923084 conn(0x55df73206030 msgr2=0x55df732d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:35.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.870+0000 7f17b4c48b80 1 -- 172.21.15.40:0/497923084 shutdown_connections 2024-06-06T03:37:35.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.870+0000 7f17b4c48b80 1 -- 172.21.15.40:0/497923084 wait complete. 2024-06-06T03:37:35.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.871+0000 7f17b4c48b80 1 -- start start 2024-06-06T03:37:35.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.871+0000 7f17b4c48b80 1 Processor -- start 2024-06-06T03:37:35.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.871+0000 7f17b4c48b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732e5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.871+0000 7f17b4c48b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732def10 0x55df732e28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.871+0000 7f17b4c48b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732e2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.871+0000 7f17b4c48b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55df731caf70 con 0x55df732df830 2024-06-06T03:37:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.871+0000 7f17b4c48b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55df732267c0 con 0x55df732d9750 2024-06-06T03:37:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.871+0000 7f17b4c48b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55df732e5fc0 con 0x55df732def10 2024-06-06T03:37:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.871+0000 7f17b4c48b80 10 client.0 _ll_register_callbacks cb 0x55df731c0560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:37:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.871+0000 7f17a8962700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732e2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17a8962700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732e2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40458/0 (socket says 172.21.15.40:40458) 2024-06-06T03:37:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17a8962700 1 -- 172.21.15.40:0/1103198211 learned_addr learned my addr 172.21.15.40:0/1103198211 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17a3fff700 1 --2- 172.21.15.40:0/1103198211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f179bfff700 1 --2- 172.21.15.40:0/1103198211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732def10 0x55df732e28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17a8962700 1 -- 172.21.15.40:0/1103198211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732def10 msgr2=0x55df732e28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17a8962700 1 --2- 172.21.15.40:0/1103198211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732def10 0x55df732e28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17a8962700 1 -- 172.21.15.40:0/1103198211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 msgr2=0x55df732e5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17a8962700 1 --2- 172.21.15.40:0/1103198211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732e5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17a8962700 1 -- 172.21.15.40:0/1103198211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df731f0700 con 0x55df732df830 2024-06-06T03:37:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17a8962700 1 --2- 172.21.15.40:0/1103198211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732e2e20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f179c012d70 tx=0x7f179c010520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bf6c88c7e349d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17837fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:37:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17837fe700 1 -- 172.21.15.40:0/1103198211 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c01a070 con 0x55df732df830 2024-06-06T03:37:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17b4c48b80 1 -- 172.21.15.40:0/1103198211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178c09c740 con 0x55df732df830 2024-06-06T03:37:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17837fe700 1 -- 172.21.15.40:0/1103198211 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f179c010d40 con 0x55df732df830 2024-06-06T03:37:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17837fe700 1 -- 172.21.15.40:0/1103198211 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f179c025050 con 0x55df732df830 2024-06-06T03:37:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17b4c48b80 10 client.14940 fetch_fsmap learned FSMap version 28 2024-06-06T03:37:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.872+0000 7f17b4c48b80 1 -- 172.21.15.40:0/1103198211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df732f0710 con 0x55df732df830 2024-06-06T03:37:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.873+0000 7f17837fe700 1 -- 172.21.15.40:0/1103198211 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178c09c740 con 0x55df732df830 2024-06-06T03:37:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.873+0000 7f17837fe700 1 -- 172.21.15.40:0/1103198211 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f179c0143a0 con 0x55df732df830 2024-06-06T03:37:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.873+0000 7f17b4c48b80 10 client.14940 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:37:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.873+0000 7f17b4c48b80 10 client.14940 resolve_mds: resolved name 'k' to daemon mds.k 2024-06-06T03:37:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.873+0000 7f17b4c48b80 20 client.14940 populate_metadata read hostname 'smithi040' 2024-06-06T03:37:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.873+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/1103198211 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x55df732f2d40 0x55df732f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.873+0000 7f17b4c48b80 4 client.14940 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.873+0000 7f17b4c48b80 1 -- 172.21.15.40:0/1103198211 --> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df73237d00 con 0x55df732f2d40 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.873+0000 7f17a3fff700 1 --2- 172.21.15.40:0/1103198211 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x55df732f2d40 0x55df732f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.874+0000 7f17837fe700 10 client.14940 ms_handle_connect on v2:172.21.15.104:6832/1414464971 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.874+0000 7f17a3fff700 1 --2- 172.21.15.40:0/1103198211 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x55df732f2d40 0x55df732f41b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=664385021c9629b8 server_cookie=8c456c08125dfca in_seq=0 out_seq=0 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.874+0000 7f17837fe700 1 -- 172.21.15.40:0/1103198211 <== mds.4 v2:172.21.15.104:6832/1414464971 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55df73237d00 con 0x55df732f2d40 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.874+0000 7f17837fe700 10 client.14940 handle_command_reply: tid=0 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.874+0000 7f17b4c48b80 1 client.14940 shutdown 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.875+0000 7f17b4c48b80 1 -- 172.21.15.40:0/1103198211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 msgr2=0x55df732e2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.875+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/1103198211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732e2e20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f179c012d70 tx=0x7f179c010520 comp rx=0 tx=0).stop 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.875+0000 7f17b4c48b80 1 -- 172.21.15.40:0/1103198211 shutdown_connections 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.875+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/1103198211 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x55df732f2d40 0x55df732f41b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.875+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/1103198211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732e2e20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.875+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/1103198211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732e5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.875+0000 7f17837fe700 0 client.14940 ms_handle_reset on v2:172.21.15.104:6832/1414464971 2024-06-06T03:37:35.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.875+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/1103198211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732def10 0x55df732e28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.875+0000 7f17b4c48b80 1 -- 172.21.15.40:0/1103198211 >> 172.21.15.40:0/1103198211 conn(0x55df73206030 msgr2=0x55df732d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:35.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.875+0000 7f17b4c48b80 1 -- 172.21.15.40:0/1103198211 shutdown_connections 2024-06-06T03:37:35.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.875+0000 7f17b4c48b80 1 -- 172.21.15.40:0/1103198211 wait complete. 2024-06-06T03:37:35.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.875+0000 7f17b4c48b80 20 client.14940 trim_cache size 0 max 16384 2024-06-06T03:37:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17b4c48b80 1 Processor -- start 2024-06-06T03:37:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17b4c48b80 1 -- start start 2024-06-06T03:37:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17b4c48b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732d9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17b4c48b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732def10 0x55df732f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17b4c48b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732dd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17b4c48b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55df732f24f0 con 0x55df732df830 2024-06-06T03:37:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17b4c48b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55df732f2bc0 con 0x55df732d9750 2024-06-06T03:37:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17b4c48b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55df732f3290 con 0x55df732def10 2024-06-06T03:37:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17a8962700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732dd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17a8962700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732dd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40470/0 (socket says 172.21.15.40:40470) 2024-06-06T03:37:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17a8962700 1 -- 172.21.15.40:0/764462744 learned_addr learned my addr 172.21.15.40:0/764462744 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17a3fff700 1 --2- 172.21.15.40:0/764462744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732d9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17a8962700 1 -- 172.21.15.40:0/764462744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732def10 msgr2=0x55df732f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17a8962700 1 --2- 172.21.15.40:0/764462744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732def10 0x55df732f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17a8962700 1 -- 172.21.15.40:0/764462744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 msgr2=0x55df732d9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17a8962700 1 --2- 172.21.15.40:0/764462744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732d9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17a8962700 1 -- 172.21.15.40:0/764462744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df732dea00 con 0x55df732df830 2024-06-06T03:37:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17a3fff700 1 --2- 172.21.15.40:0/764462744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732d9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:37:35.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.897+0000 7f17a8962700 1 --2- 172.21.15.40:0/764462744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732dd000 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f179c00a040 tx=0x7f179c012f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5c3ecb44ccab25 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:35.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.898+0000 7f1799ffb700 1 -- 172.21.15.40:0/764462744 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c01a040 con 0x55df732df830 2024-06-06T03:37:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.898+0000 7f1799ffb700 1 -- 172.21.15.40:0/764462744 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179c010d40 con 0x55df732df830 2024-06-06T03:37:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.898+0000 7f1799ffb700 1 -- 172.21.15.40:0/764462744 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c003c60 con 0x55df732df830 2024-06-06T03:37:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.898+0000 7f17b4c48b80 1 -- 172.21.15.40:0/764462744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 msgr2=0x55df732dd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.898+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/764462744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732dd000 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f179c00a040 tx=0x7f179c012f60 comp rx=0 tx=0).stop 2024-06-06T03:37:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.898+0000 7f17b4c48b80 1 -- 172.21.15.40:0/764462744 shutdown_connections 2024-06-06T03:37:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.898+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/764462744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732df830 0x55df732dd000 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.898+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/764462744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732d9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.898+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/764462744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732def10 0x55df732f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.898+0000 7f17b4c48b80 1 -- 172.21.15.40:0/764462744 >> 172.21.15.40:0/764462744 conn(0x55df73206030 msgr2=0x55df732dfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.898+0000 7f17b4c48b80 1 -- 172.21.15.40:0/764462744 shutdown_connections 2024-06-06T03:37:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.898+0000 7f17b4c48b80 1 -- 172.21.15.40:0/764462744 wait complete. 2024-06-06T03:37:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.899+0000 7f17b4c48b80 1 -- start start 2024-06-06T03:37:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.899+0000 7f17b4c48b80 1 Processor -- start 2024-06-06T03:37:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.899+0000 7f17b4c48b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732fddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.899+0000 7f17b4c48b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732def10 0x55df732d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.899+0000 7f17b4c48b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732df830 0x55df732d3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.899+0000 7f17b4c48b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55df732f4030 con 0x55df732def10 2024-06-06T03:37:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.899+0000 7f17b4c48b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55df732f3960 con 0x55df732d9750 2024-06-06T03:37:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.899+0000 7f17b4c48b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55df732f4700 con 0x55df732df830 2024-06-06T03:37:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.899+0000 7f17b4c48b80 10 client.0 _ll_register_callbacks cb 0x55df731c0560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:37:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.899+0000 7f179bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732def10 0x55df732d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.899+0000 7f17a3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732fddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f179bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732def10 0x55df732d33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40484/0 (socket says 172.21.15.40:40484) 2024-06-06T03:37:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f17a3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732fddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:33162/0 (socket says 172.21.15.40:33162) 2024-06-06T03:37:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f179bfff700 1 -- 172.21.15.40:0/495239554 learned_addr learned my addr 172.21.15.40:0/495239554 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f17a8962700 1 --2- 172.21.15.40:0/495239554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732df830 0x55df732d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f17a3fff700 1 -- 172.21.15.40:0/495239554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732df830 msgr2=0x55df732d3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f17a3fff700 1 --2- 172.21.15.40:0/495239554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732df830 0x55df732d3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f17a3fff700 1 -- 172.21.15.40:0/495239554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732def10 msgr2=0x55df732d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f17a3fff700 1 --2- 172.21.15.40:0/495239554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732def10 0x55df732d33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f17a3fff700 1 -- 172.21.15.40:0/495239554 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df732d4140 con 0x55df732d9750 2024-06-06T03:37:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f179bfff700 1 --2- 172.21.15.40:0/495239554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732def10 0x55df732d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:37:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f17a3fff700 1 --2- 172.21.15.40:0/495239554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732fddf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55df732f4b70 tx=0x7f17940047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5684ff8f9139ad53 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f17837fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-06-06T03:37:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f17837fe700 1 -- 172.21.15.40:0/495239554 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179402c070 con 0x55df732d9750 2024-06-06T03:37:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f17b4c48b80 1 -- 172.21.15.40:0/495239554 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f179c010d40 con 0x55df732d9750 2024-06-06T03:37:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f17837fe700 1 -- 172.21.15.40:0/495239554 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f17940284c0 con 0x55df732d9750 2024-06-06T03:37:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f17837fe700 1 -- 172.21.15.40:0/495239554 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1794038050 con 0x55df732d9750 2024-06-06T03:37:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.900+0000 7f17b4c48b80 10 client.24710 fetch_fsmap learned FSMap version 28 2024-06-06T03:37:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.901+0000 7f17b4c48b80 1 -- 172.21.15.40:0/495239554 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df7335e5d0 con 0x55df732d9750 2024-06-06T03:37:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.901+0000 7f17837fe700 1 -- 172.21.15.40:0/495239554 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179c010d40 con 0x55df732d9750 2024-06-06T03:37:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.901+0000 7f17837fe700 1 -- 172.21.15.40:0/495239554 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1794031840 con 0x55df732d9750 2024-06-06T03:37:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.901+0000 7f17b4c48b80 10 client.24710 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:37:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.901+0000 7f17b4c48b80 10 client.24710 resolve_mds: resolved name 'k' to daemon mds.k 2024-06-06T03:37:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.901+0000 7f17b4c48b80 20 client.24710 populate_metadata read hostname 'smithi040' 2024-06-06T03:37:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.901+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/495239554 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x55df73363170 0x55df73361e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.901+0000 7f17b4c48b80 4 client.24710 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:37:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.901+0000 7f17b4c48b80 1 -- 172.21.15.40:0/495239554 --> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] -- command(tid 0: {"prefix": "status"}) -- 0x55df73232280 con 0x55df73363170 2024-06-06T03:37:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.901+0000 7f179bfff700 1 --2- 172.21.15.40:0/495239554 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x55df73363170 0x55df73361e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f179bfff700 1 --2- 172.21.15.40:0/495239554 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x55df73363170 0x55df73361e80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e9d7e1bd90eefb47 server_cookie=434e0b32644b1e16 in_seq=0 out_seq=0 2024-06-06T03:37:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17837fe700 10 client.24710 ms_handle_connect on v2:172.21.15.104:6832/1414464971 2024-06-06T03:37:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17837fe700 1 -- 172.21.15.40:0/495239554 <== mds.4 v2:172.21.15.104:6832/1414464971 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55df73232280 con 0x55df73363170 2024-06-06T03:37:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17837fe700 10 client.24710 handle_command_reply: tid=0 2024-06-06T03:37:35.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17b4c48b80 1 client.24710 shutdown 2024-06-06T03:37:35.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17b4c48b80 1 -- 172.21.15.40:0/495239554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 msgr2=0x55df732fddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:35.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/495239554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732fddf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55df732f4b70 tx=0x7f17940047c0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17b4c48b80 1 -- 172.21.15.40:0/495239554 shutdown_connections 2024-06-06T03:37:35.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/495239554 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x55df73363170 0x55df73361e80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/495239554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df732df830 0x55df732d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/495239554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df732d9750 0x55df732fddf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17837fe700 0 client.24710 ms_handle_reset on v2:172.21.15.104:6832/1414464971 2024-06-06T03:37:35.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17b4c48b80 1 --2- 172.21.15.40:0/495239554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df732def10 0x55df732d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17b4c48b80 1 -- 172.21.15.40:0/495239554 >> 172.21.15.40:0/495239554 conn(0x55df73206030 msgr2=0x55df732de280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:35.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17b4c48b80 1 -- 172.21.15.40:0/495239554 shutdown_connections 2024-06-06T03:37:35.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17b4c48b80 1 -- 172.21.15.40:0/495239554 wait complete. 2024-06-06T03:37:35.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.902+0000 7f17b4c48b80 20 client.24710 trim_cache size 0 max 16384 2024-06-06T03:37:35.907 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:37:35.907 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:37:35.907 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 4, 2024-06-06T03:37:35.907 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24398, 2024-06-06T03:37:35.907 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:37:35.907 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:37:35.907 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:37:35.907 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 121.97239354600001, 2024-06-06T03:37:35.907 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 28, 2024-06-06T03:37:35.908 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 90, 2024-06-06T03:37:35.908 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 88, 2024-06-06T03:37:35.908 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 189.70395917799999 2024-06-06T03:37:35.908 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:37:35.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.903+0000 7f17aa3c5700 1 -- 172.21.15.40:0/1652265143 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f17900a2c20 msgr2=0x7f17900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:35.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.903+0000 7f17aa3c5700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f17900a2c20 0x7f17900a50e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f179400bcf0 tx=0x7f179400b2c0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.903+0000 7f17aa3c5700 1 -- 172.21.15.40:0/1652265143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17a41030f0 msgr2=0x7f17a4074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:35.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.903+0000 7f17aa3c5700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17a41030f0 0x7f17a4074cd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f178c006010 tx=0x7f178c009f80 comp rx=0 tx=0).stop 2024-06-06T03:37:35.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.903+0000 7f17aa3c5700 1 -- 172.21.15.40:0/1652265143 shutdown_connections 2024-06-06T03:37:35.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.903+0000 7f17aa3c5700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f17900a2c20 0x7f17900a50e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.903+0000 7f17aa3c5700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a4107840 0x7f17a4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.903+0000 7f17aa3c5700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17a4102740 0x7f17a4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.903+0000 7f17aa3c5700 1 --2- 172.21.15.40:0/1652265143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17a41030f0 0x7f17a4074cd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:35.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.903+0000 7f17aa3c5700 1 -- 172.21.15.40:0/1652265143 >> 172.21.15.40:0/1652265143 conn(0x7f17a40795f0 msgr2=0x7f17a4101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:35.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.903+0000 7f17aa3c5700 1 -- 172.21.15.40:0/1652265143 shutdown_connections 2024-06-06T03:37:35.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:35.903+0000 7f17aa3c5700 1 -- 172.21.15.40:0/1652265143 wait complete. 2024-06-06T03:37:36.572 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-06T03:37:36.826 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:36.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:36 smithi040 ceph-mon[32220]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-06-06T03:37:37.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:36 smithi131 ceph-mon[36687]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-06-06T03:37:37.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:36 smithi104 ceph-mon[36569]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-06-06T03:37:38.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.392+0000 7fd9cd31b700 1 -- 172.21.15.40:0/2811162953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c810eef0 msgr2=0x7fd9c81112e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:38.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.392+0000 7fd9cd31b700 1 --2- 172.21.15.40:0/2811162953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c810eef0 0x7fd9c81112e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009990 tx=0x7fd9c0009ca0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.392+0000 7fd9cd31b700 1 -- 172.21.15.40:0/2811162953 shutdown_connections 2024-06-06T03:37:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.392+0000 7fd9cd31b700 1 --2- 172.21.15.40:0/2811162953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c810eef0 0x7fd9c81112e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.392+0000 7fd9cd31b700 1 --2- 172.21.15.40:0/2811162953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9c80746c0 0x7fd9c8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.392+0000 7fd9cd31b700 1 --2- 172.21.15.40:0/2811162953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9c8074fe0 0x7fd9c810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.392+0000 7fd9cd31b700 1 -- 172.21.15.40:0/2811162953 >> 172.21.15.40:0/2811162953 conn(0x7fd9c8079520 msgr2=0x7fd9c8079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.392+0000 7fd9cd31b700 1 -- 172.21.15.40:0/2811162953 shutdown_connections 2024-06-06T03:37:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.392+0000 7fd9cd31b700 1 -- 172.21.15.40:0/2811162953 wait complete. 2024-06-06T03:37:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.393+0000 7fd9cd31b700 1 Processor -- start 2024-06-06T03:37:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.393+0000 7fd9cd31b700 1 -- start start 2024-06-06T03:37:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.393+0000 7fd9cd31b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9c80746c0 0x7fd9c819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.393+0000 7fd9cd31b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c8074fe0 0x7fd9c819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.393+0000 7fd9cd31b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9c810eef0 0x7fd9c819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.393+0000 7fd9cd31b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81149a0 con 0x7fd9c8074fe0 2024-06-06T03:37:38.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.393+0000 7fd9cd31b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8114820 con 0x7fd9c810eef0 2024-06-06T03:37:38.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.393+0000 7fd9cd31b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8114b20 con 0x7fd9c80746c0 2024-06-06T03:37:38.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.393+0000 7fd9c67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c8074fe0 0x7fd9c819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.393+0000 7fd9c67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c8074fe0 0x7fd9c819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40500/0 (socket says 172.21.15.40:40500) 2024-06-06T03:37:38.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.393+0000 7fd9c67fc700 1 -- 172.21.15.40:0/667167584 learned_addr learned my addr 172.21.15.40:0/667167584 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:38.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.393+0000 7fd9c6ffd700 1 --2- 172.21.15.40:0/667167584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9c80746c0 0x7fd9c819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.393+0000 7fd9c77fe700 1 --2- 172.21.15.40:0/667167584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9c810eef0 0x7fd9c819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.394+0000 7fd9c67fc700 1 -- 172.21.15.40:0/667167584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9c80746c0 msgr2=0x7fd9c819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:38.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.394+0000 7fd9c67fc700 1 --2- 172.21.15.40:0/667167584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9c80746c0 0x7fd9c819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.394+0000 7fd9c67fc700 1 -- 172.21.15.40:0/667167584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9c810eef0 msgr2=0x7fd9c819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:38.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.394+0000 7fd9c67fc700 1 --2- 172.21.15.40:0/667167584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9c810eef0 0x7fd9c819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.394+0000 7fd9c67fc700 1 -- 172.21.15.40:0/667167584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c819fe00 con 0x7fd9c8074fe0 2024-06-06T03:37:38.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.394+0000 7fd9c67fc700 1 --2- 172.21.15.40:0/667167584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c8074fe0 0x7fd9c819b330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c920 tx=0x7fd9b000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:38.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.394+0000 7fd9bffff700 1 -- 172.21.15.40:0/667167584 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0004a30 con 0x7fd9c8074fe0 2024-06-06T03:37:38.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.394+0000 7fd9cd31b700 1 -- 172.21.15.40:0/667167584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c81a00f0 con 0x7fd9c8074fe0 2024-06-06T03:37:38.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.394+0000 7fd9bffff700 1 -- 172.21.15.40:0/667167584 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b0005500 con 0x7fd9c8074fe0 2024-06-06T03:37:38.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.394+0000 7fd9bffff700 1 -- 172.21.15.40:0/667167584 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0007d00 con 0x7fd9c8074fe0 2024-06-06T03:37:38.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.394+0000 7fd9cd31b700 1 -- 172.21.15.40:0/667167584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c81a7970 con 0x7fd9c8074fe0 2024-06-06T03:37:38.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.397+0000 7fd9bffff700 1 -- 172.21.15.40:0/667167584 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd9b0005000 con 0x7fd9c8074fe0 2024-06-06T03:37:38.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.398+0000 7fd9bffff700 1 --2- 172.21.15.40:0/667167584 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd9b40a2c20 0x7fd9b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.398+0000 7fd9c6ffd700 1 --2- 172.21.15.40:0/667167584 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd9b40a2c20 0x7fd9b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.398+0000 7fd9bffff700 1 -- 172.21.15.40:0/667167584 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fd9b00ceaa0 con 0x7fd9c8074fe0 2024-06-06T03:37:38.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.398+0000 7fd9d7b9eb80 1 Processor -- start 2024-06-06T03:37:38.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.398+0000 7fd9c6ffd700 1 --2- 172.21.15.40:0/667167584 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd9b40a2c20 0x7fd9b40a50e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009550 tx=0x7fd9b80091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:38.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9d7b9eb80 1 -- start start 2024-06-06T03:37:38.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9d7b9eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefbf10 0x5649aaefc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9d7b9eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefc830 0x5649aaef6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9d7b9eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaef6750 0x5649aaef6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9d7b9eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5649aade8f10 con 0x5649aaefbf10 2024-06-06T03:37:38.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9d7b9eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5649aae47800 con 0x5649aaefc830 2024-06-06T03:37:38.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9d7b9eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5649aadaf410 con 0x5649aaef6750 2024-06-06T03:37:38.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9c6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefbf10 0x5649aaefc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9c6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefbf10 0x5649aaefc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40504/0 (socket says 172.21.15.40:40504) 2024-06-06T03:37:38.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9c6ffd700 1 -- 172.21.15.40:0/2198337345 learned_addr learned my addr 172.21.15.40:0/2198337345 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:38.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9c77fe700 1 --2- 172.21.15.40:0/2198337345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaef6750 0x5649aaef6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9c67fc700 1 --2- 172.21.15.40:0/2198337345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefc830 0x5649aaef6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9c6ffd700 1 -- 172.21.15.40:0/2198337345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaef6750 msgr2=0x5649aaef6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9c6ffd700 1 --2- 172.21.15.40:0/2198337345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaef6750 0x5649aaef6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9c6ffd700 1 -- 172.21.15.40:0/2198337345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefc830 msgr2=0x5649aaef6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9c6ffd700 1 --2- 172.21.15.40:0/2198337345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefc830 0x5649aaef6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.399+0000 7fd9c6ffd700 1 -- 172.21.15.40:0/2198337345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649aad96880 con 0x5649aaefbf10 2024-06-06T03:37:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.400+0000 7fd9c6ffd700 1 --2- 172.21.15.40:0/2198337345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefbf10 0x5649aaefc2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80046e0 tx=0x7fd9b8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a19e7af4d0e30d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.400+0000 7fd9bd7fa700 1 -- 172.21.15.40:0/2198337345 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b801c050 con 0x5649aaefbf10 2024-06-06T03:37:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.400+0000 7fd9bd7fa700 1 -- 172.21.15.40:0/2198337345 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b8026070 con 0x5649aaefbf10 2024-06-06T03:37:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.400+0000 7fd9bd7fa700 1 -- 172.21.15.40:0/2198337345 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8015680 con 0x5649aaefbf10 2024-06-06T03:37:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.400+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2198337345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefbf10 msgr2=0x5649aaefc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.400+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2198337345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefbf10 0x5649aaefc2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80046e0 tx=0x7fd9b8013000 comp rx=0 tx=0).stop 2024-06-06T03:37:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.401+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2198337345 shutdown_connections 2024-06-06T03:37:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.401+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2198337345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaef6750 0x5649aaef6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.401+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2198337345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefbf10 0x5649aaefc2f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.401+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2198337345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefc830 0x5649aaef6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.401+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2198337345 >> 172.21.15.40:0/2198337345 conn(0x5649aae23030 msgr2=0x5649aaef11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.401+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2198337345 shutdown_connections 2024-06-06T03:37:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.401+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2198337345 wait complete. 2024-06-06T03:37:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.401+0000 7fd9d7b9eb80 1 -- start start 2024-06-06T03:37:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.401+0000 7fd9d7b9eb80 1 Processor -- start 2024-06-06T03:37:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.401+0000 7fd9d7b9eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaef6750 0x5649aaf02550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.401+0000 7fd9d7b9eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaefbf10 0x5649aaeff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.401+0000 7fd9d7b9eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefc830 0x5649aaeffd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9d7b9eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5649aade7f70 con 0x5649aaef6750 2024-06-06T03:37:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9d7b9eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5649aae437c0 con 0x5649aaefc830 2024-06-06T03:37:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9d7b9eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5649aaf03210 con 0x5649aaefbf10 2024-06-06T03:37:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9c6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaef6750 0x5649aaf02550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9d7b9eb80 10 client.0 _ll_register_callbacks cb 0x5649aaddd560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:37:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9c6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaef6750 0x5649aaf02550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40508/0 (socket says 172.21.15.40:40508) 2024-06-06T03:37:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9c6ffd700 1 -- 172.21.15.40:0/2852861138 learned_addr learned my addr 172.21.15.40:0/2852861138 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9c77fe700 1 --2- 172.21.15.40:0/2852861138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefc830 0x5649aaeffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9c6ffd700 1 -- 172.21.15.40:0/2852861138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaefbf10 msgr2=0x5649aaeff850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9c6ffd700 1 --2- 172.21.15.40:0/2852861138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaefbf10 0x5649aaeff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9c6ffd700 1 -- 172.21.15.40:0/2852861138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefc830 msgr2=0x5649aaeffd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9c6ffd700 1 --2- 172.21.15.40:0/2852861138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefc830 0x5649aaeffd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9c6ffd700 1 -- 172.21.15.40:0/2852861138 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649aae0d700 con 0x5649aaef6750 2024-06-06T03:37:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9c77fe700 1 --2- 172.21.15.40:0/2852861138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefc830 0x5649aaeffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:37:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9c6ffd700 1 --2- 172.21.15.40:0/2852861138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaef6750 0x5649aaf02550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009610 tx=0x7fd9b8004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9554a22d7c0aa2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9ae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:37:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9ae7fc700 1 -- 172.21.15.40:0/2852861138 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8021910 con 0x5649aaef6750 2024-06-06T03:37:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2852861138 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9b8026070 con 0x5649aaef6750 2024-06-06T03:37:38.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9ae7fc700 1 -- 172.21.15.40:0/2852861138 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fd9b8021ab0 con 0x5649aaef6750 2024-06-06T03:37:38.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9ae7fc700 1 -- 172.21.15.40:0/2852861138 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9b801c020 con 0x5649aaef6750 2024-06-06T03:37:38.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9d7b9eb80 10 client.14958 fetch_fsmap learned FSMap version 28 2024-06-06T03:37:38.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.402+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2852861138 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649aaf0d940 con 0x5649aaef6750 2024-06-06T03:37:38.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.403+0000 7fd9ae7fc700 1 -- 172.21.15.40:0/2852861138 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b8026070 con 0x5649aaef6750 2024-06-06T03:37:38.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.403+0000 7fd9ae7fc700 1 -- 172.21.15.40:0/2852861138 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd9b803e080 con 0x5649aaef6750 2024-06-06T03:37:38.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.403+0000 7fd9d7b9eb80 10 client.14958 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:37:38.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.403+0000 7fd9d7b9eb80 10 client.14958 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T03:37:38.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.403+0000 7fd9d7b9eb80 20 client.14958 populate_metadata read hostname 'smithi040' 2024-06-06T03:37:38.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.403+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2852861138 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x5649aaf0fbc0 0x5649aaf11030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.403+0000 7fd9d7b9eb80 4 client.14958 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.403+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2852861138 --> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649aae54d00 con 0x5649aaf0fbc0 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.403+0000 7fd9c67fc700 1 --2- 172.21.15.40:0/2852861138 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x5649aaf0fbc0 0x5649aaf11030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.404+0000 7fd9c67fc700 1 --2- 172.21.15.40:0/2852861138 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x5649aaf0fbc0 0x5649aaf11030 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=679b8b008b19222a server_cookie=a5c7d0622b671991 in_seq=0 out_seq=0 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.404+0000 7fd9ae7fc700 10 client.14958 ms_handle_connect on v2:172.21.15.131:6838/782268782 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.404+0000 7fd9ae7fc700 1 -- 172.21.15.40:0/2852861138 <== mds.3 v2:172.21.15.131:6838/782268782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5649aae54d00 con 0x5649aaf0fbc0 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.404+0000 7fd9ae7fc700 10 client.14958 handle_command_reply: tid=0 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.404+0000 7fd9d7b9eb80 1 client.14958 shutdown 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.405+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2852861138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaef6750 msgr2=0x5649aaf02550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.405+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2852861138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaef6750 0x5649aaf02550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009610 tx=0x7fd9b8004d70 comp rx=0 tx=0).stop 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.405+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2852861138 shutdown_connections 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.405+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2852861138 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x5649aaf0fbc0 0x5649aaf11030 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.405+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2852861138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefc830 0x5649aaeffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.405+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2852861138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaef6750 0x5649aaf02550 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.405+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2852861138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaefbf10 0x5649aaeff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.405+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2852861138 >> 172.21.15.40:0/2852861138 conn(0x5649aae23030 msgr2=0x5649aaef06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.405+0000 7fd9ae7fc700 0 client.14958 ms_handle_reset on v2:172.21.15.131:6838/782268782 2024-06-06T03:37:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.405+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2852861138 shutdown_connections 2024-06-06T03:37:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.405+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2852861138 wait complete. 2024-06-06T03:37:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.405+0000 7fd9d7b9eb80 20 client.14958 trim_cache size 0 max 16384 2024-06-06T03:37:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.426+0000 7fd9d7b9eb80 1 Processor -- start 2024-06-06T03:37:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.426+0000 7fd9d7b9eb80 1 -- start start 2024-06-06T03:37:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.426+0000 7fd9d7b9eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaef6750 0x5649aaef6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.426+0000 7fd9d7b9eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefbf10 0x5649aaef0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.426+0000 7fd9d7b9eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaefc830 0x5649aaef1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.426+0000 7fd9d7b9eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5649aaf0f370 con 0x5649aaefbf10 2024-06-06T03:37:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9d7b9eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5649aaf0fa40 con 0x5649aaef6750 2024-06-06T03:37:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9d7b9eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5649aaf10110 con 0x5649aaefc830 2024-06-06T03:37:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9c67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefbf10 0x5649aaef0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9c67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefbf10 0x5649aaef0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40510/0 (socket says 172.21.15.40:40510) 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9c67fc700 1 -- 172.21.15.40:0/2194941584 learned_addr learned my addr 172.21.15.40:0/2194941584 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9c6ffd700 1 --2- 172.21.15.40:0/2194941584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaef6750 0x5649aaef6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9c77fe700 1 --2- 172.21.15.40:0/2194941584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaefc830 0x5649aaef1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9c67fc700 1 -- 172.21.15.40:0/2194941584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaefc830 msgr2=0x5649aaef1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9c67fc700 1 --2- 172.21.15.40:0/2194941584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaefc830 0x5649aaef1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9c67fc700 1 -- 172.21.15.40:0/2194941584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaef6750 msgr2=0x5649aaef6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9c67fc700 1 --2- 172.21.15.40:0/2194941584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaef6750 0x5649aaef6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9c67fc700 1 -- 172.21.15.40:0/2194941584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649aaf02a40 con 0x5649aaefbf10 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9c77fe700 1 --2- 172.21.15.40:0/2194941584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaefc830 0x5649aaef1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9c67fc700 1 --2- 172.21.15.40:0/2194941584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefbf10 0x5649aaef0e30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd9b009b2b0 tx=0x7fd9b00cc490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263bedbcb7fad881 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9bcff9700 1 -- 172.21.15.40:0/2194941584 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b00b4070 con 0x5649aaefbf10 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9bcff9700 1 -- 172.21.15.40:0/2194941584 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b0084070 con 0x5649aaefbf10 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.427+0000 7fd9bcff9700 1 -- 172.21.15.40:0/2194941584 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b001d040 con 0x5649aaefbf10 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.428+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2194941584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefbf10 msgr2=0x5649aaef0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.428+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2194941584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefbf10 0x5649aaef0e30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd9b009b2b0 tx=0x7fd9b00cc490 comp rx=0 tx=0).stop 2024-06-06T03:37:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.428+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2194941584 shutdown_connections 2024-06-06T03:37:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.428+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2194941584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaefc830 0x5649aaef1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.428+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2194941584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaef6750 0x5649aaef6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.428+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2194941584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefbf10 0x5649aaef0e30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.428+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2194941584 >> 172.21.15.40:0/2194941584 conn(0x5649aae23030 msgr2=0x5649aaefe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.428+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2194941584 shutdown_connections 2024-06-06T03:37:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.428+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2194941584 wait complete. 2024-06-06T03:37:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9d7b9eb80 1 -- start start 2024-06-06T03:37:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9d7b9eb80 1 Processor -- start 2024-06-06T03:37:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9d7b9eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaef6750 0x5649aaef0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9d7b9eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefbf10 0x5649aaef0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9d7b9eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefc830 0x5649aaf194a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9d7b9eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5649aaf10eb0 con 0x5649aaefc830 2024-06-06T03:37:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9d7b9eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5649aaf107e0 con 0x5649aaefbf10 2024-06-06T03:37:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9d7b9eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5649aaf11580 con 0x5649aaef6750 2024-06-06T03:37:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9d7b9eb80 10 client.0 _ll_register_callbacks cb 0x5649aaddd560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:37:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9c77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefc830 0x5649aaf194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9c77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefc830 0x5649aaf194a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40514/0 (socket says 172.21.15.40:40514) 2024-06-06T03:37:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9c77fe700 1 -- 172.21.15.40:0/2575095063 learned_addr learned my addr 172.21.15.40:0/2575095063 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9c6ffd700 1 --2- 172.21.15.40:0/2575095063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaef6750 0x5649aaef0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9c67fc700 1 --2- 172.21.15.40:0/2575095063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefbf10 0x5649aaef0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9c77fe700 1 -- 172.21.15.40:0/2575095063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaef6750 msgr2=0x5649aaef0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9c77fe700 1 --2- 172.21.15.40:0/2575095063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaef6750 0x5649aaef0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9c77fe700 1 -- 172.21.15.40:0/2575095063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefbf10 msgr2=0x5649aaef0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9c77fe700 1 --2- 172.21.15.40:0/2575095063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefbf10 0x5649aaef0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.429+0000 7fd9c77fe700 1 -- 172.21.15.40:0/2575095063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649aaf19a90 con 0x5649aaefc830 2024-06-06T03:37:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9c77fe700 1 --2- 172.21.15.40:0/2575095063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefc830 0x5649aaf194a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009990 tx=0x7fd9c0003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425c4c67e9c85715 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9ae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:37:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9ae7fc700 1 -- 172.21.15.40:0/2575095063 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c00052b0 con 0x5649aaefc830 2024-06-06T03:37:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9ae7fc700 1 -- 172.21.15.40:0/2575095063 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fd9c0005450 con 0x5649aaefc830 2024-06-06T03:37:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2575095063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9b0084070 con 0x5649aaefc830 2024-06-06T03:37:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9ae7fc700 1 -- 172.21.15.40:0/2575095063 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9c0005650 con 0x5649aaefc830 2024-06-06T03:37:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9d7b9eb80 10 client.14964 fetch_fsmap learned FSMap version 28 2024-06-06T03:37:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2575095063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649aaf1ad00 con 0x5649aaefc830 2024-06-06T03:37:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9ae7fc700 1 -- 172.21.15.40:0/2575095063 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b0084070 con 0x5649aaefc830 2024-06-06T03:37:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9ae7fc700 1 -- 172.21.15.40:0/2575095063 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd9c0023890 con 0x5649aaefc830 2024-06-06T03:37:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9d7b9eb80 10 client.14964 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:37:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9d7b9eb80 10 client.14964 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T03:37:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9d7b9eb80 20 client.14964 populate_metadata read hostname 'smithi040' 2024-06-06T03:37:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2575095063 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x5649aaf7faf0 0x5649aaf7e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9d7b9eb80 4 client.14964 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:37:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.430+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2575095063 --> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] -- command(tid 0: {"prefix": "status"}) -- 0x5649aae4f280 con 0x5649aaf7faf0 2024-06-06T03:37:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9c6ffd700 1 --2- 172.21.15.40:0/2575095063 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x5649aaf7faf0 0x5649aaf7e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9c6ffd700 1 --2- 172.21.15.40:0/2575095063 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x5649aaf7faf0 0x5649aaf7e800 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab3da148895b754c server_cookie=3c91ee4b2f2045c1 in_seq=0 out_seq=0 2024-06-06T03:37:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9ae7fc700 10 client.14964 ms_handle_connect on v2:172.21.15.131:6838/782268782 2024-06-06T03:37:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9ae7fc700 1 -- 172.21.15.40:0/2575095063 <== mds.3 v2:172.21.15.131:6838/782268782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5649aae4f280 con 0x5649aaf7faf0 2024-06-06T03:37:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9ae7fc700 10 client.14964 handle_command_reply: tid=0 2024-06-06T03:37:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9d7b9eb80 1 client.14964 shutdown 2024-06-06T03:37:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2575095063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefc830 msgr2=0x5649aaf194a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2575095063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefc830 0x5649aaf194a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009990 tx=0x7fd9c0003ef0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2575095063 shutdown_connections 2024-06-06T03:37:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2575095063 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x5649aaf7faf0 0x5649aaf7e800 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2575095063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649aaef6750 0x5649aaef0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2575095063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5649aaefc830 0x5649aaf194a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9d7b9eb80 1 --2- 172.21.15.40:0/2575095063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5649aaefbf10 0x5649aaef0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9ae7fc700 0 client.14964 ms_handle_reset on v2:172.21.15.131:6838/782268782 2024-06-06T03:37:38.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2575095063 >> 172.21.15.40:0/2575095063 conn(0x5649aae23030 msgr2=0x5649aaefcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:38.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2575095063 shutdown_connections 2024-06-06T03:37:38.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9d7b9eb80 1 -- 172.21.15.40:0/2575095063 wait complete. 2024-06-06T03:37:38.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.431+0000 7fd9d7b9eb80 20 client.14964 trim_cache size 0 max 16384 2024-06-06T03:37:38.454 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:37:38.454 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:37:38.454 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 3, 2024-06-06T03:37:38.455 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24403, 2024-06-06T03:37:38.455 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:37:38.455 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:37:38.455 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:37:38.455 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 125.51081438200001, 2024-06-06T03:37:38.455 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 28, 2024-06-06T03:37:38.455 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 90, 2024-06-06T03:37:38.455 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 88, 2024-06-06T03:37:38.455 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 170.03194217800001 2024-06-06T03:37:38.455 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:37:38.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.432+0000 7fd9cd31b700 1 -- 172.21.15.40:0/667167584 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd9b40a2c20 msgr2=0x7fd9b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:38.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.432+0000 7fd9cd31b700 1 --2- 172.21.15.40:0/667167584 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd9b40a2c20 0x7fd9b40a50e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009550 tx=0x7fd9b80091b0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.432+0000 7fd9cd31b700 1 -- 172.21.15.40:0/667167584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c8074fe0 msgr2=0x7fd9c819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.432+0000 7fd9cd31b700 1 --2- 172.21.15.40:0/667167584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c8074fe0 0x7fd9c819b330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c920 tx=0x7fd9b000cc30 comp rx=0 tx=0).stop 2024-06-06T03:37:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.432+0000 7fd9cd31b700 1 -- 172.21.15.40:0/667167584 shutdown_connections 2024-06-06T03:37:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.432+0000 7fd9cd31b700 1 --2- 172.21.15.40:0/667167584 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fd9b40a2c20 0x7fd9b40a50e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.432+0000 7fd9cd31b700 1 --2- 172.21.15.40:0/667167584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9c80746c0 0x7fd9c819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.432+0000 7fd9cd31b700 1 --2- 172.21.15.40:0/667167584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9c810eef0 0x7fd9c819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.432+0000 7fd9cd31b700 1 --2- 172.21.15.40:0/667167584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c8074fe0 0x7fd9c819b330 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.432+0000 7fd9cd31b700 1 -- 172.21.15.40:0/667167584 >> 172.21.15.40:0/667167584 conn(0x7fd9c8079520 msgr2=0x7fd9c8100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.432+0000 7fd9cd31b700 1 -- 172.21.15.40:0/667167584 shutdown_connections 2024-06-06T03:37:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:38.432+0000 7fd9cd31b700 1 -- 172.21.15.40:0/667167584 wait complete. 2024-06-06T03:37:39.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:38 smithi131 ceph-mon[36687]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-06-06T03:37:39.062 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:38 smithi040 ceph-mon[32220]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-06-06T03:37:39.090 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-06-06T03:37:39.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:38 smithi104 ceph-mon[36569]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-06-06T03:37:39.340 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:40.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.979+0000 7fe42b696700 1 -- 172.21.15.40:0/3716039065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe424101d40 msgr2=0x7fe42410e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:40.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.979+0000 7fe42b696700 1 --2- 172.21.15.40:0/3716039065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe424101d40 0x7fe42410e230 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00b0a0 tx=0x7fe41c00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.979+0000 7fe42b696700 1 -- 172.21.15.40:0/3716039065 shutdown_connections 2024-06-06T03:37:40.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.979+0000 7fe42b696700 1 --2- 172.21.15.40:0/3716039065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4241073f0 0x7fe4241077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.979+0000 7fe42b696700 1 --2- 172.21.15.40:0/3716039065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe424101d40 0x7fe42410e230 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.979+0000 7fe42b696700 1 --2- 172.21.15.40:0/3716039065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe424101380 0x7fe424101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.979+0000 7fe42b696700 1 -- 172.21.15.40:0/3716039065 >> 172.21.15.40:0/3716039065 conn(0x7fe424078300 msgr2=0x7fe424078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.979+0000 7fe42b696700 1 -- 172.21.15.40:0/3716039065 shutdown_connections 2024-06-06T03:37:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.979+0000 7fe42b696700 1 -- 172.21.15.40:0/3716039065 wait complete. 2024-06-06T03:37:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.980+0000 7fe42b696700 1 Processor -- start 2024-06-06T03:37:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.980+0000 7fe42b696700 1 -- start start 2024-06-06T03:37:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.980+0000 7fe42b696700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe424101380 0x7fe424199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.980+0000 7fe42b696700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424101d40 0x7fe42419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.980+0000 7fe42b696700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4241073f0 0x7fe4241a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.980+0000 7fe42b696700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe424111760 con 0x7fe424101380 2024-06-06T03:37:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.980+0000 7fe42b696700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241115e0 con 0x7fe4241073f0 2024-06-06T03:37:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.980+0000 7fe42b696700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241118e0 con 0x7fe424101d40 2024-06-06T03:37:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.980+0000 7fe429432700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe424101380 0x7fe424199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.980+0000 7fe429432700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe424101380 0x7fe424199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55846/0 (socket says 172.21.15.40:55846) 2024-06-06T03:37:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.980+0000 7fe429432700 1 -- 172.21.15.40:0/2000147275 learned_addr learned my addr 172.21.15.40:0/2000147275 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.981+0000 7fe429c33700 1 --2- 172.21.15.40:0/2000147275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4241073f0 0x7fe4241a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.981+0000 7fe428c31700 1 --2- 172.21.15.40:0/2000147275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424101d40 0x7fe42419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.981+0000 7fe429432700 1 -- 172.21.15.40:0/2000147275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424101d40 msgr2=0x7fe42419a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.981+0000 7fe429432700 1 --2- 172.21.15.40:0/2000147275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424101d40 0x7fe42419a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.981+0000 7fe429432700 1 -- 172.21.15.40:0/2000147275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4241073f0 msgr2=0x7fe4241a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.981+0000 7fe429432700 1 --2- 172.21.15.40:0/2000147275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4241073f0 0x7fe4241a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.981+0000 7fe429432700 1 -- 172.21.15.40:0/2000147275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4241a5140 con 0x7fe424101380 2024-06-06T03:37:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.981+0000 7fe429432700 1 --2- 172.21.15.40:0/2000147275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe424101380 0x7fe424199b80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe41800b530 tx=0x7fe41800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.981+0000 7fe4167fc700 1 -- 172.21.15.40:0/2000147275 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418015020 con 0x7fe424101380 2024-06-06T03:37:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.981+0000 7fe42b696700 1 -- 172.21.15.40:0/2000147275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4241a5370 con 0x7fe424101380 2024-06-06T03:37:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.981+0000 7fe4167fc700 1 -- 172.21.15.40:0/2000147275 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe418011d10 con 0x7fe424101380 2024-06-06T03:37:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.981+0000 7fe4167fc700 1 -- 172.21.15.40:0/2000147275 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41800f2e0 con 0x7fe424101380 2024-06-06T03:37:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.981+0000 7fe42b696700 1 -- 172.21.15.40:0/2000147275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4241a58d0 con 0x7fe424101380 2024-06-06T03:37:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.983+0000 7fe4167fc700 1 -- 172.21.15.40:0/2000147275 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe418018020 con 0x7fe424101380 2024-06-06T03:37:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.984+0000 7fe4167fc700 1 --2- 172.21.15.40:0/2000147275 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe4100a2c20 0x7fe4100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.984+0000 7fe428c31700 1 --2- 172.21.15.40:0/2000147275 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe4100a2c20 0x7fe4100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.984+0000 7fe4167fc700 1 -- 172.21.15.40:0/2000147275 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fe4180ce630 con 0x7fe424101380 2024-06-06T03:37:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.985+0000 7fe428c31700 1 --2- 172.21.15.40:0/2000147275 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe4100a2c20 0x7fe4100a50e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe424078870 tx=0x7fe420009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.985+0000 7fe435f19b80 1 Processor -- start 2024-06-06T03:37:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.985+0000 7fe435f19b80 1 -- start start 2024-06-06T03:37:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.985+0000 7fe435f19b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344320 0x55a05a344700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.985+0000 7fe435f19b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a344c40 0x55a05a33e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.985+0000 7fe435f19b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a33eb60 0x55a05a33efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.985+0000 7fe435f19b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a05a22cf10 con 0x55a05a344c40 2024-06-06T03:37:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.985+0000 7fe435f19b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a05a28b800 con 0x55a05a344320 2024-06-06T03:37:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.985+0000 7fe435f19b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a05a1f3410 con 0x55a05a33eb60 2024-06-06T03:37:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.985+0000 7fe428c31700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a344c40 0x55a05a33e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.985+0000 7fe428c31700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a344c40 0x55a05a33e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55854/0 (socket says 172.21.15.40:55854) 2024-06-06T03:37:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.985+0000 7fe428c31700 1 -- 172.21.15.40:0/2150948504 learned_addr learned my addr 172.21.15.40:0/2150948504 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.986+0000 7fe429432700 1 --2- 172.21.15.40:0/2150948504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344320 0x55a05a344700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.986+0000 7fe428c31700 1 -- 172.21.15.40:0/2150948504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a33eb60 msgr2=0x55a05a33efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.986+0000 7fe429c33700 1 --2- 172.21.15.40:0/2150948504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a33eb60 0x55a05a33efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.986+0000 7fe428c31700 1 --2- 172.21.15.40:0/2150948504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a33eb60 0x55a05a33efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.986+0000 7fe428c31700 1 -- 172.21.15.40:0/2150948504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344320 msgr2=0x55a05a344700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.986+0000 7fe428c31700 1 --2- 172.21.15.40:0/2150948504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344320 0x55a05a344700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.986+0000 7fe428c31700 1 -- 172.21.15.40:0/2150948504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05a1da880 con 0x55a05a344c40 2024-06-06T03:37:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.986+0000 7fe428c31700 1 --2- 172.21.15.40:0/2150948504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a344c40 0x55a05a33e620 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe420009fd0 tx=0x7fe420011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5900fc233b8e3bf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.986+0000 7fe40f7fe700 1 -- 172.21.15.40:0/2150948504 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42001b040 con 0x55a05a344c40 2024-06-06T03:37:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.986+0000 7fe40f7fe700 1 -- 172.21.15.40:0/2150948504 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe420024070 con 0x55a05a344c40 2024-06-06T03:37:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.986+0000 7fe40f7fe700 1 -- 172.21.15.40:0/2150948504 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420013770 con 0x55a05a344c40 2024-06-06T03:37:40.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.987+0000 7fe435f19b80 1 -- 172.21.15.40:0/2150948504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a344c40 msgr2=0x55a05a33e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:40.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.987+0000 7fe435f19b80 1 --2- 172.21.15.40:0/2150948504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a344c40 0x55a05a33e620 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe420009fd0 tx=0x7fe420011000 comp rx=0 tx=0).stop 2024-06-06T03:37:40.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.987+0000 7fe435f19b80 1 -- 172.21.15.40:0/2150948504 shutdown_connections 2024-06-06T03:37:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.987+0000 7fe435f19b80 1 --2- 172.21.15.40:0/2150948504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a33eb60 0x55a05a33efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.987+0000 7fe435f19b80 1 --2- 172.21.15.40:0/2150948504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344320 0x55a05a344700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.987+0000 7fe435f19b80 1 --2- 172.21.15.40:0/2150948504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a344c40 0x55a05a33e620 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.987+0000 7fe435f19b80 1 -- 172.21.15.40:0/2150948504 >> 172.21.15.40:0/2150948504 conn(0x55a05a267030 msgr2=0x55a05a3351d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.987+0000 7fe435f19b80 1 -- 172.21.15.40:0/2150948504 shutdown_connections 2024-06-06T03:37:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.987+0000 7fe435f19b80 1 -- 172.21.15.40:0/2150948504 wait complete. 2024-06-06T03:37:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.987+0000 7fe435f19b80 1 -- start start 2024-06-06T03:37:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.987+0000 7fe435f19b80 1 Processor -- start 2024-06-06T03:37:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe435f19b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a33eb60 0x55a05a34aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe435f19b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344320 0x55a05a347d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe435f19b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344c40 0x55a05a348280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe435f19b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a05a22bf70 con 0x55a05a33eb60 2024-06-06T03:37:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe435f19b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a05a2877c0 con 0x55a05a344c40 2024-06-06T03:37:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe435f19b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a05a34b630 con 0x55a05a344320 2024-06-06T03:37:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe435f19b80 10 client.0 _ll_register_callbacks cb 0x55a05a221560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:37:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe429432700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a33eb60 0x55a05a34aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe429432700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a33eb60 0x55a05a34aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55864/0 (socket says 172.21.15.40:55864) 2024-06-06T03:37:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe429432700 1 -- 172.21.15.40:0/119754189 learned_addr learned my addr 172.21.15.40:0/119754189 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe429c33700 1 --2- 172.21.15.40:0/119754189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344c40 0x55a05a348280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe428c31700 1 --2- 172.21.15.40:0/119754189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344320 0x55a05a347d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe429432700 1 -- 172.21.15.40:0/119754189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344320 msgr2=0x55a05a347d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe429432700 1 --2- 172.21.15.40:0/119754189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344320 0x55a05a347d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe429432700 1 -- 172.21.15.40:0/119754189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344c40 msgr2=0x55a05a348280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe429432700 1 --2- 172.21.15.40:0/119754189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344c40 0x55a05a348280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.988+0000 7fe429432700 1 -- 172.21.15.40:0/119754189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a05a251700 con 0x55a05a33eb60 2024-06-06T03:37:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe429432700 1 --2- 172.21.15.40:0/119754189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a33eb60 0x55a05a34aa40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe41800bb90 tx=0x7fe41809ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c848eecd374b4c30 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe40cff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:37:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe40cff9700 1 -- 172.21.15.40:0/119754189 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41809af50 con 0x55a05a33eb60 2024-06-06T03:37:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe40cff9700 1 -- 172.21.15.40:0/119754189 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fe4180105a0 con 0x55a05a33eb60 2024-06-06T03:37:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe435f19b80 1 -- 172.21.15.40:0/119754189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe420024070 con 0x55a05a33eb60 2024-06-06T03:37:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe40cff9700 1 -- 172.21.15.40:0/119754189 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe41809f300 con 0x55a05a33eb60 2024-06-06T03:37:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe435f19b80 10 client.14976 fetch_fsmap learned FSMap version 28 2024-06-06T03:37:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe435f19b80 1 -- 172.21.15.40:0/119754189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a05a355e90 con 0x55a05a33eb60 2024-06-06T03:37:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe40cff9700 1 -- 172.21.15.40:0/119754189 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe420024070 con 0x55a05a33eb60 2024-06-06T03:37:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe40cff9700 1 -- 172.21.15.40:0/119754189 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe4180b9b90 con 0x55a05a33eb60 2024-06-06T03:37:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe435f19b80 10 client.14976 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:37:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe435f19b80 10 client.14976 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-06T03:37:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe435f19b80 20 client.14976 populate_metadata read hostname 'smithi040' 2024-06-06T03:37:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe435f19b80 1 --2- 172.21.15.40:0/119754189 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55a05a358450 0x55a05a3598c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe435f19b80 4 client.14976 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:37:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.989+0000 7fe435f19b80 1 -- 172.21.15.40:0/119754189 --> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a05a298d00 con 0x55a05a358450 2024-06-06T03:37:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.990+0000 7fe428c31700 1 --2- 172.21.15.40:0/119754189 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55a05a358450 0x55a05a3598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.990+0000 7fe40cff9700 10 client.14976 ms_handle_connect on v2:172.21.15.104:6834/75462915 2024-06-06T03:37:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.990+0000 7fe428c31700 1 --2- 172.21.15.40:0/119754189 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55a05a358450 0x55a05a3598c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b33c2bea99e07653 server_cookie=843f49269cbc2bd3 in_seq=0 out_seq=0 2024-06-06T03:37:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe40cff9700 1 -- 172.21.15.40:0/119754189 <== mds.2 v2:172.21.15.104:6834/75462915 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a05a298d00 con 0x55a05a358450 2024-06-06T03:37:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe40cff9700 10 client.14976 handle_command_reply: tid=0 2024-06-06T03:37:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe435f19b80 1 client.14976 shutdown 2024-06-06T03:37:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe435f19b80 1 -- 172.21.15.40:0/119754189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a33eb60 msgr2=0x55a05a34aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe435f19b80 1 --2- 172.21.15.40:0/119754189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a33eb60 0x55a05a34aa40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe41800bb90 tx=0x7fe41809ab10 comp rx=0 tx=0).stop 2024-06-06T03:37:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe435f19b80 1 -- 172.21.15.40:0/119754189 shutdown_connections 2024-06-06T03:37:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe435f19b80 1 --2- 172.21.15.40:0/119754189 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55a05a358450 0x55a05a3598c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe435f19b80 1 --2- 172.21.15.40:0/119754189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344c40 0x55a05a348280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe435f19b80 1 --2- 172.21.15.40:0/119754189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a33eb60 0x55a05a34aa40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe40cff9700 0 client.14976 ms_handle_reset on v2:172.21.15.104:6834/75462915 2024-06-06T03:37:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe435f19b80 1 --2- 172.21.15.40:0/119754189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344320 0x55a05a347d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:40.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe435f19b80 1 -- 172.21.15.40:0/119754189 >> 172.21.15.40:0/119754189 conn(0x55a05a267030 msgr2=0x55a05a3346b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:40.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe435f19b80 1 -- 172.21.15.40:0/119754189 shutdown_connections 2024-06-06T03:37:40.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe435f19b80 1 -- 172.21.15.40:0/119754189 wait complete. 2024-06-06T03:37:40.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:40.991+0000 7fe435f19b80 20 client.14976 trim_cache size 0 max 16384 2024-06-06T03:37:41.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.013+0000 7fe435f19b80 1 Processor -- start 2024-06-06T03:37:41.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.013+0000 7fe435f19b80 1 -- start start 2024-06-06T03:37:41.012 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:40 smithi040 ceph-mon[32220]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-06T03:37:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe435f19b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a33eb60 0x55a05a3482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe435f19b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a344320 0x55a05a345770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe435f19b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344c40 0x55a05a334e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe435f19b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a05a357c00 con 0x55a05a344320 2024-06-06T03:37:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe435f19b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a05a3582d0 con 0x55a05a33eb60 2024-06-06T03:37:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe435f19b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a05a3589a0 con 0x55a05a344c40 2024-06-06T03:37:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe428c31700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a344320 0x55a05a345770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe428c31700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a344320 0x55a05a345770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55868/0 (socket says 172.21.15.40:55868) 2024-06-06T03:37:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe428c31700 1 -- 172.21.15.40:0/2111671052 learned_addr learned my addr 172.21.15.40:0/2111671052 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe429c33700 1 --2- 172.21.15.40:0/2111671052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344c40 0x55a05a334e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe429432700 1 --2- 172.21.15.40:0/2111671052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a33eb60 0x55a05a3482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe428c31700 1 -- 172.21.15.40:0/2111671052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344c40 msgr2=0x55a05a334e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe428c31700 1 --2- 172.21.15.40:0/2111671052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344c40 0x55a05a334e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe428c31700 1 -- 172.21.15.40:0/2111671052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a33eb60 msgr2=0x55a05a3482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe428c31700 1 --2- 172.21.15.40:0/2111671052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a33eb60 0x55a05a3482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe428c31700 1 -- 172.21.15.40:0/2111671052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05a33f190 con 0x55a05a344320 2024-06-06T03:37:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe429c33700 1 --2- 172.21.15.40:0/2111671052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344c40 0x55a05a334e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:37:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.014+0000 7fe428c31700 1 --2- 172.21.15.40:0/2111671052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a344320 0x55a05a345770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe42000ecb0 tx=0x7fe420015e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9412e16877034ee server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.015+0000 7fe40effd700 1 -- 172.21.15.40:0/2111671052 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420036070 con 0x55a05a344320 2024-06-06T03:37:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.015+0000 7fe40effd700 1 -- 172.21.15.40:0/2111671052 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe420024040 con 0x55a05a344320 2024-06-06T03:37:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.015+0000 7fe40effd700 1 -- 172.21.15.40:0/2111671052 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42000ee60 con 0x55a05a344320 2024-06-06T03:37:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.015+0000 7fe435f19b80 1 -- 172.21.15.40:0/2111671052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a344320 msgr2=0x55a05a345770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.015+0000 7fe435f19b80 1 --2- 172.21.15.40:0/2111671052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a344320 0x55a05a345770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe42000ecb0 tx=0x7fe420015e60 comp rx=0 tx=0).stop 2024-06-06T03:37:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.015+0000 7fe435f19b80 1 -- 172.21.15.40:0/2111671052 shutdown_connections 2024-06-06T03:37:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.015+0000 7fe435f19b80 1 --2- 172.21.15.40:0/2111671052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344c40 0x55a05a334e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.015+0000 7fe435f19b80 1 --2- 172.21.15.40:0/2111671052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a33eb60 0x55a05a3482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.015+0000 7fe435f19b80 1 --2- 172.21.15.40:0/2111671052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a344320 0x55a05a345770 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.015+0000 7fe435f19b80 1 -- 172.21.15.40:0/2111671052 >> 172.21.15.40:0/2111671052 conn(0x55a05a267030 msgr2=0x55a05a345060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.015+0000 7fe435f19b80 1 -- 172.21.15.40:0/2111671052 shutdown_connections 2024-06-06T03:37:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.015+0000 7fe435f19b80 1 -- 172.21.15.40:0/2111671052 wait complete. 2024-06-06T03:37:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.016+0000 7fe435f19b80 1 -- start start 2024-06-06T03:37:41.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.016+0000 7fe435f19b80 1 Processor -- start 2024-06-06T03:37:41.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.016+0000 7fe435f19b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a33eb60 0x55a05a362f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:41.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.016+0000 7fe435f19b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344320 0x55a05a363460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.016+0000 7fe435f19b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344c40 0x55a05a35dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.016+0000 7fe435f19b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a05a359740 con 0x55a05a33eb60 2024-06-06T03:37:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.016+0000 7fe435f19b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a05a359070 con 0x55a05a344c40 2024-06-06T03:37:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.016+0000 7fe435f19b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a05a359e10 con 0x55a05a344320 2024-06-06T03:37:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.016+0000 7fe435f19b80 10 client.0 _ll_register_callbacks cb 0x55a05a221560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:37:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.016+0000 7fe429432700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a33eb60 0x55a05a362f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.016+0000 7fe429432700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a33eb60 0x55a05a362f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55872/0 (socket says 172.21.15.40:55872) 2024-06-06T03:37:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.016+0000 7fe429432700 1 -- 172.21.15.40:0/2373343356 learned_addr learned my addr 172.21.15.40:0/2373343356 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe429c33700 1 --2- 172.21.15.40:0/2373343356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344c40 0x55a05a35dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe428c31700 1 --2- 172.21.15.40:0/2373343356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344320 0x55a05a363460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe429432700 1 -- 172.21.15.40:0/2373343356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344320 msgr2=0x55a05a363460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe429432700 1 --2- 172.21.15.40:0/2373343356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344320 0x55a05a363460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe429432700 1 -- 172.21.15.40:0/2373343356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344c40 msgr2=0x55a05a35dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe429432700 1 --2- 172.21.15.40:0/2373343356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344c40 0x55a05a35dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe429432700 1 -- 172.21.15.40:0/2373343356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a05a35e3a0 con 0x55a05a33eb60 2024-06-06T03:37:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe428c31700 1 --2- 172.21.15.40:0/2373343356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344320 0x55a05a363460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:37:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe429432700 1 --2- 172.21.15.40:0/2373343356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a33eb60 0x55a05a362f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe418000c00 tx=0x7fe4180175a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1120160dde8fdd44 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe40cff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:37:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe40cff9700 1 -- 172.21.15.40:0/2373343356 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41809af50 con 0x55a05a33eb60 2024-06-06T03:37:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe40cff9700 1 -- 172.21.15.40:0/2373343356 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fe41809a540 con 0x55a05a33eb60 2024-06-06T03:37:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe435f19b80 1 -- 172.21.15.40:0/2373343356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe420024040 con 0x55a05a33eb60 2024-06-06T03:37:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe40cff9700 1 -- 172.21.15.40:0/2373343356 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe41809a8b0 con 0x55a05a33eb60 2024-06-06T03:37:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe435f19b80 10 client.14982 fetch_fsmap learned FSMap version 28 2024-06-06T03:37:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.017+0000 7fe435f19b80 1 -- 172.21.15.40:0/2373343356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a05a3c4660 con 0x55a05a33eb60 2024-06-06T03:37:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.018+0000 7fe40cff9700 1 -- 172.21.15.40:0/2373343356 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe420024040 con 0x55a05a33eb60 2024-06-06T03:37:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.018+0000 7fe40cff9700 1 -- 172.21.15.40:0/2373343356 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe4180ac020 con 0x55a05a33eb60 2024-06-06T03:37:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.018+0000 7fe435f19b80 10 client.14982 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:37:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.018+0000 7fe435f19b80 10 client.14982 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-06T03:37:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.018+0000 7fe435f19b80 20 client.14982 populate_metadata read hostname 'smithi040' 2024-06-06T03:37:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.018+0000 7fe435f19b80 1 --2- 172.21.15.40:0/2373343356 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55a05a3c92c0 0x55a05a3c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.018+0000 7fe435f19b80 4 client.14982 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:37:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.018+0000 7fe435f19b80 1 -- 172.21.15.40:0/2373343356 --> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] -- command(tid 0: {"prefix": "status"}) -- 0x55a05a293280 con 0x55a05a3c92c0 2024-06-06T03:37:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.018+0000 7fe428c31700 1 --2- 172.21.15.40:0/2373343356 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55a05a3c92c0 0x55a05a3c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe40cff9700 10 client.14982 ms_handle_connect on v2:172.21.15.104:6834/75462915 2024-06-06T03:37:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe428c31700 1 --2- 172.21.15.40:0/2373343356 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55a05a3c92c0 0x55a05a3c7fd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40f4bcd783622605 server_cookie=cfc078838b51861a in_seq=0 out_seq=0 2024-06-06T03:37:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe40cff9700 1 -- 172.21.15.40:0/2373343356 <== mds.2 v2:172.21.15.104:6834/75462915 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a05a293280 con 0x55a05a3c92c0 2024-06-06T03:37:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe40cff9700 10 client.14982 handle_command_reply: tid=0 2024-06-06T03:37:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe435f19b80 1 client.14982 shutdown 2024-06-06T03:37:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe435f19b80 1 -- 172.21.15.40:0/2373343356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a33eb60 msgr2=0x55a05a362f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe435f19b80 1 --2- 172.21.15.40:0/2373343356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a33eb60 0x55a05a362f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe418000c00 tx=0x7fe4180175a0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe435f19b80 1 -- 172.21.15.40:0/2373343356 shutdown_connections 2024-06-06T03:37:41.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe435f19b80 1 --2- 172.21.15.40:0/2373343356 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55a05a3c92c0 0x55a05a3c7fd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe435f19b80 1 --2- 172.21.15.40:0/2373343356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a05a344c40 0x55a05a35dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe40cff9700 0 client.14982 ms_handle_reset on v2:172.21.15.104:6834/75462915 2024-06-06T03:37:41.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe435f19b80 1 --2- 172.21.15.40:0/2373343356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a05a33eb60 0x55a05a362f20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe435f19b80 1 --2- 172.21.15.40:0/2373343356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a05a344320 0x55a05a363460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe435f19b80 1 -- 172.21.15.40:0/2373343356 >> 172.21.15.40:0/2373343356 conn(0x55a05a267030 msgr2=0x55a05a339200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:41.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe435f19b80 1 -- 172.21.15.40:0/2373343356 shutdown_connections 2024-06-06T03:37:41.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe435f19b80 1 -- 172.21.15.40:0/2373343356 wait complete. 2024-06-06T03:37:41.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.019+0000 7fe435f19b80 20 client.14982 trim_cache size 0 max 16384 2024-06-06T03:37:41.023 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:37:41.024 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:37:41.024 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 2, 2024-06-06T03:37:41.024 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24425, 2024-06-06T03:37:41.024 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:37:41.024 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:37:41.024 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:37:41.024 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 129.10601013499999, 2024-06-06T03:37:41.024 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 28, 2024-06-06T03:37:41.024 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 90, 2024-06-06T03:37:41.024 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 88, 2024-06-06T03:37:41.024 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 184.05010654599999 2024-06-06T03:37:41.024 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:37:41.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.020+0000 7fe42b696700 1 -- 172.21.15.40:0/2000147275 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe4100a2c20 msgr2=0x7fe4100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:41.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.020+0000 7fe42b696700 1 --2- 172.21.15.40:0/2000147275 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe4100a2c20 0x7fe4100a50e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe424078870 tx=0x7fe420009190 comp rx=0 tx=0).stop 2024-06-06T03:37:41.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.020+0000 7fe42b696700 1 -- 172.21.15.40:0/2000147275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe424101380 msgr2=0x7fe424199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:41.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.020+0000 7fe42b696700 1 --2- 172.21.15.40:0/2000147275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe424101380 0x7fe424199b80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe41800b530 tx=0x7fe41800b840 comp rx=0 tx=0).stop 2024-06-06T03:37:41.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.020+0000 7fe42b696700 1 -- 172.21.15.40:0/2000147275 shutdown_connections 2024-06-06T03:37:41.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.020+0000 7fe42b696700 1 --2- 172.21.15.40:0/2000147275 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe4100a2c20 0x7fe4100a50e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.020+0000 7fe42b696700 1 --2- 172.21.15.40:0/2000147275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4241073f0 0x7fe4241a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.020+0000 7fe42b696700 1 --2- 172.21.15.40:0/2000147275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe424101380 0x7fe424199b80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.020+0000 7fe42b696700 1 --2- 172.21.15.40:0/2000147275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424101d40 0x7fe42419a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:41.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.020+0000 7fe42b696700 1 -- 172.21.15.40:0/2000147275 >> 172.21.15.40:0/2000147275 conn(0x7fe424078300 msgr2=0x7fe4240ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:41.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.020+0000 7fe42b696700 1 -- 172.21.15.40:0/2000147275 shutdown_connections 2024-06-06T03:37:41.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:41.020+0000 7fe42b696700 1 -- 172.21.15.40:0/2000147275 wait complete. 2024-06-06T03:37:41.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:40 smithi131 ceph-mon[36687]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-06T03:37:41.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:40 smithi104 ceph-mon[36569]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-06T03:37:41.697 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-06T03:37:41.948 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:43.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:42 smithi131 ceph-mon[36687]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-06-06T03:37:43.048 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:42 smithi040 ceph-mon[32220]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-06-06T03:37:43.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:42 smithi104 ceph-mon[36569]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-06-06T03:37:43.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.595+0000 7fa414470700 1 -- 172.21.15.40:0/3279365046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa40c107400 msgr2=0x7fa40c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:43.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.595+0000 7fa414470700 1 --2- 172.21.15.40:0/3279365046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa40c107400 0x7fa40c1077e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa408009990 tx=0x7fa408009ca0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.595+0000 7fa414470700 1 -- 172.21.15.40:0/3279365046 shutdown_connections 2024-06-06T03:37:43.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.595+0000 7fa414470700 1 --2- 172.21.15.40:0/3279365046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa40c101d50 0x7fa40c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.595+0000 7fa414470700 1 --2- 172.21.15.40:0/3279365046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa40c107400 0x7fa40c1077e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.595+0000 7fa414470700 1 --2- 172.21.15.40:0/3279365046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa40c101390 0x7fa40c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.595+0000 7fa414470700 1 -- 172.21.15.40:0/3279365046 >> 172.21.15.40:0/3279365046 conn(0x7fa40c078340 msgr2=0x7fa40c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:43.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.595+0000 7fa414470700 1 -- 172.21.15.40:0/3279365046 shutdown_connections 2024-06-06T03:37:43.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.595+0000 7fa414470700 1 -- 172.21.15.40:0/3279365046 wait complete. 2024-06-06T03:37:43.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.596+0000 7fa414470700 1 Processor -- start 2024-06-06T03:37:43.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.596+0000 7fa414470700 1 -- start start 2024-06-06T03:37:43.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.596+0000 7fa414470700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa40c101390 0x7fa40c19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.596+0000 7fa414470700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa40c101d50 0x7fa40c19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.596+0000 7fa414470700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa40c107400 0x7fa40c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.596+0000 7fa414470700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c111670 con 0x7fa40c107400 2024-06-06T03:37:43.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.596+0000 7fa414470700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1114f0 con 0x7fa40c101390 2024-06-06T03:37:43.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.596+0000 7fa414470700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1117f0 con 0x7fa40c101d50 2024-06-06T03:37:43.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.596+0000 7fa412a0d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa40c107400 0x7fa40c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.596+0000 7fa412a0d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa40c107400 0x7fa40c198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55892/0 (socket says 172.21.15.40:55892) 2024-06-06T03:37:43.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.596+0000 7fa412a0d700 1 -- 172.21.15.40:0/1335534553 learned_addr learned my addr 172.21.15.40:0/1335534553 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:43.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.597+0000 7fa41220c700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa40c101390 0x7fa40c19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.597+0000 7fa411a0b700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa40c101d50 0x7fa40c19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.597+0000 7fa412a0d700 1 -- 172.21.15.40:0/1335534553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa40c101d50 msgr2=0x7fa40c19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:43.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.597+0000 7fa412a0d700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa40c101d50 0x7fa40c19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.597+0000 7fa412a0d700 1 -- 172.21.15.40:0/1335534553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa40c101390 msgr2=0x7fa40c19e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:43.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.597+0000 7fa412a0d700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa40c101390 0x7fa40c19e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.597+0000 7fa412a0d700 1 -- 172.21.15.40:0/1335534553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c199080 con 0x7fa40c107400 2024-06-06T03:37:43.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.597+0000 7fa411a0b700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa40c101d50 0x7fa40c19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:37:43.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.597+0000 7fa412a0d700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa40c107400 0x7fa40c198920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa404007d70 tx=0x7fa40400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:43.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.597+0000 7fa4037fe700 1 -- 172.21.15.40:0/1335534553 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404016020 con 0x7fa40c107400 2024-06-06T03:37:43.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.597+0000 7fa414470700 1 -- 172.21.15.40:0/1335534553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c199370 con 0x7fa40c107400 2024-06-06T03:37:43.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.597+0000 7fa4037fe700 1 -- 172.21.15.40:0/1335534553 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa40400ad40 con 0x7fa40c107400 2024-06-06T03:37:43.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.597+0000 7fa4037fe700 1 -- 172.21.15.40:0/1335534553 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4040116f0 con 0x7fa40c107400 2024-06-06T03:37:43.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.597+0000 7fa414470700 1 -- 172.21.15.40:0/1335534553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c1a5630 con 0x7fa40c107400 2024-06-06T03:37:43.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.599+0000 7fa4037fe700 1 -- 172.21.15.40:0/1335534553 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa404019020 con 0x7fa40c107400 2024-06-06T03:37:43.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.600+0000 7fa4037fe700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa3f80a2c20 0x7fa3f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.601+0000 7fa41220c700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa3f80a2c20 0x7fa3f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.601+0000 7fa4037fe700 1 -- 172.21.15.40:0/1335534553 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fa4040cefb0 con 0x7fa40c107400 2024-06-06T03:37:43.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.601+0000 7fa41220c700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa3f80a2c20 0x7fa3f80a50e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa408006010 tx=0x7fa40800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:43.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.601+0000 7fa41ecf3b80 1 Processor -- start 2024-06-06T03:37:43.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa41ecf3b80 1 -- start start 2024-06-06T03:37:43.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa41ecf3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f73420 0x560229f73800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa41ecf3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6d620 0x560229f6daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa41ecf3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f6dfe0 0x560229f7a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa41ecf3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560229e5bf10 con 0x560229f6dfe0 2024-06-06T03:37:43.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa41ecf3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560229eba800 con 0x560229f6d620 2024-06-06T03:37:43.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa41ecf3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560229e22410 con 0x560229f73420 2024-06-06T03:37:43.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa412a0d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f6dfe0 0x560229f7a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa412a0d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f6dfe0 0x560229f7a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55908/0 (socket says 172.21.15.40:55908) 2024-06-06T03:37:43.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa412a0d700 1 -- 172.21.15.40:0/1486924779 learned_addr learned my addr 172.21.15.40:0/1486924779 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa411a0b700 1 --2- 172.21.15.40:0/1486924779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6d620 0x560229f6daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa41220c700 1 --2- 172.21.15.40:0/1486924779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f73420 0x560229f73800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa412a0d700 1 -- 172.21.15.40:0/1486924779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f73420 msgr2=0x560229f73800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa412a0d700 1 --2- 172.21.15.40:0/1486924779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f73420 0x560229f73800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa412a0d700 1 -- 172.21.15.40:0/1486924779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6d620 msgr2=0x560229f6daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa412a0d700 1 --2- 172.21.15.40:0/1486924779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6d620 0x560229f6daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa412a0d700 1 -- 172.21.15.40:0/1486924779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560229e09880 con 0x560229f6dfe0 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.602+0000 7fa412a0d700 1 --2- 172.21.15.40:0/1486924779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f6dfe0 0x560229f7a1c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa404098200 tx=0x7fa40409b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdd52edd96b8b1f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.603+0000 7fa400d70700 1 -- 172.21.15.40:0/1486924779 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40409bf30 con 0x560229f6dfe0 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.603+0000 7fa400d70700 1 -- 172.21.15.40:0/1486924779 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa40409b830 con 0x560229f6dfe0 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.603+0000 7fa400d70700 1 -- 172.21.15.40:0/1486924779 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404004bd0 con 0x560229f6dfe0 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.603+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/1486924779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f6dfe0 msgr2=0x560229f7a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.603+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/1486924779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f6dfe0 0x560229f7a1c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa404098200 tx=0x7fa40409b370 comp rx=0 tx=0).stop 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.603+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/1486924779 shutdown_connections 2024-06-06T03:37:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.603+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/1486924779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f73420 0x560229f73800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.603+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/1486924779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f6dfe0 0x560229f7a1c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.603+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/1486924779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6d620 0x560229f6daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.603+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/1486924779 >> 172.21.15.40:0/1486924779 conn(0x560229e96030 msgr2=0x560229f641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.604+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/1486924779 shutdown_connections 2024-06-06T03:37:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.604+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/1486924779 wait complete. 2024-06-06T03:37:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.604+0000 7fa41ecf3b80 1 -- start start 2024-06-06T03:37:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.604+0000 7fa41ecf3b80 1 Processor -- start 2024-06-06T03:37:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.604+0000 7fa41ecf3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6d620 0x560229f79e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.604+0000 7fa41ecf3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6dfe0 0x560229f77130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.604+0000 7fa41ecf3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f73420 0x560229f77670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.604+0000 7fa41ecf3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560229e5af70 con 0x560229f73420 2024-06-06T03:37:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.604+0000 7fa41ecf3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560229eb67c0 con 0x560229f6dfe0 2024-06-06T03:37:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.604+0000 7fa41ecf3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560229f7a700 con 0x560229f6d620 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.604+0000 7fa41ecf3b80 10 client.0 _ll_register_callbacks cb 0x560229e50560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.604+0000 7fa412a0d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f73420 0x560229f77670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.604+0000 7fa412a0d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f73420 0x560229f77670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55916/0 (socket says 172.21.15.40:55916) 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa412a0d700 1 -- 172.21.15.40:0/868474682 learned_addr learned my addr 172.21.15.40:0/868474682 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa41220c700 1 --2- 172.21.15.40:0/868474682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6d620 0x560229f79e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa411a0b700 1 --2- 172.21.15.40:0/868474682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6dfe0 0x560229f77130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa412a0d700 1 -- 172.21.15.40:0/868474682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6d620 msgr2=0x560229f79e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa412a0d700 1 --2- 172.21.15.40:0/868474682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6d620 0x560229f79e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa412a0d700 1 -- 172.21.15.40:0/868474682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6dfe0 msgr2=0x560229f77130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa412a0d700 1 --2- 172.21.15.40:0/868474682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6dfe0 0x560229f77130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa412a0d700 1 -- 172.21.15.40:0/868474682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560229e80700 con 0x560229f73420 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa411a0b700 1 --2- 172.21.15.40:0/868474682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6dfe0 0x560229f77130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa412a0d700 1 --2- 172.21.15.40:0/868474682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f73420 0x560229f77670 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa40409b4e0 tx=0x7fa40409fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a083b2cfca949051 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa3f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:37:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa3f5ffb700 1 -- 172.21.15.40:0/868474682 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4040b9060 con 0x560229f73420 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa3f5ffb700 1 -- 172.21.15.40:0/868474682 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fa404099b30 con 0x560229f73420 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/868474682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa40409b830 con 0x560229f73420 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa3f5ffb700 1 -- 172.21.15.40:0/868474682 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4040b3030 con 0x560229f73420 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa41ecf3b80 10 client.14994 fetch_fsmap learned FSMap version 28 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/868474682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560229f84d50 con 0x560229f73420 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.605+0000 7fa3f5ffb700 1 -- 172.21.15.40:0/868474682 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa40409b830 con 0x560229f73420 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.606+0000 7fa3f5ffb700 1 -- 172.21.15.40:0/868474682 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa404004c40 con 0x560229f73420 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.606+0000 7fa41ecf3b80 10 client.14994 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.606+0000 7fa41ecf3b80 10 client.14994 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.606+0000 7fa41ecf3b80 20 client.14994 populate_metadata read hostname 'smithi040' 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.606+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/868474682 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x560229f86e70 0x560229f882e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.606+0000 7fa41ecf3b80 4 client.14994 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.606+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/868474682 --> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560229ec7d00 con 0x560229f86e70 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.606+0000 7fa41220c700 1 --2- 172.21.15.40:0/868474682 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x560229f86e70 0x560229f882e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.607+0000 7fa41220c700 1 --2- 172.21.15.40:0/868474682 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x560229f86e70 0x560229f882e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=746d4e4216a624bf server_cookie=95ce1764438d95c2 in_seq=0 out_seq=0 2024-06-06T03:37:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.607+0000 7fa3f5ffb700 10 client.14994 ms_handle_connect on v2:172.21.15.104:6836/2498466212 2024-06-06T03:37:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.607+0000 7fa3f5ffb700 1 -- 172.21.15.40:0/868474682 <== mds.1 v2:172.21.15.104:6836/2498466212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560229ec7d00 con 0x560229f86e70 2024-06-06T03:37:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.607+0000 7fa3f5ffb700 10 client.14994 handle_command_reply: tid=0 2024-06-06T03:37:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.607+0000 7fa41ecf3b80 1 client.14994 shutdown 2024-06-06T03:37:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.608+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/868474682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f73420 msgr2=0x560229f77670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.608+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/868474682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f73420 0x560229f77670 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa40409b4e0 tx=0x7fa40409fcd0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.608+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/868474682 shutdown_connections 2024-06-06T03:37:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.608+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/868474682 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x560229f86e70 0x560229f882e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.608+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/868474682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6d620 0x560229f79e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.608+0000 7fa3f5ffb700 0 client.14994 ms_handle_reset on v2:172.21.15.104:6836/2498466212 2024-06-06T03:37:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.608+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/868474682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f73420 0x560229f77670 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.608+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/868474682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6dfe0 0x560229f77130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.608+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/868474682 >> 172.21.15.40:0/868474682 conn(0x560229e96030 msgr2=0x560229f636b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.608+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/868474682 shutdown_connections 2024-06-06T03:37:43.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.608+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/868474682 wait complete. 2024-06-06T03:37:43.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.608+0000 7fa41ecf3b80 20 client.14994 trim_cache size 0 max 16384 2024-06-06T03:37:43.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa41ecf3b80 1 Processor -- start 2024-06-06T03:37:43.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa41ecf3b80 1 -- start start 2024-06-06T03:37:43.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa41ecf3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6d620 0x560229f76130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa41ecf3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6dfe0 0x560229f63e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa41ecf3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f73420 0x560229f64370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa41ecf3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560229f86620 con 0x560229f73420 2024-06-06T03:37:43.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa41ecf3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560229f86cf0 con 0x560229f6d620 2024-06-06T03:37:43.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa41ecf3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560229f873c0 con 0x560229f6dfe0 2024-06-06T03:37:43.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa412a0d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f73420 0x560229f64370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa412a0d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f73420 0x560229f64370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55922/0 (socket says 172.21.15.40:55922) 2024-06-06T03:37:43.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa412a0d700 1 -- 172.21.15.40:0/2861231011 learned_addr learned my addr 172.21.15.40:0/2861231011 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:43.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa41220c700 1 --2- 172.21.15.40:0/2861231011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6d620 0x560229f76130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa411a0b700 1 --2- 172.21.15.40:0/2861231011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6dfe0 0x560229f63e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa412a0d700 1 -- 172.21.15.40:0/2861231011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6dfe0 msgr2=0x560229f63e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa412a0d700 1 --2- 172.21.15.40:0/2861231011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6dfe0 0x560229f63e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa412a0d700 1 -- 172.21.15.40:0/2861231011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6d620 msgr2=0x560229f76130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa412a0d700 1 --2- 172.21.15.40:0/2861231011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6d620 0x560229f76130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa412a0d700 1 -- 172.21.15.40:0/2861231011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560229f64df0 con 0x560229f73420 2024-06-06T03:37:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.630+0000 7fa412a0d700 1 --2- 172.21.15.40:0/2861231011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f73420 0x560229f64370 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa404097c90 tx=0x7fa4040a40e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6e35fa7d7e2c18 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.631+0000 7fa3f7fff700 1 -- 172.21.15.40:0/2861231011 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4040b9030 con 0x560229f73420 2024-06-06T03:37:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.631+0000 7fa3f7fff700 1 -- 172.21.15.40:0/2861231011 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa40409b830 con 0x560229f73420 2024-06-06T03:37:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.631+0000 7fa3f7fff700 1 -- 172.21.15.40:0/2861231011 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4040860b0 con 0x560229f73420 2024-06-06T03:37:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.631+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/2861231011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f73420 msgr2=0x560229f64370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.631+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/2861231011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f73420 0x560229f64370 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa404097c90 tx=0x7fa4040a40e0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.631+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/2861231011 shutdown_connections 2024-06-06T03:37:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.631+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/2861231011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f73420 0x560229f64370 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.631+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/2861231011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f6d620 0x560229f76130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.631+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/2861231011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6dfe0 0x560229f63e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.631+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/2861231011 >> 172.21.15.40:0/2861231011 conn(0x560229e96030 msgr2=0x560229f75a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/2861231011 shutdown_connections 2024-06-06T03:37:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/2861231011 wait complete. 2024-06-06T03:37:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa41ecf3b80 1 -- start start 2024-06-06T03:37:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa41ecf3b80 1 Processor -- start 2024-06-06T03:37:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa41ecf3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f6d620 0x560229f63760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa41ecf3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6dfe0 0x560229f63ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa41ecf3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f73420 0x560229f641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa41ecf3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560229f88160 con 0x560229f6d620 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa41ecf3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560229f87a90 con 0x560229f73420 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa41ecf3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560229f88830 con 0x560229f6dfe0 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa41ecf3b80 10 client.0 _ll_register_callbacks cb 0x560229e50560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa41220c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f6d620 0x560229f63760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa41220c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f6d620 0x560229f63760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55932/0 (socket says 172.21.15.40:55932) 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa41220c700 1 -- 172.21.15.40:0/1286616084 learned_addr learned my addr 172.21.15.40:0/1286616084 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.632+0000 7fa412a0d700 1 --2- 172.21.15.40:0/1286616084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f73420 0x560229f641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa411a0b700 1 --2- 172.21.15.40:0/1286616084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6dfe0 0x560229f63ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa41220c700 1 -- 172.21.15.40:0/1286616084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6dfe0 msgr2=0x560229f63ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa41220c700 1 --2- 172.21.15.40:0/1286616084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6dfe0 0x560229f63ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa41220c700 1 -- 172.21.15.40:0/1286616084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f73420 msgr2=0x560229f641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa41220c700 1 --2- 172.21.15.40:0/1286616084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f73420 0x560229f641e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa41220c700 1 -- 172.21.15.40:0/1286616084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560229f89240 con 0x560229f6d620 2024-06-06T03:37:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa411a0b700 1 --2- 172.21.15.40:0/1286616084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6dfe0 0x560229f63ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:37:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa41220c700 1 --2- 172.21.15.40:0/1286616084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f6d620 0x560229f63760 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa408009990 tx=0x7fa40801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e716327ca8440ca9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa3f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:37:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa3f5ffb700 1 -- 172.21.15.40:0/1286616084 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40800f870 con 0x560229f6d620 2024-06-06T03:37:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa3f5ffb700 1 -- 172.21.15.40:0/1286616084 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fa40800fa10 con 0x560229f6d620 2024-06-06T03:37:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/1286616084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa40409b830 con 0x560229f6d620 2024-06-06T03:37:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa3f5ffb700 1 -- 172.21.15.40:0/1286616084 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa408042030 con 0x560229f6d620 2024-06-06T03:37:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa41ecf3b80 10 client.15000 fetch_fsmap learned FSMap version 28 2024-06-06T03:37:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.633+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/1286616084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560229ff2990 con 0x560229f6d620 2024-06-06T03:37:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.634+0000 7fa3f5ffb700 1 -- 172.21.15.40:0/1286616084 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa40409b830 con 0x560229f6d620 2024-06-06T03:37:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.634+0000 7fa3f5ffb700 1 -- 172.21.15.40:0/1286616084 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa40803a070 con 0x560229f6d620 2024-06-06T03:37:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.634+0000 7fa41ecf3b80 10 client.15000 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:37:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.634+0000 7fa41ecf3b80 10 client.15000 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T03:37:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.634+0000 7fa41ecf3b80 20 client.15000 populate_metadata read hostname 'smithi040' 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.634+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/1286616084 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x560229f86e70 0x560229ff6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.634+0000 7fa41ecf3b80 4 client.15000 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.634+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/1286616084 --> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] -- command(tid 0: {"prefix": "status"}) -- 0x560229ec2280 con 0x560229f86e70 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.634+0000 7fa411a0b700 1 --2- 172.21.15.40:0/1286616084 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x560229f86e70 0x560229ff6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.634+0000 7fa411a0b700 1 --2- 172.21.15.40:0/1286616084 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x560229f86e70 0x560229ff6280 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b58a35dfb387fa28 server_cookie=9ff53fa23c76fc18 in_seq=0 out_seq=0 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.634+0000 7fa3f5ffb700 10 client.15000 ms_handle_connect on v2:172.21.15.104:6836/2498466212 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa3f5ffb700 1 -- 172.21.15.40:0/1286616084 <== mds.1 v2:172.21.15.104:6836/2498466212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560229ec2280 con 0x560229f86e70 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa3f5ffb700 10 client.15000 handle_command_reply: tid=0 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa41ecf3b80 1 client.15000 shutdown 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/1286616084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f6d620 msgr2=0x560229f63760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/1286616084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f6d620 0x560229f63760 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa408009990 tx=0x7fa40801a050 comp rx=0 tx=0).stop 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/1286616084 shutdown_connections 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/1286616084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560229f73420 0x560229f641e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/1286616084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560229f6d620 0x560229f63760 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/1286616084 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x560229f86e70 0x560229ff6280 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa41ecf3b80 1 --2- 172.21.15.40:0/1286616084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560229f6dfe0 0x560229f63ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa3f5ffb700 0 client.15000 ms_handle_reset on v2:172.21.15.104:6836/2498466212 2024-06-06T03:37:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/1286616084 >> 172.21.15.40:0/1286616084 conn(0x560229e96030 msgr2=0x560229f70e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/1286616084 shutdown_connections 2024-06-06T03:37:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa41ecf3b80 1 -- 172.21.15.40:0/1286616084 wait complete. 2024-06-06T03:37:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa41ecf3b80 20 client.15000 trim_cache size 0 max 16384 2024-06-06T03:37:43.639 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:37:43.639 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:37:43.639 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 1, 2024-06-06T03:37:43.639 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24443, 2024-06-06T03:37:43.640 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:37:43.640 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:37:43.640 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:37:43.640 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 132.73685885399999, 2024-06-06T03:37:43.640 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 28, 2024-06-06T03:37:43.640 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 90, 2024-06-06T03:37:43.640 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 88, 2024-06-06T03:37:43.640 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 178.18534055399999 2024-06-06T03:37:43.640 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:37:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa414470700 1 -- 172.21.15.40:0/1335534553 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa3f80a2c20 msgr2=0x7fa3f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa414470700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa3f80a2c20 0x7fa3f80a50e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa408006010 tx=0x7fa40800b540 comp rx=0 tx=0).stop 2024-06-06T03:37:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa414470700 1 -- 172.21.15.40:0/1335534553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa40c107400 msgr2=0x7fa40c198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.635+0000 7fa414470700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa40c107400 0x7fa40c198920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa404007d70 tx=0x7fa40400a340 comp rx=0 tx=0).stop 2024-06-06T03:37:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.636+0000 7fa414470700 1 -- 172.21.15.40:0/1335534553 shutdown_connections 2024-06-06T03:37:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.636+0000 7fa414470700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa3f80a2c20 0x7fa3f80a50e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.636+0000 7fa414470700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa40c107400 0x7fa40c198920 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.636+0000 7fa414470700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa40c101390 0x7fa40c19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.636+0000 7fa414470700 1 --2- 172.21.15.40:0/1335534553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa40c101d50 0x7fa40c19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.636+0000 7fa414470700 1 -- 172.21.15.40:0/1335534553 >> 172.21.15.40:0/1335534553 conn(0x7fa40c078340 msgr2=0x7fa40c100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.636+0000 7fa414470700 1 -- 172.21.15.40:0/1335534553 shutdown_connections 2024-06-06T03:37:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:43.636+0000 7fa414470700 1 -- 172.21.15.40:0/1335534553 wait complete. 2024-06-06T03:37:44.430 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T03:37:44.682 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:45.029 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:44 smithi040 ceph-mon[32220]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:37:45.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:44 smithi131 ceph-mon[36687]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:37:45.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:44 smithi104 ceph-mon[36569]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:37:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.247+0000 7f672fa29700 1 -- 172.21.15.40:0/2064629334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f672806ac40 msgr2=0x7f672806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.247+0000 7f672fa29700 1 --2- 172.21.15.40:0/2064629334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f672806ac40 0x7f672806b020 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6724009990 tx=0x7f6724009ca0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.247+0000 7f672fa29700 1 -- 172.21.15.40:0/2064629334 shutdown_connections 2024-06-06T03:37:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.247+0000 7f672fa29700 1 --2- 172.21.15.40:0/2064629334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f672810eee0 0x7f67281112d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.247+0000 7f672fa29700 1 --2- 172.21.15.40:0/2064629334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f672806ac40 0x7f672806b020 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.247+0000 7f672fa29700 1 --2- 172.21.15.40:0/2064629334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f672806b560 0x7f672810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.247+0000 7f672fa29700 1 -- 172.21.15.40:0/2064629334 >> 172.21.15.40:0/2064629334 conn(0x7f6728078430 msgr2=0x7f6728078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.247+0000 7f672fa29700 1 -- 172.21.15.40:0/2064629334 shutdown_connections 2024-06-06T03:37:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.247+0000 7f672fa29700 1 -- 172.21.15.40:0/2064629334 wait complete. 2024-06-06T03:37:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672fa29700 1 Processor -- start 2024-06-06T03:37:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672fa29700 1 -- start start 2024-06-06T03:37:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672fa29700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f672806ac40 0x7f672819afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672fa29700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f672806b560 0x7f672819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672fa29700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f672810eee0 0x7f672819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672fa29700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6728114ae0 con 0x7f672806b560 2024-06-06T03:37:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672fa29700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6728114960 con 0x7f672806ac40 2024-06-06T03:37:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672fa29700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6728114c60 con 0x7f672810eee0 2024-06-06T03:37:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672cfc4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f672806b560 0x7f672819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672cfc4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f672806b560 0x7f672819b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55942/0 (socket says 172.21.15.40:55942) 2024-06-06T03:37:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672cfc4700 1 -- 172.21.15.40:0/1329670822 learned_addr learned my addr 172.21.15.40:0/1329670822 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672d7c5700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f672806ac40 0x7f672819afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672dfc6700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f672810eee0 0x7f672819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672cfc4700 1 -- 172.21.15.40:0/1329670822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f672810eee0 msgr2=0x7f672819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672cfc4700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f672810eee0 0x7f672819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672cfc4700 1 -- 172.21.15.40:0/1329670822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f672806ac40 msgr2=0x7f672819afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672cfc4700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f672806ac40 0x7f672819afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672cfc4700 1 -- 172.21.15.40:0/1329670822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672819ffe0 con 0x7f672806b560 2024-06-06T03:37:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672dfc6700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f672810eee0 0x7f672819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:37:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.248+0000 7f672d7c5700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f672806ac40 0x7f672819afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:37:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.249+0000 7f672cfc4700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f672806b560 0x7f672819b510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f671800b530 tx=0x7f671800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.249+0000 7f671e7fc700 1 -- 172.21.15.40:0/1329670822 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718015020 con 0x7f672806b560 2024-06-06T03:37:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.249+0000 7f672fa29700 1 -- 172.21.15.40:0/1329670822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67281a02d0 con 0x7f672806b560 2024-06-06T03:37:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.249+0000 7f671e7fc700 1 -- 172.21.15.40:0/1329670822 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6718011d10 con 0x7f672806b560 2024-06-06T03:37:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.249+0000 7f671e7fc700 1 -- 172.21.15.40:0/1329670822 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718010e50 con 0x7f672806b560 2024-06-06T03:37:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.249+0000 7f672fa29700 1 -- 172.21.15.40:0/1329670822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67281a7b50 con 0x7f672806b560 2024-06-06T03:37:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.251+0000 7f671e7fc700 1 -- 172.21.15.40:0/1329670822 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6718018020 con 0x7f672806b560 2024-06-06T03:37:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.252+0000 7f671e7fc700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f67140a2dc0 0x7f67140a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.252+0000 7f672d7c5700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f67140a2dc0 0x7f67140a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.252+0000 7f671e7fc700 1 -- 172.21.15.40:0/1329670822 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f67180ce8b0 con 0x7f672806b560 2024-06-06T03:37:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.252+0000 7f672d7c5700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f67140a2dc0 0x7f67140a5280 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f672400b5c0 tx=0x7f6724005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f673a2acb80 1 Processor -- start 2024-06-06T03:37:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f673a2acb80 1 -- start start 2024-06-06T03:37:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f673a2acb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faff10 0x55db08fb02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f673a2acb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08fb0830 0x55db08faa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f673a2acb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08faab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f673a2acb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55db08e9cf10 con 0x55db08faff10 2024-06-06T03:37:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f673a2acb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55db08efb800 con 0x55db08faa750 2024-06-06T03:37:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f673a2acb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55db08e63410 con 0x55db08fb0830 2024-06-06T03:37:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f672d7c5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faff10 0x55db08fb02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f672d7c5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faff10 0x55db08fb02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55946/0 (socket says 172.21.15.40:55946) 2024-06-06T03:37:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f672d7c5700 1 -- 172.21.15.40:0/4237954846 learned_addr learned my addr 172.21.15.40:0/4237954846 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f672cfc4700 1 --2- 172.21.15.40:0/4237954846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08fb0830 0x55db08faa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f672dfc6700 1 --2- 172.21.15.40:0/4237954846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08faab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f672d7c5700 1 -- 172.21.15.40:0/4237954846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08fb0830 msgr2=0x55db08faa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f672d7c5700 1 --2- 172.21.15.40:0/4237954846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08fb0830 0x55db08faa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f672d7c5700 1 -- 172.21.15.40:0/4237954846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 msgr2=0x55db08faab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f672d7c5700 1 --2- 172.21.15.40:0/4237954846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08faab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f672d7c5700 1 -- 172.21.15.40:0/4237954846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db08e4a880 con 0x55db08faff10 2024-06-06T03:37:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.253+0000 7f672dfc6700 1 --2- 172.21.15.40:0/4237954846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08faab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:37:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.254+0000 7f672d7c5700 1 --2- 172.21.15.40:0/4237954846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faff10 0x55db08fb02f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f67240059d0 tx=0x7f672401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd4518fb91e610e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.254+0000 7f67137fe700 1 -- 172.21.15.40:0/4237954846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724003740 con 0x55db08faff10 2024-06-06T03:37:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.254+0000 7f67137fe700 1 -- 172.21.15.40:0/4237954846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6724003e40 con 0x55db08faff10 2024-06-06T03:37:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.254+0000 7f67137fe700 1 -- 172.21.15.40:0/4237954846 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672400f880 con 0x55db08faff10 2024-06-06T03:37:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.255+0000 7f673a2acb80 1 -- 172.21.15.40:0/4237954846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faff10 msgr2=0x55db08fb02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.255+0000 7f673a2acb80 1 --2- 172.21.15.40:0/4237954846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faff10 0x55db08fb02f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f67240059d0 tx=0x7f672401e000 comp rx=0 tx=0).stop 2024-06-06T03:37:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.255+0000 7f673a2acb80 1 -- 172.21.15.40:0/4237954846 shutdown_connections 2024-06-06T03:37:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.255+0000 7f673a2acb80 1 --2- 172.21.15.40:0/4237954846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08faab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.255+0000 7f673a2acb80 1 --2- 172.21.15.40:0/4237954846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faff10 0x55db08fb02f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.255+0000 7f673a2acb80 1 --2- 172.21.15.40:0/4237954846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08fb0830 0x55db08faa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.255+0000 7f673a2acb80 1 -- 172.21.15.40:0/4237954846 >> 172.21.15.40:0/4237954846 conn(0x55db08ed7030 msgr2=0x55db08fa51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.255+0000 7f673a2acb80 1 -- 172.21.15.40:0/4237954846 shutdown_connections 2024-06-06T03:37:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.255+0000 7f673a2acb80 1 -- 172.21.15.40:0/4237954846 wait complete. 2024-06-06T03:37:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.255+0000 7f673a2acb80 1 -- start start 2024-06-06T03:37:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.255+0000 7f673a2acb80 1 Processor -- start 2024-06-06T03:37:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.255+0000 7f673a2acb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08fb65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.255+0000 7f673a2acb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faff10 0x55db08fb38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f673a2acb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08fb0830 0x55db08fb3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f673a2acb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55db08e9bf70 con 0x55db08faff10 2024-06-06T03:37:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f673a2acb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55db08ef77c0 con 0x55db08faa750 2024-06-06T03:37:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f673a2acb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55db08fb71b0 con 0x55db08fb0830 2024-06-06T03:37:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f673a2acb80 10 client.0 _ll_register_callbacks cb 0x55db08e91560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:37:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f672cfc4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faff10 0x55db08fb38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f672cfc4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faff10 0x55db08fb38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55954/0 (socket says 172.21.15.40:55954) 2024-06-06T03:37:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f672cfc4700 1 -- 172.21.15.40:0/3897661023 learned_addr learned my addr 172.21.15.40:0/3897661023 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f672d7c5700 1 --2- 172.21.15.40:0/3897661023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08fb65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f672dfc6700 1 --2- 172.21.15.40:0/3897661023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08fb0830 0x55db08fb3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f672cfc4700 1 -- 172.21.15.40:0/3897661023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08fb0830 msgr2=0x55db08fb3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f672cfc4700 1 --2- 172.21.15.40:0/3897661023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08fb0830 0x55db08fb3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f672cfc4700 1 -- 172.21.15.40:0/3897661023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 msgr2=0x55db08fb65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f672cfc4700 1 --2- 172.21.15.40:0/3897661023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08fb65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f672cfc4700 1 -- 172.21.15.40:0/3897661023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db08ec1700 con 0x55db08faff10 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f672d7c5700 1 --2- 172.21.15.40:0/3897661023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08fb65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f672cfc4700 1 --2- 172.21.15.40:0/3897661023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faff10 0x55db08fb38e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f671809ae30 tx=0x7f6718004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15216144306c82a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f6710ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f6710ff9700 1 -- 172.21.15.40:0/3897661023 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67180104f0 con 0x55db08faff10 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f6710ff9700 1 -- 172.21.15.40:0/3897661023 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f671809f0d0 con 0x55db08faff10 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f673a2acb80 1 -- 172.21.15.40:0/3897661023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55db08f08d00 con 0x55db08faff10 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.256+0000 7f6710ff9700 1 -- 172.21.15.40:0/3897661023 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f671809f2d0 con 0x55db08faff10 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.257+0000 7f673a2acb80 10 client.15012 fetch_fsmap learned FSMap version 28 2024-06-06T03:37:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.257+0000 7f673a2acb80 1 -- 172.21.15.40:0/3897661023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db08fc1970 con 0x55db08faff10 2024-06-06T03:37:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.257+0000 7f6710ff9700 1 -- 172.21.15.40:0/3897661023 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55db08f08d00 con 0x55db08faff10 2024-06-06T03:37:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.257+0000 7f6710ff9700 1 -- 172.21.15.40:0/3897661023 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f671809a580 con 0x55db08faff10 2024-06-06T03:37:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.257+0000 7f673a2acb80 10 client.15012 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:37:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.257+0000 7f673a2acb80 10 client.15012 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T03:37:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.257+0000 7f673a2acb80 20 client.15012 populate_metadata read hostname 'smithi040' 2024-06-06T03:37:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.257+0000 7f673a2acb80 1 --2- 172.21.15.40:0/3897661023 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55db08fc3f70 0x55db08fc53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.257+0000 7f673a2acb80 4 client.15012 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:37:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.257+0000 7f673a2acb80 1 -- 172.21.15.40:0/3897661023 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db08f03280 con 0x55db08fc3f70 2024-06-06T03:37:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.257+0000 7f672d7c5700 1 --2- 172.21.15.40:0/3897661023 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55db08fc3f70 0x55db08fc53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.258+0000 7f6710ff9700 10 client.15012 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:37:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.258+0000 7f672d7c5700 1 --2- 172.21.15.40:0/3897661023 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55db08fc3f70 0x55db08fc53e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e52a76f8f97319 server_cookie=a96d98ae0ed1e747 in_seq=0 out_seq=0 2024-06-06T03:37:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f6710ff9700 1 -- 172.21.15.40:0/3897661023 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55db08f03280 con 0x55db08fc3f70 2024-06-06T03:37:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f6710ff9700 10 client.15012 handle_command_reply: tid=0 2024-06-06T03:37:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f673a2acb80 1 client.15012 shutdown 2024-06-06T03:37:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f673a2acb80 1 -- 172.21.15.40:0/3897661023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faff10 msgr2=0x55db08fb38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f673a2acb80 1 --2- 172.21.15.40:0/3897661023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faff10 0x55db08fb38e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f671809ae30 tx=0x7f6718004860 comp rx=0 tx=0).stop 2024-06-06T03:37:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f673a2acb80 1 -- 172.21.15.40:0/3897661023 shutdown_connections 2024-06-06T03:37:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f673a2acb80 1 --2- 172.21.15.40:0/3897661023 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55db08fc3f70 0x55db08fc53e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f673a2acb80 1 --2- 172.21.15.40:0/3897661023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08fb0830 0x55db08fb3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f673a2acb80 1 --2- 172.21.15.40:0/3897661023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08fb65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f673a2acb80 1 --2- 172.21.15.40:0/3897661023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faff10 0x55db08fb38e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f673a2acb80 1 -- 172.21.15.40:0/3897661023 >> 172.21.15.40:0/3897661023 conn(0x55db08ed7030 msgr2=0x55db08fa46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f6710ff9700 0 client.15012 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:37:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f673a2acb80 1 -- 172.21.15.40:0/3897661023 shutdown_connections 2024-06-06T03:37:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f673a2acb80 1 -- 172.21.15.40:0/3897661023 wait complete. 2024-06-06T03:37:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.259+0000 7f673a2acb80 20 client.15012 trim_cache size 0 max 16384 2024-06-06T03:37:46.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.281+0000 7f673a2acb80 1 Processor -- start 2024-06-06T03:37:46.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.281+0000 7f673a2acb80 1 -- start start 2024-06-06T03:37:46.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.281+0000 7f673a2acb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faa750 0x55db08fa4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.281+0000 7f673a2acb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08faff10 0x55db08fa5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.281+0000 7f673a2acb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08fb0830 0x55db08fada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.281+0000 7f673a2acb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55db08fc3720 con 0x55db08faa750 2024-06-06T03:37:46.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.281+0000 7f673a2acb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55db08fc3df0 con 0x55db08fb0830 2024-06-06T03:37:46.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.281+0000 7f673a2acb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55db08fc44c0 con 0x55db08faff10 2024-06-06T03:37:46.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.281+0000 7f672d7c5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faa750 0x55db08fa4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.281+0000 7f672d7c5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faa750 0x55db08fa4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55966/0 (socket says 172.21.15.40:55966) 2024-06-06T03:37:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.281+0000 7f672d7c5700 1 -- 172.21.15.40:0/2198323065 learned_addr learned my addr 172.21.15.40:0/2198323065 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.282+0000 7f672cfc4700 1 --2- 172.21.15.40:0/2198323065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08faff10 0x55db08fa5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.282+0000 7f672dfc6700 1 --2- 172.21.15.40:0/2198323065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08fb0830 0x55db08fada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.282+0000 7f672d7c5700 1 -- 172.21.15.40:0/2198323065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08faff10 msgr2=0x55db08fa5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.282+0000 7f672d7c5700 1 --2- 172.21.15.40:0/2198323065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08faff10 0x55db08fa5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.282+0000 7f672d7c5700 1 -- 172.21.15.40:0/2198323065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08fb0830 msgr2=0x55db08fada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.282+0000 7f672d7c5700 1 --2- 172.21.15.40:0/2198323065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08fb0830 0x55db08fada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.282+0000 7f672d7c5700 1 -- 172.21.15.40:0/2198323065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db08fb4820 con 0x55db08faa750 2024-06-06T03:37:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.282+0000 7f672d7c5700 1 --2- 172.21.15.40:0/2198323065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faa750 0x55db08fa4e30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f672400ba30 tx=0x7f6724021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34a627123e3b0e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.282+0000 7f6712ffd700 1 -- 172.21.15.40:0/2198323065 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724003960 con 0x55db08faa750 2024-06-06T03:37:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.282+0000 7f6712ffd700 1 -- 172.21.15.40:0/2198323065 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6724003b00 con 0x55db08faa750 2024-06-06T03:37:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.282+0000 7f6712ffd700 1 -- 172.21.15.40:0/2198323065 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724038b40 con 0x55db08faa750 2024-06-06T03:37:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.283+0000 7f673a2acb80 1 -- 172.21.15.40:0/2198323065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faa750 msgr2=0x55db08fa4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.283+0000 7f673a2acb80 1 --2- 172.21.15.40:0/2198323065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faa750 0x55db08fa4e30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f672400ba30 tx=0x7f6724021000 comp rx=0 tx=0).stop 2024-06-06T03:37:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.283+0000 7f673a2acb80 1 -- 172.21.15.40:0/2198323065 shutdown_connections 2024-06-06T03:37:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.283+0000 7f673a2acb80 1 --2- 172.21.15.40:0/2198323065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08fb0830 0x55db08fada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.283+0000 7f673a2acb80 1 --2- 172.21.15.40:0/2198323065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08faa750 0x55db08fa4e30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.283+0000 7f673a2acb80 1 --2- 172.21.15.40:0/2198323065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08faff10 0x55db08fa5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.283+0000 7f673a2acb80 1 -- 172.21.15.40:0/2198323065 >> 172.21.15.40:0/2198323065 conn(0x55db08ed7030 msgr2=0x55db08fa67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.283+0000 7f673a2acb80 1 -- 172.21.15.40:0/2198323065 shutdown_connections 2024-06-06T03:37:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.283+0000 7f673a2acb80 1 -- 172.21.15.40:0/2198323065 wait complete. 2024-06-06T03:37:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.284+0000 7f673a2acb80 1 -- start start 2024-06-06T03:37:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.284+0000 7f673a2acb80 1 Processor -- start 2024-06-06T03:37:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.284+0000 7f673a2acb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08fa3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.284+0000 7f673a2acb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08faff10 0x55db08fa4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.284+0000 7f673a2acb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08fb0830 0x55db08fa48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f673a2acb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55db08fc5260 con 0x55db08fb0830 2024-06-06T03:37:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f673a2acb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55db08fc4b90 con 0x55db08faa750 2024-06-06T03:37:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f673a2acb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55db08fc5930 con 0x55db08faff10 2024-06-06T03:37:46.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f672dfc6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08fb0830 0x55db08fa48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f672dfc6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08fb0830 0x55db08fa48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55974/0 (socket says 172.21.15.40:55974) 2024-06-06T03:37:46.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f672dfc6700 1 -- 172.21.15.40:0/3331352741 learned_addr learned my addr 172.21.15.40:0/3331352741 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:46.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f672d7c5700 1 --2- 172.21.15.40:0/3331352741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08fa3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f673a2acb80 10 client.0 _ll_register_callbacks cb 0x55db08e91560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:37:46.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f672cfc4700 1 --2- 172.21.15.40:0/3331352741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08faff10 0x55db08fa4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f672d7c5700 1 -- 172.21.15.40:0/3331352741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08faff10 msgr2=0x55db08fa4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:46.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f672d7c5700 1 --2- 172.21.15.40:0/3331352741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08faff10 0x55db08fa4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f672d7c5700 1 -- 172.21.15.40:0/3331352741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08fb0830 msgr2=0x55db08fa48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:37:46.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f672d7c5700 1 --2- 172.21.15.40:0/3331352741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08fb0830 0x55db08fa48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f672d7c5700 1 -- 172.21.15.40:0/3331352741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db08fcd850 con 0x55db08faa750 2024-06-06T03:37:46.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f672dfc6700 1 --2- 172.21.15.40:0/3331352741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08fb0830 0x55db08fa48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:37:46.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f672d7c5700 1 --2- 172.21.15.40:0/3331352741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08fa3e20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6724009990 tx=0x7f6724003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4efcfe58cb4a6859 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:46.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f6710ff9700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-06-06T03:37:46.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f6710ff9700 1 -- 172.21.15.40:0/3331352741 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67240050d0 con 0x55db08faa750 2024-06-06T03:37:46.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.285+0000 7f673a2acb80 1 -- 172.21.15.40:0/3331352741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6724003b00 con 0x55db08faa750 2024-06-06T03:37:46.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.286+0000 7f6710ff9700 1 -- 172.21.15.40:0/3331352741 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f6724005270 con 0x55db08faa750 2024-06-06T03:37:46.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.286+0000 7f6710ff9700 1 -- 172.21.15.40:0/3331352741 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f67240387c0 con 0x55db08faa750 2024-06-06T03:37:46.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.286+0000 7f673a2acb80 10 client.24764 fetch_fsmap learned FSMap version 28 2024-06-06T03:37:46.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.286+0000 7f673a2acb80 1 -- 172.21.15.40:0/3331352741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db08fceab0 con 0x55db08faa750 2024-06-06T03:37:46.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.286+0000 7f6710ff9700 1 -- 172.21.15.40:0/3331352741 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6724003b00 con 0x55db08faa750 2024-06-06T03:37:46.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.286+0000 7f6710ff9700 1 -- 172.21.15.40:0/3331352741 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f672400fa40 con 0x55db08faa750 2024-06-06T03:37:46.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.286+0000 7f673a2acb80 10 client.24764 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:37:46.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.286+0000 7f673a2acb80 10 client.24764 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T03:37:46.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.286+0000 7f673a2acb80 20 client.24764 populate_metadata read hostname 'smithi040' 2024-06-06T03:37:46.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.286+0000 7f673a2acb80 1 --2- 172.21.15.40:0/3331352741 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55db090330f0 0x55db09031e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:46.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.286+0000 7f673a2acb80 4 client.24764 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:37:46.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.286+0000 7f673a2acb80 1 -- 172.21.15.40:0/3331352741 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "status"}) -- 0x55db08eb6f00 con 0x55db090330f0 2024-06-06T03:37:46.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f672cfc4700 1 --2- 172.21.15.40:0/3331352741 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55db090330f0 0x55db09031e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f672cfc4700 1 --2- 172.21.15.40:0/3331352741 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55db090330f0 0x55db09031e00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47272edb1e4eebe3 server_cookie=e7c158dbee600374 in_seq=0 out_seq=0 2024-06-06T03:37:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f6710ff9700 10 client.24764 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:37:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f6710ff9700 1 -- 172.21.15.40:0/3331352741 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55db08eb6f00 con 0x55db090330f0 2024-06-06T03:37:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f6710ff9700 10 client.24764 handle_command_reply: tid=0 2024-06-06T03:37:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f673a2acb80 1 client.24764 shutdown 2024-06-06T03:37:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f673a2acb80 1 -- 172.21.15.40:0/3331352741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 msgr2=0x55db08fa3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f673a2acb80 1 --2- 172.21.15.40:0/3331352741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08fa3e20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6724009990 tx=0x7f6724003a10 comp rx=0 tx=0).stop 2024-06-06T03:37:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f673a2acb80 1 -- 172.21.15.40:0/3331352741 shutdown_connections 2024-06-06T03:37:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f673a2acb80 1 --2- 172.21.15.40:0/3331352741 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55db090330f0 0x55db09031e00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f673a2acb80 1 --2- 172.21.15.40:0/3331352741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db08fb0830 0x55db08fa48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f673a2acb80 1 --2- 172.21.15.40:0/3331352741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55db08faa750 0x55db08fa3e20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f6710ff9700 0 client.24764 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:37:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f673a2acb80 1 --2- 172.21.15.40:0/3331352741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db08faff10 0x55db08fa4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.287+0000 7f673a2acb80 1 -- 172.21.15.40:0/3331352741 >> 172.21.15.40:0/3331352741 conn(0x55db08ed7030 msgr2=0x55db08fada60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:46.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.288+0000 7f673a2acb80 1 -- 172.21.15.40:0/3331352741 shutdown_connections 2024-06-06T03:37:46.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.288+0000 7f673a2acb80 1 -- 172.21.15.40:0/3331352741 wait complete. 2024-06-06T03:37:46.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.288+0000 7f673a2acb80 20 client.24764 trim_cache size 0 max 16384 2024-06-06T03:37:46.292 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:37:46.292 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:37:46.292 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-06-06T03:37:46.292 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24470, 2024-06-06T03:37:46.292 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:37:46.292 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:37:46.292 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:37:46.292 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 149.427925606, 2024-06-06T03:37:46.293 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 28, 2024-06-06T03:37:46.293 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 90, 2024-06-06T03:37:46.293 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 88, 2024-06-06T03:37:46.293 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 166.73308682000001 2024-06-06T03:37:46.293 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:37:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.288+0000 7f672fa29700 1 -- 172.21.15.40:0/1329670822 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f67140a2dc0 msgr2=0x7f67140a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.288+0000 7f672fa29700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f67140a2dc0 0x7f67140a5280 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f672400b5c0 tx=0x7f6724005fb0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.288+0000 7f672fa29700 1 -- 172.21.15.40:0/1329670822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f672806b560 msgr2=0x7f672819b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.288+0000 7f672fa29700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f672806b560 0x7f672819b510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f671800b530 tx=0x7f671800b8f0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.288+0000 7f672fa29700 1 -- 172.21.15.40:0/1329670822 shutdown_connections 2024-06-06T03:37:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.288+0000 7f672fa29700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f67140a2dc0 0x7f67140a5280 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.288+0000 7f672fa29700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f672810eee0 0x7f672819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.288+0000 7f672fa29700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f672806ac40 0x7f672819afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.288+0000 7f672fa29700 1 --2- 172.21.15.40:0/1329670822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f672806b560 0x7f672819b510 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.288+0000 7f672fa29700 1 -- 172.21.15.40:0/1329670822 >> 172.21.15.40:0/1329670822 conn(0x7f6728078430 msgr2=0x7f67281103f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.289+0000 7f672fa29700 1 -- 172.21.15.40:0/1329670822 shutdown_connections 2024-06-06T03:37:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:46.289+0000 7f672fa29700 1 -- 172.21.15.40:0/1329670822 wait complete. 2024-06-06T03:37:47.031 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:46 smithi131 ceph-mon[36687]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:37:47.044 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:46 smithi040 ceph-mon[32220]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:37:47.080 INFO:tasks.cephadm:Setup complete, yielding 2024-06-06T03:37:47.080 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-06T03:37:47.089 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi040.front.sepia.ceph.com 2024-06-06T03:37:47.090 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-06T03:37:47.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:46 smithi104 ceph-mon[36569]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:37:47.354 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:48.814 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:48 smithi040 ceph-mon[32220]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:37:49.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:48 smithi131 ceph-mon[36687]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:37:49.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.042+0000 7fba77ed0700 1 -- 172.21.15.40:0/1886425942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70102f90 msgr2=0x7fba70109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:49.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.042+0000 7fba77ed0700 1 --2- 172.21.15.40:0/1886425942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70102f90 0x7fba70109840 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fba680099c0 tx=0x7fba68009cd0 comp rx=0 tx=0).stop 2024-06-06T03:37:49.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.042+0000 7fba77ed0700 1 -- 172.21.15.40:0/1886425942 shutdown_connections 2024-06-06T03:37:49.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.042+0000 7fba77ed0700 1 --2- 172.21.15.40:0/1886425942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70102f90 0x7fba70109840 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:49.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.042+0000 7fba77ed0700 1 --2- 172.21.15.40:0/1886425942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba701013f0 0x7fba70101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:49.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.042+0000 7fba77ed0700 1 --2- 172.21.15.40:0/1886425942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba701025f0 0x7fba70102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:49.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.042+0000 7fba77ed0700 1 -- 172.21.15.40:0/1886425942 >> 172.21.15.40:0/1886425942 conn(0x7fba700fc990 msgr2=0x7fba700fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:49.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.042+0000 7fba77ed0700 1 -- 172.21.15.40:0/1886425942 shutdown_connections 2024-06-06T03:37:49.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.042+0000 7fba77ed0700 1 -- 172.21.15.40:0/1886425942 wait complete. 2024-06-06T03:37:49.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.043+0000 7fba77ed0700 1 Processor -- start 2024-06-06T03:37:49.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.043+0000 7fba77ed0700 1 -- start start 2024-06-06T03:37:49.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.043+0000 7fba77ed0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba701013f0 0x7fba70198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:49.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.043+0000 7fba77ed0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba701025f0 0x7fba70198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:49.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.043+0000 7fba77ed0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70102f90 0x7fba7019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:49.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.043+0000 7fba77ed0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fba7010d0d0 con 0x7fba70102f90 2024-06-06T03:37:49.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.043+0000 7fba77ed0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fba7010cf50 con 0x7fba701025f0 2024-06-06T03:37:49.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.043+0000 7fba77ed0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fba7010d250 con 0x7fba701013f0 2024-06-06T03:37:49.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.043+0000 7fba7646d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70102f90 0x7fba7019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:49.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.043+0000 7fba75c6c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba701013f0 0x7fba70198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:49.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.043+0000 7fba7646d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70102f90 0x7fba7019fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55630/0 (socket says 172.21.15.40:55630) 2024-06-06T03:37:49.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.043+0000 7fba7646d700 1 -- 172.21.15.40:0/838206942 learned_addr learned my addr 172.21.15.40:0/838206942 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:49.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.043+0000 7fba7546b700 1 --2- 172.21.15.40:0/838206942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba701025f0 0x7fba70198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:49.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.044+0000 7fba7646d700 1 -- 172.21.15.40:0/838206942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba701013f0 msgr2=0x7fba70198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:49.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.044+0000 7fba7646d700 1 --2- 172.21.15.40:0/838206942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba701013f0 0x7fba70198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:49.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.044+0000 7fba7646d700 1 -- 172.21.15.40:0/838206942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba701025f0 msgr2=0x7fba70198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:49.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.044+0000 7fba7646d700 1 --2- 172.21.15.40:0/838206942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba701025f0 0x7fba70198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:49.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.044+0000 7fba7646d700 1 -- 172.21.15.40:0/838206942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba701a04f0 con 0x7fba70102f90 2024-06-06T03:37:49.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.044+0000 7fba75c6c700 1 --2- 172.21.15.40:0/838206942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba701013f0 0x7fba70198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:37:49.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.044+0000 7fba7646d700 1 --2- 172.21.15.40:0/838206942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70102f90 0x7fba7019fe50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fba68009990 tx=0x7fba68005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:49.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.044+0000 7fba66ffd700 1 -- 172.21.15.40:0/838206942 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68026070 con 0x7fba70102f90 2024-06-06T03:37:49.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.044+0000 7fba66ffd700 1 -- 172.21.15.40:0/838206942 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba68004150 con 0x7fba70102f90 2024-06-06T03:37:49.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.044+0000 7fba77ed0700 1 -- 172.21.15.40:0/838206942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba701a0780 con 0x7fba70102f90 2024-06-06T03:37:49.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.044+0000 7fba66ffd700 1 -- 172.21.15.40:0/838206942 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68004aa0 con 0x7fba70102f90 2024-06-06T03:37:49.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.044+0000 7fba77ed0700 1 -- 172.21.15.40:0/838206942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba701a0c50 con 0x7fba70102f90 2024-06-06T03:37:49.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.045+0000 7fba77ed0700 1 -- 172.21.15.40:0/838206942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba70069180 con 0x7fba70102f90 2024-06-06T03:37:49.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.046+0000 7fba66ffd700 1 -- 172.21.15.40:0/838206942 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fba68029070 con 0x7fba70102f90 2024-06-06T03:37:49.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.047+0000 7fba66ffd700 1 --2- 172.21.15.40:0/838206942 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fba5c0a2c20 0x7fba5c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:49.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.048+0000 7fba75c6c700 1 --2- 172.21.15.40:0/838206942 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fba5c0a2c20 0x7fba5c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:49.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.048+0000 7fba66ffd700 1 -- 172.21.15.40:0/838206942 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fba680cfae0 con 0x7fba70102f90 2024-06-06T03:37:49.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.048+0000 7fba75c6c700 1 --2- 172.21.15.40:0/838206942 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fba5c0a2c20 0x7fba5c0a50e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fba70102450 tx=0x7fba6c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:49.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.052+0000 7fba66ffd700 1 -- 172.21.15.40:0/838206942 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fba6805d850 con 0x7fba70102f90 2024-06-06T03:37:49.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:48 smithi104 ceph-mon[36569]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:37:49.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.281+0000 7fba77ed0700 1 -- 172.21.15.40:0/838206942 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fba700febb0 con 0x7fba5c0a2c20 2024-06-06T03:37:49.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.282+0000 7fba66ffd700 1 -- 172.21.15.40:0/838206942 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fba700febb0 con 0x7fba5c0a2c20 2024-06-06T03:37:49.280 INFO:teuthology.orchestra.run.smithi040.stdout:Backend: cephadm 2024-06-06T03:37:49.280 INFO:teuthology.orchestra.run.smithi040.stdout:Available: Yes 2024-06-06T03:37:49.281 INFO:teuthology.orchestra.run.smithi040.stdout:Paused: No 2024-06-06T03:37:49.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.284+0000 7fba77ed0700 1 -- 172.21.15.40:0/838206942 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fba5c0a2c20 msgr2=0x7fba5c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:49.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.284+0000 7fba77ed0700 1 --2- 172.21.15.40:0/838206942 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fba5c0a2c20 0x7fba5c0a50e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fba70102450 tx=0x7fba6c00d040 comp rx=0 tx=0).stop 2024-06-06T03:37:49.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.284+0000 7fba77ed0700 1 -- 172.21.15.40:0/838206942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70102f90 msgr2=0x7fba7019fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:49.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.284+0000 7fba77ed0700 1 --2- 172.21.15.40:0/838206942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70102f90 0x7fba7019fe50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fba68009990 tx=0x7fba68005ef0 comp rx=0 tx=0).stop 2024-06-06T03:37:49.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.284+0000 7fba77ed0700 1 -- 172.21.15.40:0/838206942 shutdown_connections 2024-06-06T03:37:49.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.284+0000 7fba77ed0700 1 --2- 172.21.15.40:0/838206942 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fba5c0a2c20 0x7fba5c0a50e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:49.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.284+0000 7fba77ed0700 1 --2- 172.21.15.40:0/838206942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba701013f0 0x7fba70198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:49.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.284+0000 7fba77ed0700 1 --2- 172.21.15.40:0/838206942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70102f90 0x7fba7019fe50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:49.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.284+0000 7fba77ed0700 1 --2- 172.21.15.40:0/838206942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba701025f0 0x7fba70198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:49.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.284+0000 7fba77ed0700 1 -- 172.21.15.40:0/838206942 >> 172.21.15.40:0/838206942 conn(0x7fba700fc990 msgr2=0x7fba700fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:49.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.284+0000 7fba77ed0700 1 -- 172.21.15.40:0/838206942 shutdown_connections 2024-06-06T03:37:49.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:49.284+0000 7fba77ed0700 1 -- 172.21.15.40:0/838206942 wait complete. 2024-06-06T03:37:49.953 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-06T03:37:49.979 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:49 smithi040 ceph-mon[32220]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:37:50.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:49 smithi104 ceph-mon[36569]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:37:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:50.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:49 smithi131 ceph-mon[36687]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:37:51.037 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:50 smithi040 ceph-mon[32220]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:37:51.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:50 smithi104 ceph-mon[36569]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:37:51.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:50 smithi131 ceph-mon[36687]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:37:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.018+0000 7fbd333e3700 1 -- 172.21.15.40:0/2980021868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd2c102fa0 msgr2=0x7fbd2c105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.018+0000 7fbd333e3700 1 --2- 172.21.15.40:0/2980021868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd2c102fa0 0x7fbd2c105390 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009990 tx=0x7fbd28009ca0 comp rx=0 tx=0).stop 2024-06-06T03:37:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.019+0000 7fbd333e3700 1 -- 172.21.15.40:0/2980021868 shutdown_connections 2024-06-06T03:37:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.019+0000 7fbd333e3700 1 --2- 172.21.15.40:0/2980021868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd2c108200 0x7fbd2c10a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.019+0000 7fbd333e3700 1 --2- 172.21.15.40:0/2980021868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd2c102fa0 0x7fbd2c105390 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.019+0000 7fbd333e3700 1 --2- 172.21.15.40:0/2980021868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd2c1058d0 0x7fbd2c107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.019+0000 7fbd333e3700 1 -- 172.21.15.40:0/2980021868 >> 172.21.15.40:0/2980021868 conn(0x7fbd2c0fc9b0 msgr2=0x7fbd2c0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.019+0000 7fbd333e3700 1 -- 172.21.15.40:0/2980021868 shutdown_connections 2024-06-06T03:37:52.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.019+0000 7fbd333e3700 1 -- 172.21.15.40:0/2980021868 wait complete. 2024-06-06T03:37:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.019+0000 7fbd333e3700 1 Processor -- start 2024-06-06T03:37:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.019+0000 7fbd333e3700 1 -- start start 2024-06-06T03:37:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.019+0000 7fbd333e3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd2c102fa0 0x7fbd2c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.019+0000 7fbd333e3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd2c1058d0 0x7fbd2c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd333e3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd2c108200 0x7fbd2c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd333e3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c10e0e0 con 0x7fbd2c1058d0 2024-06-06T03:37:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd333e3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c10df60 con 0x7fbd2c102fa0 2024-06-06T03:37:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd333e3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c10e260 con 0x7fbd2c108200 2024-06-06T03:37:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd3097e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd2c1058d0 0x7fbd2c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd3097e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd2c1058d0 0x7fbd2c198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55646/0 (socket says 172.21.15.40:55646) 2024-06-06T03:37:52.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd3097e700 1 -- 172.21.15.40:0/873437941 learned_addr learned my addr 172.21.15.40:0/873437941 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:52.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd31980700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd2c108200 0x7fbd2c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:52.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd3117f700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd2c102fa0 0x7fbd2c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:52.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd3097e700 1 -- 172.21.15.40:0/873437941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd2c108200 msgr2=0x7fbd2c19fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:52.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd3097e700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd2c108200 0x7fbd2c19fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:52.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd3097e700 1 -- 172.21.15.40:0/873437941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd2c102fa0 msgr2=0x7fbd2c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:52.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd3097e700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd2c102fa0 0x7fbd2c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:52.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd3097e700 1 -- 172.21.15.40:0/873437941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2c1a02c0 con 0x7fbd2c1058d0 2024-06-06T03:37:52.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd3117f700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd2c102fa0 0x7fbd2c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:37:52.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd3097e700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd2c1058d0 0x7fbd2c198ba0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00c950 tx=0x7fbd1c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:52.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd227fc700 1 -- 172.21.15.40:0/873437941 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c007ac0 con 0x7fbd2c1058d0 2024-06-06T03:37:52.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd227fc700 1 -- 172.21.15.40:0/873437941 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd1c004d10 con 0x7fbd2c1058d0 2024-06-06T03:37:52.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd333e3700 1 -- 172.21.15.40:0/873437941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2c1a05b0 con 0x7fbd2c1058d0 2024-06-06T03:37:52.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.020+0000 7fbd227fc700 1 -- 172.21.15.40:0/873437941 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c00f450 con 0x7fbd2c1058d0 2024-06-06T03:37:52.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.021+0000 7fbd333e3700 1 -- 172.21.15.40:0/873437941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2c1a0af0 con 0x7fbd2c1058d0 2024-06-06T03:37:52.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.021+0000 7fbd333e3700 1 -- 172.21.15.40:0/873437941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd2c069180 con 0x7fbd2c1058d0 2024-06-06T03:37:52.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.023+0000 7fbd227fc700 1 -- 172.21.15.40:0/873437941 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbd1c004750 con 0x7fbd2c1058d0 2024-06-06T03:37:52.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.023+0000 7fbd227fc700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbd180a2c20 0x7fbd180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:52.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.024+0000 7fbd3117f700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbd180a2c20 0x7fbd180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:52.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.024+0000 7fbd227fc700 1 -- 172.21.15.40:0/873437941 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fbd1c0cee70 con 0x7fbd2c1058d0 2024-06-06T03:37:52.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.024+0000 7fbd3117f700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbd180a2c20 0x7fbd180a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbd280039c0 tx=0x7fbd2800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:52.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.027+0000 7fbd227fc700 1 -- 172.21.15.40:0/873437941 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbd1c014030 con 0x7fbd2c1058d0 2024-06-06T03:37:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.244+0000 7fbd333e3700 1 -- 172.21.15.40:0/873437941 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbd2c0628d0 con 0x7fbd180a2c20 2024-06-06T03:37:52.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.257+0000 7fbd227fc700 1 -- 172.21.15.40:0/873437941 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fbd2c0628d0 con 0x7fbd180a2c20 2024-06-06T03:37:52.256 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-06T03:37:52.257 INFO:teuthology.orchestra.run.smithi040.stdout:mds.a smithi040 running (2m) 32s ago 2m 18.9M - 19.0.0-2946-g3d89accf b045ac545f82 8f73f8f00067 2024-06-06T03:37:52.257 INFO:teuthology.orchestra.run.smithi040.stdout:mds.b smithi104 running (2m) 34s ago 2m 28.5M - 19.0.0-2946-g3d89accf b045ac545f82 0226ae581893 2024-06-06T03:37:52.257 INFO:teuthology.orchestra.run.smithi040.stdout:mds.c smithi131 running (3m) 34s ago 3m 20.1M - 19.0.0-2946-g3d89accf b045ac545f82 0e5f414d00ff 2024-06-06T03:37:52.257 INFO:teuthology.orchestra.run.smithi040.stdout:mds.d smithi040 running (3m) 32s ago 3m 14.5M - 19.0.0-2946-g3d89accf b045ac545f82 e566d1e35dc8 2024-06-06T03:37:52.257 INFO:teuthology.orchestra.run.smithi040.stdout:mds.e smithi104 running (3m) 34s ago 3m 21.4M - 19.0.0-2946-g3d89accf b045ac545f82 4643e9f7b449 2024-06-06T03:37:52.257 INFO:teuthology.orchestra.run.smithi040.stdout:mds.f smithi131 running (3m) 34s ago 3m 21.7M - 19.0.0-2946-g3d89accf b045ac545f82 b3bcd351fd22 2024-06-06T03:37:52.257 INFO:teuthology.orchestra.run.smithi040.stdout:mds.g smithi040 running (2m) 32s ago 2m 18.2M - 19.0.0-2946-g3d89accf b045ac545f82 b0faccf813fd 2024-06-06T03:37:52.257 INFO:teuthology.orchestra.run.smithi040.stdout:mds.h smithi104 running (3m) 34s ago 3m 20.9M - 19.0.0-2946-g3d89accf b045ac545f82 969a1f2cb41a 2024-06-06T03:37:52.257 INFO:teuthology.orchestra.run.smithi040.stdout:mds.i smithi131 running (3m) 34s ago 3m 18.9M - 19.0.0-2946-g3d89accf b045ac545f82 4d9bd292e896 2024-06-06T03:37:52.257 INFO:teuthology.orchestra.run.smithi040.stdout:mds.j smithi040 running (3m) 32s ago 3m 13.6M - 19.0.0-2946-g3d89accf b045ac545f82 d0b64280c765 2024-06-06T03:37:52.257 INFO:teuthology.orchestra.run.smithi040.stdout:mds.k smithi104 running (3m) 34s ago 3m 19.0M - 19.0.0-2946-g3d89accf b045ac545f82 940392d80d72 2024-06-06T03:37:52.257 INFO:teuthology.orchestra.run.smithi040.stdout:mds.l smithi131 running (3m) 34s ago 3m 18.4M - 19.0.0-2946-g3d89accf b045ac545f82 220c34dc6a94 2024-06-06T03:37:52.257 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.x smithi040 *:9283,8765 running (10m) 32s ago 10m 517M - 19.0.0-2946-g3d89accf b045ac545f82 941bccf24d8a 2024-06-06T03:37:52.257 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.y smithi104 *:8443,8765 running (8m) 34s ago 8m 456M - 19.0.0-2946-g3d89accf b045ac545f82 87df9e888403 2024-06-06T03:37:52.258 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.z smithi131 *:8443,8765 running (8m) 34s ago 8m 454M - 19.0.0-2946-g3d89accf b045ac545f82 c1118b82cadb 2024-06-06T03:37:52.258 INFO:teuthology.orchestra.run.smithi040.stdout:mon.a smithi040 running (10m) 32s ago 10m 50.4M 2048M 19.0.0-2946-g3d89accf b045ac545f82 f7dcb07db357 2024-06-06T03:37:52.258 INFO:teuthology.orchestra.run.smithi040.stdout:mon.b smithi104 running (9m) 34s ago 9m 39.4M 2048M 19.0.0-2946-g3d89accf b045ac545f82 7589498360b1 2024-06-06T03:37:52.258 INFO:teuthology.orchestra.run.smithi040.stdout:mon.c smithi131 running (9m) 34s ago 9m 40.0M 2048M 19.0.0-2946-g3d89accf b045ac545f82 39bf71dc4919 2024-06-06T03:37:52.258 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (8m) 32s ago 8m 59.4M 4273M 19.0.0-2946-g3d89accf b045ac545f82 55d4eb08ab05 2024-06-06T03:37:52.258 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi104 running (8m) 34s ago 8m 56.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 18abb4ec4423 2024-06-06T03:37:52.258 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi131 running (7m) 34s ago 7m 56.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 a64c72ab59e7 2024-06-06T03:37:52.258 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi040 running (7m) 32s ago 7m 44.2M 4273M 19.0.0-2946-g3d89accf b045ac545f82 26474bff4d32 2024-06-06T03:37:52.258 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi104 running (6m) 34s ago 6m 42.9M 4272M 19.0.0-2946-g3d89accf b045ac545f82 0ccd66761365 2024-06-06T03:37:52.258 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi131 running (6m) 34s ago 6m 55.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 68048391c66c 2024-06-06T03:37:52.258 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6 smithi040 running (6m) 32s ago 6m 45.1M 4273M 19.0.0-2946-g3d89accf b045ac545f82 13a950e0cbf8 2024-06-06T03:37:52.258 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7 smithi104 running (5m) 34s ago 5m 57.4M 4272M 19.0.0-2946-g3d89accf b045ac545f82 ce22a21785d0 2024-06-06T03:37:52.258 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8 smithi131 running (5m) 34s ago 5m 57.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 9818cf9103de 2024-06-06T03:37:52.258 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9 smithi040 running (4m) 32s ago 4m 42.6M 4273M 19.0.0-2946-g3d89accf b045ac545f82 92eed7580928 2024-06-06T03:37:52.259 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10 smithi104 running (4m) 34s ago 4m 45.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 4d4ec4a286e8 2024-06-06T03:37:52.259 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11 smithi131 running (3m) 34s ago 3m 55.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 ba2d5cc80df1 2024-06-06T03:37:52.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.260+0000 7fbd333e3700 1 -- 172.21.15.40:0/873437941 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbd180a2c20 msgr2=0x7fbd180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:52.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.260+0000 7fbd333e3700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbd180a2c20 0x7fbd180a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbd280039c0 tx=0x7fbd2800b540 comp rx=0 tx=0).stop 2024-06-06T03:37:52.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.260+0000 7fbd333e3700 1 -- 172.21.15.40:0/873437941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd2c1058d0 msgr2=0x7fbd2c198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:52.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.260+0000 7fbd333e3700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd2c1058d0 0x7fbd2c198ba0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00c950 tx=0x7fbd1c00cc60 comp rx=0 tx=0).stop 2024-06-06T03:37:52.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.260+0000 7fbd333e3700 1 -- 172.21.15.40:0/873437941 shutdown_connections 2024-06-06T03:37:52.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.260+0000 7fbd333e3700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbd180a2c20 0x7fbd180a50e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:52.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.260+0000 7fbd333e3700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd2c108200 0x7fbd2c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:52.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.260+0000 7fbd333e3700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd2c102fa0 0x7fbd2c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:52.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.260+0000 7fbd333e3700 1 --2- 172.21.15.40:0/873437941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd2c1058d0 0x7fbd2c198ba0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:52.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.260+0000 7fbd333e3700 1 -- 172.21.15.40:0/873437941 >> 172.21.15.40:0/873437941 conn(0x7fbd2c0fc9b0 msgr2=0x7fbd2c1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:52.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.260+0000 7fbd333e3700 1 -- 172.21.15.40:0/873437941 shutdown_connections 2024-06-06T03:37:52.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:52.260+0000 7fbd333e3700 1 -- 172.21.15.40:0/873437941 wait complete. 2024-06-06T03:37:52.806 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:52 smithi040 ceph-mon[32220]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:37:52.806 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:52 smithi040 ceph-mon[32220]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:37:52.834 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-06T03:37:53.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:52 smithi104 ceph-mon[36569]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:37:53.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:52 smithi104 ceph-mon[36569]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:37:53.121 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:53.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:52 smithi131 ceph-mon[36687]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:37:53.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:52 smithi131 ceph-mon[36687]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:37:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.816+0000 7efc5af32700 1 -- 172.21.15.40:0/942482989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc541082a0 msgr2=0x7efc5410a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:54.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.816+0000 7efc5af32700 1 --2- 172.21.15.40:0/942482989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc541082a0 0x7efc5410a690 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00b0a0 tx=0x7efc4c00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:37:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.816+0000 7efc5af32700 1 -- 172.21.15.40:0/942482989 shutdown_connections 2024-06-06T03:37:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.816+0000 7efc5af32700 1 --2- 172.21.15.40:0/942482989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc541082a0 0x7efc5410a690 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.816+0000 7efc5af32700 1 --2- 172.21.15.40:0/942482989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc54103040 0x7efc54105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.816+0000 7efc5af32700 1 --2- 172.21.15.40:0/942482989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc54105970 0x7efc54107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.816+0000 7efc5af32700 1 -- 172.21.15.40:0/942482989 >> 172.21.15.40:0/942482989 conn(0x7efc540fc990 msgr2=0x7efc540fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:54.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.816+0000 7efc5af32700 1 -- 172.21.15.40:0/942482989 shutdown_connections 2024-06-06T03:37:54.817 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:54 smithi040 ceph-mon[32220]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:37:54.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.816+0000 7efc5af32700 1 -- 172.21.15.40:0/942482989 wait complete. 2024-06-06T03:37:54.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.817+0000 7efc5af32700 1 Processor -- start 2024-06-06T03:37:54.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.817+0000 7efc5af32700 1 -- start start 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.817+0000 7efc5af32700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc54103040 0x7efc5419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.817+0000 7efc5af32700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc54105970 0x7efc5419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.817+0000 7efc5af32700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc541082a0 0x7efc541a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.817+0000 7efc5af32700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efc5410e0b0 con 0x7efc541082a0 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.817+0000 7efc5af32700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efc5410df30 con 0x7efc54103040 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.817+0000 7efc5af32700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efc5410e230 con 0x7efc54105970 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.817+0000 7efc594cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc541082a0 0x7efc541a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.817+0000 7efc594cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc541082a0 0x7efc541a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55668/0 (socket says 172.21.15.40:55668) 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.817+0000 7efc58cce700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc54103040 0x7efc5419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.817+0000 7efc594cf700 1 -- 172.21.15.40:0/499511418 learned_addr learned my addr 172.21.15.40:0/499511418 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.817+0000 7efc53fff700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc54105970 0x7efc5419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.818+0000 7efc594cf700 1 -- 172.21.15.40:0/499511418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc54105970 msgr2=0x7efc5419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.818+0000 7efc594cf700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc54105970 0x7efc5419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.818+0000 7efc594cf700 1 -- 172.21.15.40:0/499511418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc54103040 msgr2=0x7efc5419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:54.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.818+0000 7efc594cf700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc54103040 0x7efc5419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.818+0000 7efc594cf700 1 -- 172.21.15.40:0/499511418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc541a24b0 con 0x7efc541082a0 2024-06-06T03:37:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.818+0000 7efc58cce700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc54103040 0x7efc5419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:37:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.818+0000 7efc594cf700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc541082a0 0x7efc541a1e10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00d010 tx=0x7efc4c004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.818+0000 7efc51ffb700 1 -- 172.21.15.40:0/499511418 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c021070 con 0x7efc541082a0 2024-06-06T03:37:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.818+0000 7efc5af32700 1 -- 172.21.15.40:0/499511418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc541a2740 con 0x7efc541082a0 2024-06-06T03:37:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.818+0000 7efc51ffb700 1 -- 172.21.15.40:0/499511418 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc4c007b80 con 0x7efc541082a0 2024-06-06T03:37:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.818+0000 7efc51ffb700 1 -- 172.21.15.40:0/499511418 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c01d450 con 0x7efc541082a0 2024-06-06T03:37:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.818+0000 7efc5af32700 1 -- 172.21.15.40:0/499511418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc541a2ba0 con 0x7efc541082a0 2024-06-06T03:37:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.820+0000 7efc5af32700 1 -- 172.21.15.40:0/499511418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc54069180 con 0x7efc541082a0 2024-06-06T03:37:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.821+0000 7efc51ffb700 1 -- 172.21.15.40:0/499511418 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc4c004140 con 0x7efc541082a0 2024-06-06T03:37:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.821+0000 7efc51ffb700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7efc440a2c20 0x7efc440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.821+0000 7efc58cce700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7efc440a2c20 0x7efc440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.821+0000 7efc51ffb700 1 -- 172.21.15.40:0/499511418 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7efc4c0d00a0 con 0x7efc541082a0 2024-06-06T03:37:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.822+0000 7efc58cce700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7efc440a2c20 0x7efc440a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efc48005950 tx=0x7efc4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:54.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:54.826+0000 7efc51ffb700 1 -- 172.21.15.40:0/499511418 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efc4c09d150 con 0x7efc541082a0 2024-06-06T03:37:55.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:55.039+0000 7efc5af32700 1 -- 172.21.15.40:0/499511418 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7efc540628d0 con 0x7efc440a2c20 2024-06-06T03:37:55.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:55.045+0000 7efc51ffb700 1 -- 172.21.15.40:0/499511418 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7efc540628d0 con 0x7efc440a2c20 2024-06-06T03:37:55.045 INFO:teuthology.orchestra.run.smithi040.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-06T03:37:55.045 INFO:teuthology.orchestra.run.smithi040.stdout:mds.all 12/12 37s ago 3m smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l;count:12 2024-06-06T03:37:55.045 INFO:teuthology.orchestra.run.smithi040.stdout:mgr 3/3 37s ago 8m smithi040=x;smithi104=y;smithi131=z;count:3 2024-06-06T03:37:55.045 INFO:teuthology.orchestra.run.smithi040.stdout:mon 3/3 37s ago 9m smithi040:172.21.15.40=a;smithi104:172.21.15.104=b;smithi131:172.21.15.131=c;count:3 2024-06-06T03:37:55.046 INFO:teuthology.orchestra.run.smithi040.stdout:osd 12 37s ago - 2024-06-06T03:37:55.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:55.047+0000 7efc5af32700 1 -- 172.21.15.40:0/499511418 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7efc440a2c20 msgr2=0x7efc440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:55.047+0000 7efc5af32700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7efc440a2c20 0x7efc440a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efc48005950 tx=0x7efc4800b410 comp rx=0 tx=0).stop 2024-06-06T03:37:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:55.047+0000 7efc5af32700 1 -- 172.21.15.40:0/499511418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc541082a0 msgr2=0x7efc541a1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:55.047+0000 7efc5af32700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc541082a0 0x7efc541a1e10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00d010 tx=0x7efc4c004dd0 comp rx=0 tx=0).stop 2024-06-06T03:37:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:55.047+0000 7efc5af32700 1 -- 172.21.15.40:0/499511418 shutdown_connections 2024-06-06T03:37:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:55.047+0000 7efc5af32700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7efc440a2c20 0x7efc440a50e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:55.047+0000 7efc5af32700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc541082a0 0x7efc541a1e10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:55.047+0000 7efc5af32700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc54103040 0x7efc5419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:55.047+0000 7efc5af32700 1 --2- 172.21.15.40:0/499511418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc54105970 0x7efc5419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:55.048+0000 7efc5af32700 1 -- 172.21.15.40:0/499511418 >> 172.21.15.40:0/499511418 conn(0x7efc540fc990 msgr2=0x7efc54102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:55.048+0000 7efc5af32700 1 -- 172.21.15.40:0/499511418 shutdown_connections 2024-06-06T03:37:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:55.048+0000 7efc5af32700 1 -- 172.21.15.40:0/499511418 wait complete. 2024-06-06T03:37:55.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:54 smithi104 ceph-mon[36569]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:37:55.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:54 smithi131 ceph-mon[36687]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:37:55.677 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-06T03:37:55.928 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:56.050 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:55 smithi040 ceph-mon[32220]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:37:56.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:55 smithi104 ceph-mon[36569]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:37:56.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:55 smithi131 ceph-mon[36687]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:37:57.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:56 smithi040 ceph-mon[32220]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:37:57.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:56 smithi104 ceph-mon[36569]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:37:57.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:56 smithi131 ceph-mon[36687]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:37:57.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.717+0000 7fb2a9daf700 1 -- 172.21.15.40:0/2768412841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a41013f0 msgr2=0x7fb2a4101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:57.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.717+0000 7fb2a9daf700 1 --2- 172.21.15.40:0/2768412841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a41013f0 0x7fb2a4101810 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb29400b0d0 tx=0x7fb29400b3e0 comp rx=0 tx=0).stop 2024-06-06T03:37:57.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.717+0000 7fb2a9daf700 1 -- 172.21.15.40:0/2768412841 shutdown_connections 2024-06-06T03:37:57.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.717+0000 7fb2a9daf700 1 --2- 172.21.15.40:0/2768412841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2a4102f90 0x7fb2a4109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:57.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.717+0000 7fb2a9daf700 1 --2- 172.21.15.40:0/2768412841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a41013f0 0x7fb2a4101810 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:57.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.717+0000 7fb2a9daf700 1 --2- 172.21.15.40:0/2768412841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a41025f0 0x7fb2a4102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:57.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.717+0000 7fb2a9daf700 1 -- 172.21.15.40:0/2768412841 >> 172.21.15.40:0/2768412841 conn(0x7fb2a40fc990 msgr2=0x7fb2a40fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:57.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.717+0000 7fb2a9daf700 1 -- 172.21.15.40:0/2768412841 shutdown_connections 2024-06-06T03:37:57.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.717+0000 7fb2a9daf700 1 -- 172.21.15.40:0/2768412841 wait complete. 2024-06-06T03:37:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.718+0000 7fb2a9daf700 1 Processor -- start 2024-06-06T03:37:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.718+0000 7fb2a9daf700 1 -- start start 2024-06-06T03:37:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.718+0000 7fb2a9daf700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2a41013f0 0x7fb2a4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.718+0000 7fb2a9daf700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a41025f0 0x7fb2a4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.718+0000 7fb2a9daf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a4102f90 0x7fb2a419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.718+0000 7fb2a9daf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a410d030 con 0x7fb2a4102f90 2024-06-06T03:37:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.718+0000 7fb2a9daf700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a410ceb0 con 0x7fb2a41013f0 2024-06-06T03:37:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.718+0000 7fb2a9daf700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a410d1b0 con 0x7fb2a41025f0 2024-06-06T03:37:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a4102f90 0x7fb2a419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a4102f90 0x7fb2a419fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55686/0 (socket says 172.21.15.40:55686) 2024-06-06T03:37:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a3fff700 1 -- 172.21.15.40:0/341536577 learned_addr learned my addr 172.21.15.40:0/341536577 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:37:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a2ffd700 1 --2- 172.21.15.40:0/341536577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a41025f0 0x7fb2a4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a37fe700 1 --2- 172.21.15.40:0/341536577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2a41013f0 0x7fb2a4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a3fff700 1 -- 172.21.15.40:0/341536577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a41025f0 msgr2=0x7fb2a4198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a3fff700 1 --2- 172.21.15.40:0/341536577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a41025f0 0x7fb2a4198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a3fff700 1 -- 172.21.15.40:0/341536577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2a41013f0 msgr2=0x7fb2a4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a3fff700 1 --2- 172.21.15.40:0/341536577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2a41013f0 0x7fb2a4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a3fff700 1 -- 172.21.15.40:0/341536577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a41a0390 con 0x7fb2a4102f90 2024-06-06T03:37:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a3fff700 1 --2- 172.21.15.40:0/341536577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a4102f90 0x7fb2a419fcf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00b580 tx=0x7fb29c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a0ff9700 1 -- 172.21.15.40:0/341536577 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c015020 con 0x7fb2a4102f90 2024-06-06T03:37:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a9daf700 1 -- 172.21.15.40:0/341536577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a41a0680 con 0x7fb2a4102f90 2024-06-06T03:37:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a0ff9700 1 -- 172.21.15.40:0/341536577 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29c011d10 con 0x7fb2a4102f90 2024-06-06T03:37:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a0ff9700 1 -- 172.21.15.40:0/341536577 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c010e00 con 0x7fb2a4102f90 2024-06-06T03:37:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.719+0000 7fb2a9daf700 1 -- 172.21.15.40:0/341536577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a41a0bf0 con 0x7fb2a4102f90 2024-06-06T03:37:57.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.720+0000 7fb2a9daf700 1 -- 172.21.15.40:0/341536577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a4069180 con 0x7fb2a4102f90 2024-06-06T03:37:57.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.722+0000 7fb2a0ff9700 1 -- 172.21.15.40:0/341536577 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb29c018020 con 0x7fb2a4102f90 2024-06-06T03:37:57.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.722+0000 7fb2a0ff9700 1 --2- 172.21.15.40:0/341536577 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fb2900a2c20 0x7fb2900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:37:57.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.722+0000 7fb2a37fe700 1 --2- 172.21.15.40:0/341536577 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fb2900a2c20 0x7fb2900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:37:57.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.722+0000 7fb2a0ff9700 1 -- 172.21.15.40:0/341536577 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fb29c0cebe0 con 0x7fb2a4102f90 2024-06-06T03:37:57.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.723+0000 7fb2a37fe700 1 --2- 172.21.15.40:0/341536577 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fb2900a2c20 0x7fb2900a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb294009cd0 tx=0x7fb29400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:37:57.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.726+0000 7fb2a0ff9700 1 -- 172.21.15.40:0/341536577 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb29c01f020 con 0x7fb2a4102f90 2024-06-06T03:37:57.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.945+0000 7fb2a9daf700 1 -- 172.21.15.40:0/341536577 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fb2a40feb40 con 0x7fb2900a2c20 2024-06-06T03:37:57.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.947+0000 7fb2a0ff9700 1 -- 172.21.15.40:0/341536577 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fb2a40feb40 con 0x7fb2900a2c20 2024-06-06T03:37:57.946 INFO:teuthology.orchestra.run.smithi040.stdout:HOST ADDR LABELS STATUS 2024-06-06T03:37:57.946 INFO:teuthology.orchestra.run.smithi040.stdout:smithi040 172.21.15.40 2024-06-06T03:37:57.947 INFO:teuthology.orchestra.run.smithi040.stdout:smithi104 172.21.15.104 2024-06-06T03:37:57.947 INFO:teuthology.orchestra.run.smithi040.stdout:smithi131 172.21.15.131 2024-06-06T03:37:57.947 INFO:teuthology.orchestra.run.smithi040.stdout:3 hosts in cluster 2024-06-06T03:37:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.949+0000 7fb2a9daf700 1 -- 172.21.15.40:0/341536577 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fb2900a2c20 msgr2=0x7fb2900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.949+0000 7fb2a9daf700 1 --2- 172.21.15.40:0/341536577 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fb2900a2c20 0x7fb2900a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb294009cd0 tx=0x7fb29400b040 comp rx=0 tx=0).stop 2024-06-06T03:37:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.949+0000 7fb2a9daf700 1 -- 172.21.15.40:0/341536577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a4102f90 msgr2=0x7fb2a419fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:37:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.949+0000 7fb2a9daf700 1 --2- 172.21.15.40:0/341536577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a4102f90 0x7fb2a419fcf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00b580 tx=0x7fb29c00b890 comp rx=0 tx=0).stop 2024-06-06T03:37:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.949+0000 7fb2a9daf700 1 -- 172.21.15.40:0/341536577 shutdown_connections 2024-06-06T03:37:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.949+0000 7fb2a9daf700 1 --2- 172.21.15.40:0/341536577 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fb2900a2c20 0x7fb2900a50e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.949+0000 7fb2a9daf700 1 --2- 172.21.15.40:0/341536577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a4102f90 0x7fb2a419fcf0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.949+0000 7fb2a9daf700 1 --2- 172.21.15.40:0/341536577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2a41013f0 0x7fb2a4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.949+0000 7fb2a9daf700 1 --2- 172.21.15.40:0/341536577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a41025f0 0x7fb2a4198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:37:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.949+0000 7fb2a9daf700 1 -- 172.21.15.40:0/341536577 >> 172.21.15.40:0/341536577 conn(0x7fb2a40fc990 msgr2=0x7fb2a40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:37:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.949+0000 7fb2a9daf700 1 -- 172.21.15.40:0/341536577 shutdown_connections 2024-06-06T03:37:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:37:57.949+0000 7fb2a9daf700 1 -- 172.21.15.40:0/341536577 wait complete. 2024-06-06T03:37:58.845 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-06T03:37:59.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:58 smithi040 ceph-mon[32220]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:37:59.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:37:58 smithi040 ceph-mon[32220]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:37:59.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:58 smithi104 ceph-mon[36569]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:37:59.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:37:58 smithi104 ceph-mon[36569]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:37:59.102 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:37:59.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:58 smithi131 ceph-mon[36687]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:37:59.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:37:58 smithi131 ceph-mon[36687]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:01.017 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:00 smithi040 ceph-mon[32220]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:38:01.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.032+0000 7faafa7a3700 1 -- 172.21.15.40:0/526854098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf41024f0 msgr2=0x7faaf4102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:01.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.032+0000 7faafa7a3700 1 --2- 172.21.15.40:0/526854098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf41024f0 0x7faaf4102910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faae40099c0 tx=0x7faae4009cd0 comp rx=0 tx=0).stop 2024-06-06T03:38:01.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.032+0000 7faafa7a3700 1 -- 172.21.15.40:0/526854098 shutdown_connections 2024-06-06T03:38:01.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.032+0000 7faafa7a3700 1 --2- 172.21.15.40:0/526854098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faaf4104090 0x7faaf410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:01.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.032+0000 7faafa7a3700 1 --2- 172.21.15.40:0/526854098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf41024f0 0x7faaf4102910 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:01.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.032+0000 7faafa7a3700 1 --2- 172.21.15.40:0/526854098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaf41036f0 0x7faaf4103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:01.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.032+0000 7faafa7a3700 1 -- 172.21.15.40:0/526854098 >> 172.21.15.40:0/526854098 conn(0x7faaf40fda90 msgr2=0x7faaf40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:01.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.032+0000 7faafa7a3700 1 -- 172.21.15.40:0/526854098 shutdown_connections 2024-06-06T03:38:01.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.032+0000 7faafa7a3700 1 -- 172.21.15.40:0/526854098 wait complete. 2024-06-06T03:38:01.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.032+0000 7faafa7a3700 1 Processor -- start 2024-06-06T03:38:01.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.032+0000 7faafa7a3700 1 -- start start 2024-06-06T03:38:01.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faafa7a3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaf41024f0 0x7faaf4076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:01.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faafa7a3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf41036f0 0x7faaf4074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faafa7a3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faaf4104090 0x7faaf4074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faafa7a3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faaf410e0f0 con 0x7faaf41036f0 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faafa7a3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faaf410df70 con 0x7faaf4104090 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faafa7a3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faaf410e270 con 0x7faaf41024f0 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faaebfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf41036f0 0x7faaf4074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faaebfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf41036f0 0x7faaf4074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37146/0 (socket says 172.21.15.40:37146) 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faaebfff700 1 -- 172.21.15.40:0/2581807166 learned_addr learned my addr 172.21.15.40:0/2581807166 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faaf3fff700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaf41024f0 0x7faaf4076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faaf8d40700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faaf4104090 0x7faaf4074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faaebfff700 1 -- 172.21.15.40:0/2581807166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaf41024f0 msgr2=0x7faaf4076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faaebfff700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaf41024f0 0x7faaf4076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faaebfff700 1 -- 172.21.15.40:0/2581807166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faaf4104090 msgr2=0x7faaf4074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faaebfff700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faaf4104090 0x7faaf4074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faaebfff700 1 -- 172.21.15.40:0/2581807166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf4075670 con 0x7faaf41036f0 2024-06-06T03:38:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faaf3fff700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaf41024f0 0x7faaf4076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:01.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faaebfff700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf41036f0 0x7faaf4074880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faadc00d980 tx=0x7faadc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:01.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.033+0000 7faaf1ffb700 1 -- 172.21.15.40:0/2581807166 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc017070 con 0x7faaf41036f0 2024-06-06T03:38:01.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.034+0000 7faaf1ffb700 1 -- 172.21.15.40:0/2581807166 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faadc005020 con 0x7faaf41036f0 2024-06-06T03:38:01.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.034+0000 7faafa7a3700 1 -- 172.21.15.40:0/2581807166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf41a2890 con 0x7faaf41036f0 2024-06-06T03:38:01.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.034+0000 7faaf1ffb700 1 -- 172.21.15.40:0/2581807166 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc009bc0 con 0x7faaf41036f0 2024-06-06T03:38:01.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.034+0000 7faafa7a3700 1 -- 172.21.15.40:0/2581807166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf41a2d50 con 0x7faaf41036f0 2024-06-06T03:38:01.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.035+0000 7faafa7a3700 1 -- 172.21.15.40:0/2581807166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf4069180 con 0x7faaf41036f0 2024-06-06T03:38:01.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.036+0000 7faaf1ffb700 1 -- 172.21.15.40:0/2581807166 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faadc0051d0 con 0x7faaf41036f0 2024-06-06T03:38:01.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.037+0000 7faaf1ffb700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7faae00a2c20 0x7faae00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:01.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.037+0000 7faaf1ffb700 1 -- 172.21.15.40:0/2581807166 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7faadc0ceeb0 con 0x7faaf41036f0 2024-06-06T03:38:01.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.037+0000 7faaf3fff700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7faae00a2c20 0x7faae00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:01.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.037+0000 7faaf3fff700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7faae00a2c20 0x7faae00a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faae400b580 tx=0x7faae4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:01.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.041+0000 7faaf1ffb700 1 -- 172.21.15.40:0/2581807166 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faadc020020 con 0x7faaf41036f0 2024-06-06T03:38:01.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:00 smithi131 ceph-mon[36687]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:38:01.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:00 smithi104 ceph-mon[36569]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:38:01.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.257+0000 7faafa7a3700 1 -- 172.21.15.40:0/2581807166 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7faaf40628d0 con 0x7faae00a2c20 2024-06-06T03:38:01.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.259+0000 7faaf1ffb700 1 -- 172.21.15.40:0/2581807166 <== mgr.14152 v2:172.21.15.40:6800/869712339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7faaf40628d0 con 0x7faae00a2c20 2024-06-06T03:38:01.258 INFO:teuthology.orchestra.run.smithi040.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-06T03:38:01.258 INFO:teuthology.orchestra.run.smithi040.stdout:smithi040 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009J400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-06-06T03:38:01.258 INFO:teuthology.orchestra.run.smithi040.stdout:smithi104 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NW400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-06-06T03:38:01.259 INFO:teuthology.orchestra.run.smithi040.stdout:smithi131 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300N4400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-06-06T03:38:01.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.261+0000 7faafa7a3700 1 -- 172.21.15.40:0/2581807166 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7faae00a2c20 msgr2=0x7faae00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:01.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.261+0000 7faafa7a3700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7faae00a2c20 0x7faae00a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faae400b580 tx=0x7faae4005fb0 comp rx=0 tx=0).stop 2024-06-06T03:38:01.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.261+0000 7faafa7a3700 1 -- 172.21.15.40:0/2581807166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf41036f0 msgr2=0x7faaf4074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:01.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.261+0000 7faafa7a3700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf41036f0 0x7faaf4074880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faadc00d980 tx=0x7faadc00dc90 comp rx=0 tx=0).stop 2024-06-06T03:38:01.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.261+0000 7faafa7a3700 1 -- 172.21.15.40:0/2581807166 shutdown_connections 2024-06-06T03:38:01.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.261+0000 7faafa7a3700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7faae00a2c20 0x7faae00a50e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:01.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.261+0000 7faafa7a3700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaf41024f0 0x7faaf4076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:01.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.261+0000 7faafa7a3700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faaf4104090 0x7faaf4074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:01.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.261+0000 7faafa7a3700 1 --2- 172.21.15.40:0/2581807166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf41036f0 0x7faaf4074880 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:01.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.261+0000 7faafa7a3700 1 -- 172.21.15.40:0/2581807166 >> 172.21.15.40:0/2581807166 conn(0x7faaf40fda90 msgr2=0x7faaf41069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:01.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.261+0000 7faafa7a3700 1 -- 172.21.15.40:0/2581807166 shutdown_connections 2024-06-06T03:38:01.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:01.261+0000 7faafa7a3700 1 -- 172.21.15.40:0/2581807166 wait complete. 2024-06-06T03:38:01.947 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:01 smithi040 ceph-mon[32220]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:01.976 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-06T03:38:01.985 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi040.front.sepia.ceph.com 2024-06-06T03:38:01.985 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-06T03:38:02.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:01 smithi104 ceph-mon[36569]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:02.235 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:02.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:01 smithi131 ceph-mon[36687]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:03.055 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:02 smithi040 ceph-mon[32220]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:38:03.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:02 smithi104 ceph-mon[36569]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:38:03.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:02 smithi131 ceph-mon[36687]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:38:04.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.071+0000 7ff9a065e700 1 -- 172.21.15.40:0/354178165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9981036f0 msgr2=0x7ff998103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:04.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.071+0000 7ff9a065e700 1 --2- 172.21.15.40:0/354178165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9981036f0 0x7ff998103b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff9880099c0 tx=0x7ff988009cd0 comp rx=0 tx=0).stop 2024-06-06T03:38:04.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.071+0000 7ff9a065e700 1 -- 172.21.15.40:0/354178165 shutdown_connections 2024-06-06T03:38:04.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.071+0000 7ff9a065e700 1 --2- 172.21.15.40:0/354178165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff998104090 0x7ff99810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:04.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.071+0000 7ff9a065e700 1 --2- 172.21.15.40:0/354178165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9981024f0 0x7ff998102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:04.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.071+0000 7ff9a065e700 1 --2- 172.21.15.40:0/354178165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9981036f0 0x7ff998103b50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:04.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.071+0000 7ff9a065e700 1 -- 172.21.15.40:0/354178165 >> 172.21.15.40:0/354178165 conn(0x7ff9980fda90 msgr2=0x7ff9980ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:04.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.071+0000 7ff9a065e700 1 -- 172.21.15.40:0/354178165 shutdown_connections 2024-06-06T03:38:04.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.071+0000 7ff9a065e700 1 -- 172.21.15.40:0/354178165 wait complete. 2024-06-06T03:38:04.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.072+0000 7ff9a065e700 1 Processor -- start 2024-06-06T03:38:04.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.072+0000 7ff9a065e700 1 -- start start 2024-06-06T03:38:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.072+0000 7ff9a065e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9981024f0 0x7ff99819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.072+0000 7ff9a065e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9981036f0 0x7ff99819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.072+0000 7ff9a065e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff998104090 0x7ff9981a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.072+0000 7ff9a065e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff99810e160 con 0x7ff9981024f0 2024-06-06T03:38:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.072+0000 7ff9a065e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff99810dfe0 con 0x7ff9981036f0 2024-06-06T03:38:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.072+0000 7ff9a065e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff99810e2e0 con 0x7ff998104090 2024-06-06T03:38:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.072+0000 7ff99e3fa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9981024f0 0x7ff99819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.072+0000 7ff99e3fa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9981024f0 0x7ff99819a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37164/0 (socket says 172.21.15.40:37164) 2024-06-06T03:38:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.072+0000 7ff99e3fa700 1 -- 172.21.15.40:0/2117406436 learned_addr learned my addr 172.21.15.40:0/2117406436 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.073+0000 7ff99dbf9700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9981036f0 0x7ff99819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.073+0000 7ff99ebfb700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff998104090 0x7ff9981a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.073+0000 7ff99e3fa700 1 -- 172.21.15.40:0/2117406436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff998104090 msgr2=0x7ff9981a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.073+0000 7ff99e3fa700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff998104090 0x7ff9981a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.073+0000 7ff99e3fa700 1 -- 172.21.15.40:0/2117406436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9981036f0 msgr2=0x7ff99819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.073+0000 7ff99e3fa700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9981036f0 0x7ff99819aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.073+0000 7ff99e3fa700 1 -- 172.21.15.40:0/2117406436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9981a25e0 con 0x7ff9981024f0 2024-06-06T03:38:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.073+0000 7ff99ebfb700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff998104090 0x7ff9981a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:38:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.073+0000 7ff99e3fa700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9981024f0 0x7ff99819a980 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff99400b530 tx=0x7ff99400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.073+0000 7ff98f7fe700 1 -- 172.21.15.40:0/2117406436 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994015020 con 0x7ff9981024f0 2024-06-06T03:38:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.073+0000 7ff98f7fe700 1 -- 172.21.15.40:0/2117406436 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff994011d10 con 0x7ff9981024f0 2024-06-06T03:38:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.073+0000 7ff9a065e700 1 -- 172.21.15.40:0/2117406436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9981a28d0 con 0x7ff9981024f0 2024-06-06T03:38:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.073+0000 7ff98f7fe700 1 -- 172.21.15.40:0/2117406436 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99400f2e0 con 0x7ff9981024f0 2024-06-06T03:38:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.073+0000 7ff9a065e700 1 -- 172.21.15.40:0/2117406436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9981a2e80 con 0x7ff9981024f0 2024-06-06T03:38:04.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.074+0000 7ff9a065e700 1 -- 172.21.15.40:0/2117406436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9981a30b0 con 0x7ff9981024f0 2024-06-06T03:38:04.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.076+0000 7ff98f7fe700 1 -- 172.21.15.40:0/2117406436 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff994018020 con 0x7ff9981024f0 2024-06-06T03:38:04.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.076+0000 7ff98f7fe700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff9840a2ca0 0x7ff9840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:04.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.076+0000 7ff99dbf9700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff9840a2ca0 0x7ff9840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:04.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.076+0000 7ff98f7fe700 1 -- 172.21.15.40:0/2117406436 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7ff9940ce620 con 0x7ff9981024f0 2024-06-06T03:38:04.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.077+0000 7ff99dbf9700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff9840a2ca0 0x7ff9840a5160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff988000c00 tx=0x7ff988005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:04.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.081+0000 7ff98f7fe700 1 -- 172.21.15.40:0/2117406436 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff99401f020 con 0x7ff9981024f0 2024-06-06T03:38:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.331+0000 7ff9a065e700 1 -- 172.21.15.40:0/2117406436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff9981a3280 con 0x7ff9981024f0 2024-06-06T03:38:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.331+0000 7ff98f7fe700 1 -- 172.21.15.40:0/2117406436 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3092 (secure 0 0 0) 0x7ff994010e90 con 0x7ff9981024f0 2024-06-06T03:38:04.331 INFO:teuthology.orchestra.run.smithi040.stdout:e28 2024-06-06T03:38:04.331 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T03:38:04.331 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T03:38:04.331 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-06-06T03:38:04.331 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:38:04.331 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-06-06T03:38:04.331 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-06-06T03:38:04.331 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 28 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-06-06T03:35:15.961853+0000 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-06-06T03:35:38.053505+0000 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 300 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 0 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 5 2024-06-06T03:38:04.332 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1,2,3,4 2024-06-06T03:38:04.333 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=24470,1=24443,2=24425,3=24403,4=24398} 2024-06-06T03:38:04.333 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-06-06T03:38:04.333 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-06-06T03:38:04.333 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-06-06T03:38:04.333 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-06-06T03:38:04.333 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-06-06T03:38:04.333 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-06-06T03:38:04.333 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-06-06T03:38:04.333 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-06-06T03:38:04.333 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 0 2024-06-06T03:38:04.333 INFO:teuthology.orchestra.run.smithi040.stdout:qdb_cluster leader: 24470 members: 24398,24403,24425,24443,24470 2024-06-06T03:38:04.333 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b{0:24470} state up:active seq 6 addr [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T03:38:04.333 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c{0:24388} state up:standby-replay seq 1 addr [v2:172.21.15.131:6836/526159320,v1:172.21.15.131:6837/526159320] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T03:38:04.333 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.h{1:24443} state up:active seq 13 addr [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T03:38:04.334 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.f{1:24376} state up:standby-replay seq 1 addr [v2:172.21.15.131:6834/1221528906,v1:172.21.15.131:6835/1221528906] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T03:38:04.334 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.e{2:24425} state up:active seq 15 addr [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T03:38:04.334 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.l{2:24358} state up:standby-replay seq 1 addr [v2:172.21.15.131:6832/1833802668,v1:172.21.15.131:6833/1833802668] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T03:38:04.334 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.i{3:24403} state up:active seq 13 addr [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T03:38:04.334 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a{3:14586} state up:standby-replay seq 1 addr [v2:172.21.15.40:6840/2460301066,v1:172.21.15.40:6841/2460301066] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T03:38:04.334 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.k{4:24398} state up:active seq 18 addr [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T03:38:04.334 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.g{4:14580} state up:standby-replay seq 1 addr [v2:172.21.15.40:6838/1930069210,v1:172.21.15.40:6839/1930069210] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T03:38:04.334 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:38:04.334 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:38:04.334 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-06-06T03:38:04.334 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:38:04.334 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.j{-1:14532} state up:standby seq 1 addr [v2:172.21.15.40:6834/1980325130,v1:172.21.15.40:6835/1980325130] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T03:38:04.334 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.40:6836/1373355997,v1:172.21.15.40:6837/1373355997] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T03:38:04.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.333+0000 7ff9a065e700 1 -- 172.21.15.40:0/2117406436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff9840a2ca0 msgr2=0x7ff9840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:04.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.333+0000 7ff9a065e700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff9840a2ca0 0x7ff9840a5160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff988000c00 tx=0x7ff988005fb0 comp rx=0 tx=0).stop 2024-06-06T03:38:04.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.333+0000 7ff9a065e700 1 -- 172.21.15.40:0/2117406436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9981024f0 msgr2=0x7ff99819a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:04.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.333+0000 7ff9a065e700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9981024f0 0x7ff99819a980 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff99400b530 tx=0x7ff99400b840 comp rx=0 tx=0).stop 2024-06-06T03:38:04.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.333+0000 7ff9a065e700 1 -- 172.21.15.40:0/2117406436 shutdown_connections 2024-06-06T03:38:04.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.333+0000 7ff9a065e700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7ff9840a2ca0 0x7ff9840a5160 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:04.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.333+0000 7ff9a065e700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff998104090 0x7ff9981a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:04.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.333+0000 7ff9a065e700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9981024f0 0x7ff99819a980 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:04.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.333+0000 7ff9a065e700 1 --2- 172.21.15.40:0/2117406436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9981036f0 0x7ff99819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:04.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.333+0000 7ff9a065e700 1 -- 172.21.15.40:0/2117406436 >> 172.21.15.40:0/2117406436 conn(0x7ff9980fda90 msgr2=0x7ff9980746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:04.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.333+0000 7ff9a065e700 1 -- 172.21.15.40:0/2117406436 shutdown_connections 2024-06-06T03:38:04.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:04.333+0000 7ff9a065e700 1 -- 172.21.15.40:0/2117406436 wait complete. 2024-06-06T03:38:04.336 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 28 2024-06-06T03:38:04.915 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:04 smithi040 ceph-mon[32220]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:38:04.916 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:04 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2117406436' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T03:38:04.947 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-06T03:38:05.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:04 smithi104 ceph-mon[36569]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:38:05.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:04 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2117406436' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T03:38:05.209 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:05.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:04 smithi131 ceph-mon[36687]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:38:05.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:04 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2117406436' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T03:38:06.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.947+0000 7f120cca7700 1 -- 172.21.15.40:0/2912951442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1208104090 msgr2=0x7f120810a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:06.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.947+0000 7f120cca7700 1 --2- 172.21.15.40:0/2912951442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1208104090 0x7f120810a940 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f120000b0d0 tx=0x7f120000b3e0 comp rx=0 tx=0).stop 2024-06-06T03:38:06.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.947+0000 7f120cca7700 1 -- 172.21.15.40:0/2912951442 shutdown_connections 2024-06-06T03:38:06.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.947+0000 7f120cca7700 1 --2- 172.21.15.40:0/2912951442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12081024f0 0x7f1208102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:06.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.947+0000 7f120cca7700 1 --2- 172.21.15.40:0/2912951442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1208104090 0x7f120810a940 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:06.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.947+0000 7f120cca7700 1 --2- 172.21.15.40:0/2912951442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12081036f0 0x7f1208103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:06.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.947+0000 7f120cca7700 1 -- 172.21.15.40:0/2912951442 >> 172.21.15.40:0/2912951442 conn(0x7f12080fda90 msgr2=0x7f12080ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:06.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.947+0000 7f120cca7700 1 -- 172.21.15.40:0/2912951442 shutdown_connections 2024-06-06T03:38:06.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.947+0000 7f120cca7700 1 -- 172.21.15.40:0/2912951442 wait complete. 2024-06-06T03:38:06.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.948+0000 7f120cca7700 1 Processor -- start 2024-06-06T03:38:06.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.948+0000 7f120cca7700 1 -- start start 2024-06-06T03:38:06.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.948+0000 7f120cca7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12081024f0 0x7f1208198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:06.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.948+0000 7f120cca7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12081036f0 0x7f1208198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:06.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.948+0000 7f120cca7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1208104090 0x7f120819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:06.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f120cca7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f120810e150 con 0x7f12081024f0 2024-06-06T03:38:06.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f120cca7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f120810dfd0 con 0x7f12081036f0 2024-06-06T03:38:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f120cca7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f120810e2d0 con 0x7f1208104090 2024-06-06T03:38:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f120659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12081024f0 0x7f1208198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f120659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12081024f0 0x7f1208198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37182/0 (socket says 172.21.15.40:37182) 2024-06-06T03:38:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f120659c700 1 -- 172.21.15.40:0/1297253392 learned_addr learned my addr 172.21.15.40:0/1297253392 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f1205d9b700 1 --2- 172.21.15.40:0/1297253392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12081036f0 0x7f1208198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f120659c700 1 -- 172.21.15.40:0/1297253392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1208104090 msgr2=0x7f120819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f120659c700 1 --2- 172.21.15.40:0/1297253392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1208104090 0x7f120819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f120659c700 1 -- 172.21.15.40:0/1297253392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12081036f0 msgr2=0x7f1208198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f120659c700 1 --2- 172.21.15.40:0/1297253392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12081036f0 0x7f1208198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f120659c700 1 -- 172.21.15.40:0/1297253392 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12081a0360 con 0x7f12081024f0 2024-06-06T03:38:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f120659c700 1 --2- 172.21.15.40:0/1297253392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12081024f0 0x7f1208198700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f11f800c920 tx=0x7f11f800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f11f77fe700 1 -- 172.21.15.40:0/1297253392 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8004220 con 0x7f12081024f0 2024-06-06T03:38:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f11f77fe700 1 -- 172.21.15.40:0/1297253392 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f8007da0 con 0x7f12081024f0 2024-06-06T03:38:06.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f120cca7700 1 -- 172.21.15.40:0/1297253392 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12081a0650 con 0x7f12081024f0 2024-06-06T03:38:06.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f11f77fe700 1 -- 172.21.15.40:0/1297253392 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8003d10 con 0x7f12081024f0 2024-06-06T03:38:06.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.949+0000 7f120cca7700 1 -- 172.21.15.40:0/1297253392 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12081a0c10 con 0x7f12081024f0 2024-06-06T03:38:06.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.950+0000 7f120cca7700 1 -- 172.21.15.40:0/1297253392 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1208069180 con 0x7f12081024f0 2024-06-06T03:38:06.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.951+0000 7f11f77fe700 1 -- 172.21.15.40:0/1297253392 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f11f80043c0 con 0x7f12081024f0 2024-06-06T03:38:06.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.952+0000 7f11f77fe700 1 --2- 172.21.15.40:0/1297253392 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f11f00a2ca0 0x7f11f00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:06.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.952+0000 7f1205d9b700 1 --2- 172.21.15.40:0/1297253392 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f11f00a2ca0 0x7f11f00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:06.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.952+0000 7f11f77fe700 1 -- 172.21.15.40:0/1297253392 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f11f80ceed0 con 0x7f12081024f0 2024-06-06T03:38:06.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.953+0000 7f1205d9b700 1 --2- 172.21.15.40:0/1297253392 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f11f00a2ca0 0x7f11f00a5160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009a10 tx=0x7f11fc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:06.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:06.956+0000 7f11f77fe700 1 -- 172.21.15.40:0/1297253392 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f11f8014030 con 0x7f12081024f0 2024-06-06T03:38:07.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:06 smithi040 ceph-mon[32220]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:38:07.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:06 smithi104 ceph-mon[36569]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:38:07.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:07.163+0000 7f120cca7700 1 -- 172.21.15.40:0/1297253392 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f120802dd40 con 0x7f12081024f0 2024-06-06T03:38:07.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:07.164+0000 7f11f77fe700 1 -- 172.21.15.40:0/1297253392 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5212 (secure 0 0 0) 0x7f11f809bf70 con 0x7f12081024f0 2024-06-06T03:38:07.164 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 90 2024-06-06T03:38:07.164 INFO:teuthology.orchestra.run.smithi040.stdout:fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:38:07.164 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-06-06T03:26:53.165489+0000 2024-06-06T03:38:07.164 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-06-06T03:35:48.980461+0000 2024-06-06T03:38:07.164 INFO:teuthology.orchestra.run.smithi040.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-06T03:38:07.164 INFO:teuthology.orchestra.run.smithi040.stdout:crush_version 26 2024-06-06T03:38:07.164 INFO:teuthology.orchestra.run.smithi040.stdout:full_ratio 0.95 2024-06-06T03:38:07.165 INFO:teuthology.orchestra.run.smithi040.stdout:backfillfull_ratio 0.9 2024-06-06T03:38:07.165 INFO:teuthology.orchestra.run.smithi040.stdout:nearfull_ratio 0.85 2024-06-06T03:38:07.165 INFO:teuthology.orchestra.run.smithi040.stdout:require_min_compat_client luminous 2024-06-06T03:38:07.165 INFO:teuthology.orchestra.run.smithi040.stdout:min_compat_client luminous 2024-06-06T03:38:07.165 INFO:teuthology.orchestra.run.smithi040.stdout:require_osd_release squid 2024-06-06T03:38:07.165 INFO:teuthology.orchestra.run.smithi040.stdout:stretch_mode_enabled false 2024-06-06T03:38:07.165 INFO:teuthology.orchestra.run.smithi040.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-06T03:38:07.165 INFO:teuthology.orchestra.run.smithi040.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-06-06T03:38:07.165 INFO:teuthology.orchestra.run.smithi040.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-06-06T03:38:07.165 INFO:teuthology.orchestra.run.smithi040.stdout:max_osd 12 2024-06-06T03:38:07.165 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6802/3453526793,v1:172.21.15.40:6803/3453526793] [v2:172.21.15.40:6804/3453526793,v1:172.21.15.40:6805/3453526793] exists,up 63c0a320-a2f2-4b84-87e7-68501d335285 2024-06-06T03:38:07.165 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6800/405854252,v1:172.21.15.104:6801/405854252] [v2:172.21.15.104:6802/405854252,v1:172.21.15.104:6803/405854252] exists,up 70a31534-06a1-4ffa-a54d-577d45bccf98 2024-06-06T03:38:07.165 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6800/3919221141,v1:172.21.15.131:6801/3919221141] [v2:172.21.15.131:6802/3919221141,v1:172.21.15.131:6803/3919221141] exists,up f5a89f43-e533-45f0-a777-bd3e9bfa8072 2024-06-06T03:38:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6810/2008541522,v1:172.21.15.40:6811/2008541522] [v2:172.21.15.40:6812/2008541522,v1:172.21.15.40:6813/2008541522] exists,up 4df19390-6378-4346-a2be-29bce1dd2380 2024-06-06T03:38:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 up in weight 1 up_from 33 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6808/1153394847,v1:172.21.15.104:6809/1153394847] [v2:172.21.15.104:6810/1153394847,v1:172.21.15.104:6811/1153394847] exists,up 508de408-1cb1-4423-8199-db6bde9034f3 2024-06-06T03:38:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 up in weight 1 up_from 38 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6808/1051933303,v1:172.21.15.131:6809/1051933303] [v2:172.21.15.131:6810/1051933303,v1:172.21.15.131:6811/1051933303] exists,up a8c0cec9-a8ad-4a5d-800b-8b891ca6d836 2024-06-06T03:38:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6 up in weight 1 up_from 46 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6818/2670922011,v1:172.21.15.40:6819/2670922011] [v2:172.21.15.40:6820/2670922011,v1:172.21.15.40:6821/2670922011] exists,up 5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75 2024-06-06T03:38:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7 up in weight 1 up_from 51 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6816/2353571051,v1:172.21.15.104:6817/2353571051] [v2:172.21.15.104:6818/2353571051,v1:172.21.15.104:6819/2353571051] exists,up bfc536bb-6ae4-459e-afd9-8197c7420bd1 2024-06-06T03:38:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8 up in weight 1 up_from 59 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6816/1364533794,v1:172.21.15.131:6817/1364533794] [v2:172.21.15.131:6818/1364533794,v1:172.21.15.131:6819/1364533794] exists,up 1fe214f2-5ea3-4fc5-83bf-d9bccafe1529 2024-06-06T03:38:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9 up in weight 1 up_from 67 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6826/2604133190,v1:172.21.15.40:6827/2604133190] [v2:172.21.15.40:6828/2604133190,v1:172.21.15.40:6829/2604133190] exists,up 4b2b3555-6870-4ead-be50-b635799265a9 2024-06-06T03:38:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10 up in weight 1 up_from 72 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6824/2231164832,v1:172.21.15.104:6825/2231164832] [v2:172.21.15.104:6826/2231164832,v1:172.21.15.104:6827/2231164832] exists,up 70049e24-94ce-458e-97e3-dbcf9de05444 2024-06-06T03:38:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11 up in weight 1 up_from 80 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6824/4128185670,v1:172.21.15.131:6825/4128185670] [v2:172.21.15.131:6826/4128185670,v1:172.21.15.131:6827/4128185670] exists,up 80447548-1fbe-4892-ac13-eb4bb6acf10d 2024-06-06T03:38:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 2.a [2,11] 2024-06-06T03:38:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 2.10 [8,11] 2024-06-06T03:38:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 3.0 [5,9] 2024-06-06T03:38:07.167 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 3.3 [4,9] 2024-06-06T03:38:07.167 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 3.13 [4,9] 2024-06-06T03:38:07.167 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:0/23942578 expires 2024-06-07T03:27:47.578838+0000 2024-06-06T03:38:07.167 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:0/2234980855 expires 2024-06-07T03:27:47.578838+0000 2024-06-06T03:38:07.167 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:6801/1920198673 expires 2024-06-07T03:27:19.799499+0000 2024-06-06T03:38:07.167 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:6800/3930494734 expires 2024-06-07T03:27:47.578838+0000 2024-06-06T03:38:07.167 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:0/2301695357 expires 2024-06-07T03:27:47.578838+0000 2024-06-06T03:38:07.167 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:0/1676880727 expires 2024-06-07T03:27:19.799499+0000 2024-06-06T03:38:07.167 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:6801/3930494734 expires 2024-06-07T03:27:47.578838+0000 2024-06-06T03:38:07.167 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:6800/1920198673 expires 2024-06-07T03:27:19.799499+0000 2024-06-06T03:38:07.167 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:0/1597665640 expires 2024-06-07T03:27:19.799499+0000 2024-06-06T03:38:07.167 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:0/513375601 expires 2024-06-07T03:27:19.799499+0000 2024-06-06T03:38:07.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:07.166+0000 7f120cca7700 1 -- 172.21.15.40:0/1297253392 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f11f00a2ca0 msgr2=0x7f11f00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:07.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:07.166+0000 7f120cca7700 1 --2- 172.21.15.40:0/1297253392 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f11f00a2ca0 0x7f11f00a5160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009a10 tx=0x7f11fc009190 comp rx=0 tx=0).stop 2024-06-06T03:38:07.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:07.166+0000 7f120cca7700 1 -- 172.21.15.40:0/1297253392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12081024f0 msgr2=0x7f1208198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:07.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:07.166+0000 7f120cca7700 1 --2- 172.21.15.40:0/1297253392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12081024f0 0x7f1208198700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f11f800c920 tx=0x7f11f800cc30 comp rx=0 tx=0).stop 2024-06-06T03:38:07.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:07.166+0000 7f120cca7700 1 -- 172.21.15.40:0/1297253392 shutdown_connections 2024-06-06T03:38:07.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:07.167+0000 7f120cca7700 1 --2- 172.21.15.40:0/1297253392 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f11f00a2ca0 0x7f11f00a5160 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:07.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:07.167+0000 7f120cca7700 1 --2- 172.21.15.40:0/1297253392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1208104090 0x7f120819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:07.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:07.167+0000 7f120cca7700 1 --2- 172.21.15.40:0/1297253392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12081024f0 0x7f1208198700 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:07.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:07.167+0000 7f120cca7700 1 --2- 172.21.15.40:0/1297253392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12081036f0 0x7f1208198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:07.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:07.167+0000 7f120cca7700 1 -- 172.21.15.40:0/1297253392 >> 172.21.15.40:0/1297253392 conn(0x7f12080fda90 msgr2=0x7f12081069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:07.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:07.167+0000 7f120cca7700 1 -- 172.21.15.40:0/1297253392 shutdown_connections 2024-06-06T03:38:07.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:07.167+0000 7f120cca7700 1 -- 172.21.15.40:0/1297253392 wait complete. 2024-06-06T03:38:07.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:06 smithi131 ceph-mon[36687]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:38:07.813 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-06T03:38:07.825 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:38:08.076 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:08.098 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:07 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1297253392' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T03:38:08.098 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:07 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1297253392' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T03:38:08.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:07 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1297253392' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T03:38:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:08 smithi131 ceph-mon[36687]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:38:09.318 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:08 smithi040 ceph-mon[32220]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:38:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:08 smithi104 ceph-mon[36569]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:38:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.895+0000 7f5274c38700 1 -- 172.21.15.40:0/3422939896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5270108520 msgr2=0x7f5270108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.895+0000 7f5274c38700 1 --2- 172.21.15.40:0/3422939896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5270108520 0x7f5270108900 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5260009990 tx=0x7f5260009ca0 comp rx=0 tx=0).stop 2024-06-06T03:38:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.896+0000 7f5274c38700 1 -- 172.21.15.40:0/3422939896 shutdown_connections 2024-06-06T03:38:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.896+0000 7f5274c38700 1 --2- 172.21.15.40:0/3422939896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5270102ed0 0x7f527010f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.896+0000 7f5274c38700 1 --2- 172.21.15.40:0/3422939896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5270108520 0x7f5270108900 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.896+0000 7f5274c38700 1 --2- 172.21.15.40:0/3422939896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5270102510 0x7f5270102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.896+0000 7f5274c38700 1 -- 172.21.15.40:0/3422939896 >> 172.21.15.40:0/3422939896 conn(0x7f5270079640 msgr2=0x7f5270079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:09.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.896+0000 7f5274c38700 1 -- 172.21.15.40:0/3422939896 shutdown_connections 2024-06-06T03:38:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.896+0000 7f5274c38700 1 -- 172.21.15.40:0/3422939896 wait complete. 2024-06-06T03:38:09.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.896+0000 7f5274c38700 1 Processor -- start 2024-06-06T03:38:09.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.896+0000 7f5274c38700 1 -- start start 2024-06-06T03:38:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f5274c38700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5270102510 0x7f5270074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f5274c38700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5270102ed0 0x7f5270074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f5274c38700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5270108520 0x7f5270075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f5274c38700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5270112820 con 0x7f5270102ed0 2024-06-06T03:38:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f5274c38700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f52701126a0 con 0x7f5270108520 2024-06-06T03:38:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f5274c38700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f52701129a0 con 0x7f5270102510 2024-06-06T03:38:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f526dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5270102ed0 0x7f5270074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f526dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5270102ed0 0x7f5270074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55944/0 (socket says 172.21.15.40:55944) 2024-06-06T03:38:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f526dd9b700 1 -- 172.21.15.40:0/3515932598 learned_addr learned my addr 172.21.15.40:0/3515932598 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f526ed9d700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5270108520 0x7f5270075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f526e59c700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5270102510 0x7f5270074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f526dd9b700 1 -- 172.21.15.40:0/3515932598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5270102510 msgr2=0x7f5270074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f526dd9b700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5270102510 0x7f5270074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:09.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f526dd9b700 1 -- 172.21.15.40:0/3515932598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5270108520 msgr2=0x7f5270075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:09.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f526dd9b700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5270108520 0x7f5270075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:09.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f526dd9b700 1 -- 172.21.15.40:0/3515932598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52701a70c0 con 0x7f5270102ed0 2024-06-06T03:38:09.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f526e59c700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5270102510 0x7f5270074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:09.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f526dd9b700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5270102ed0 0x7f5270074cd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f526400c9b0 tx=0x7f526400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:09.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.897+0000 7f525f7fe700 1 -- 172.21.15.40:0/3515932598 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264007b00 con 0x7f5270102ed0 2024-06-06T03:38:09.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.898+0000 7f525f7fe700 1 -- 172.21.15.40:0/3515932598 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5264007ca0 con 0x7f5270102ed0 2024-06-06T03:38:09.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.898+0000 7f5274c38700 1 -- 172.21.15.40:0/3515932598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52701a73b0 con 0x7f5270102ed0 2024-06-06T03:38:09.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.898+0000 7f525f7fe700 1 -- 172.21.15.40:0/3515932598 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264018730 con 0x7f5270102ed0 2024-06-06T03:38:09.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.898+0000 7f5274c38700 1 -- 172.21.15.40:0/3515932598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52701a78f0 con 0x7f5270102ed0 2024-06-06T03:38:09.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.899+0000 7f5274c38700 1 -- 172.21.15.40:0/3515932598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5270069180 con 0x7f5270102ed0 2024-06-06T03:38:09.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.900+0000 7f525f7fe700 1 -- 172.21.15.40:0/3515932598 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f52640188d0 con 0x7f5270102ed0 2024-06-06T03:38:09.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.900+0000 7f525f7fe700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f52580a2c20 0x7f52580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:09.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.901+0000 7f526e59c700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f52580a2c20 0x7f52580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:09.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.901+0000 7f525f7fe700 1 -- 172.21.15.40:0/3515932598 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f52640cef90 con 0x7f5270102ed0 2024-06-06T03:38:09.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.901+0000 7f526e59c700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f52580a2c20 0x7f52580a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5260006010 tx=0x7f526000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:09.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:09.905+0000 7f525f7fe700 1 -- 172.21.15.40:0/3515932598 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5264014030 con 0x7f5270102ed0 2024-06-06T03:38:10.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:10.157+0000 7f5274c38700 1 -- 172.21.15.40:0/3515932598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f527002dca0 con 0x7f5270102ed0 2024-06-06T03:38:10.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:10.158+0000 7f525f7fe700 1 -- 172.21.15.40:0/3515932598 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11933 (secure 0 0 0) 0x7f526409bfb0 con 0x7f5270102ed0 2024-06-06T03:38:10.158 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:38:10.158 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:35:38.053505+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24398,24403,24425,24443,24470]},"id":1}]} 2024-06-06T03:38:10.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:10.161+0000 7f5274c38700 1 -- 172.21.15.40:0/3515932598 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f52580a2c20 msgr2=0x7f52580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:10.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:10.161+0000 7f5274c38700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f52580a2c20 0x7f52580a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5260006010 tx=0x7f526000b540 comp rx=0 tx=0).stop 2024-06-06T03:38:10.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:10.161+0000 7f5274c38700 1 -- 172.21.15.40:0/3515932598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5270102ed0 msgr2=0x7f5270074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:10.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:10.161+0000 7f5274c38700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5270102ed0 0x7f5270074cd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f526400c9b0 tx=0x7f526400ccc0 comp rx=0 tx=0).stop 2024-06-06T03:38:10.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:10.161+0000 7f5274c38700 1 -- 172.21.15.40:0/3515932598 shutdown_connections 2024-06-06T03:38:10.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:10.161+0000 7f5274c38700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f52580a2c20 0x7f52580a50e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:10.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:10.161+0000 7f5274c38700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5270102510 0x7f5270074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:10.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:10.161+0000 7f5274c38700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5270108520 0x7f5270075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:10.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:10.161+0000 7f5274c38700 1 --2- 172.21.15.40:0/3515932598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5270102ed0 0x7f5270074cd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:10.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:10.161+0000 7f5274c38700 1 -- 172.21.15.40:0/3515932598 >> 172.21.15.40:0/3515932598 conn(0x7f5270079640 msgr2=0x7f5270100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:10.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:10.161+0000 7f5274c38700 1 -- 172.21.15.40:0/3515932598 shutdown_connections 2024-06-06T03:38:10.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:10.161+0000 7f5274c38700 1 -- 172.21.15.40:0/3515932598 wait complete. 2024-06-06T03:38:10.162 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 28 2024-06-06T03:38:10.880 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:38:11.133 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:11.256 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:10 smithi040 ceph-mon[32220]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:38:11.256 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:10 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3515932598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:38:11.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:10 smithi131 ceph-mon[36687]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:38:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:10 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3515932598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:38:11.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:10 smithi104 ceph-mon[36569]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:38:11.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:10 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3515932598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:38:12.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.962+0000 7f20f5ec1700 1 -- 172.21.15.40:0/935957692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f0103150 msgr2=0x7f20f0107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:12.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.962+0000 7f20f5ec1700 1 --2- 172.21.15.40:0/935957692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f0103150 0x7f20f0107140 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f20d80099c0 tx=0x7f20d8009cd0 comp rx=0 tx=0).stop 2024-06-06T03:38:12.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.962+0000 7f20f5ec1700 1 -- 172.21.15.40:0/935957692 shutdown_connections 2024-06-06T03:38:12.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.962+0000 7f20f5ec1700 1 --2- 172.21.15.40:0/935957692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20f01027a0 0x7f20f0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:12.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.962+0000 7f20f5ec1700 1 --2- 172.21.15.40:0/935957692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20f0107870 0x7f20f010f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:12.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.962+0000 7f20f5ec1700 1 --2- 172.21.15.40:0/935957692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f0103150 0x7f20f0107140 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:12.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.962+0000 7f20f5ec1700 1 -- 172.21.15.40:0/935957692 >> 172.21.15.40:0/935957692 conn(0x7f20f0079570 msgr2=0x7f20f0079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:12.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.962+0000 7f20f5ec1700 1 -- 172.21.15.40:0/935957692 shutdown_connections 2024-06-06T03:38:12.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.962+0000 7f20f5ec1700 1 -- 172.21.15.40:0/935957692 wait complete. 2024-06-06T03:38:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.963+0000 7f20f5ec1700 1 Processor -- start 2024-06-06T03:38:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.963+0000 7f20f5ec1700 1 -- start start 2024-06-06T03:38:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.963+0000 7f20f5ec1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f01027a0 0x7f20f0198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.963+0000 7f20f5ec1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20f0103150 0x7f20f0199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.963+0000 7f20f5ec1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20f0107870 0x7f20f019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.963+0000 7f20f5ec1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0112880 con 0x7f20f01027a0 2024-06-06T03:38:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.963+0000 7f20f5ec1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0112700 con 0x7f20f0103150 2024-06-06T03:38:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.963+0000 7f20f5ec1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0112a00 con 0x7f20f0107870 2024-06-06T03:38:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20ef7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f01027a0 0x7f20f0198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20ef7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f01027a0 0x7f20f0198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55962/0 (socket says 172.21.15.40:55962) 2024-06-06T03:38:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20ef7fe700 1 -- 172.21.15.40:0/338794994 learned_addr learned my addr 172.21.15.40:0/338794994 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20e6ffd700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20f0103150 0x7f20f0199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20effff700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20f0107870 0x7f20f019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20ef7fe700 1 -- 172.21.15.40:0/338794994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20f0107870 msgr2=0x7f20f019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20ef7fe700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20f0107870 0x7f20f019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20ef7fe700 1 -- 172.21.15.40:0/338794994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20f0103150 msgr2=0x7f20f0199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20ef7fe700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20f0103150 0x7f20f0199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20ef7fe700 1 -- 172.21.15.40:0/338794994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f019dd40 con 0x7f20f01027a0 2024-06-06T03:38:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20effff700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20f0107870 0x7f20f019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:38:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20e6ffd700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20f0103150 0x7f20f0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20ef7fe700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f01027a0 0x7f20f0198d30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f20e000b530 tx=0x7f20e000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20ed7fa700 1 -- 172.21.15.40:0/338794994 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0015020 con 0x7f20f01027a0 2024-06-06T03:38:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20f5ec1700 1 -- 172.21.15.40:0/338794994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f019e030 con 0x7f20f01027a0 2024-06-06T03:38:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20ed7fa700 1 -- 172.21.15.40:0/338794994 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e0011d10 con 0x7f20f01027a0 2024-06-06T03:38:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20ed7fa700 1 -- 172.21.15.40:0/338794994 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0010e50 con 0x7f20f01027a0 2024-06-06T03:38:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.964+0000 7f20f5ec1700 1 -- 172.21.15.40:0/338794994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f01a58b0 con 0x7f20f01027a0 2024-06-06T03:38:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.965+0000 7f20f5ec1700 1 -- 172.21.15.40:0/338794994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f010aab0 con 0x7f20f01027a0 2024-06-06T03:38:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.967+0000 7f20ed7fa700 1 -- 172.21.15.40:0/338794994 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f20e0018020 con 0x7f20f01027a0 2024-06-06T03:38:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.967+0000 7f20ed7fa700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f20dc0a2dc0 0x7f20dc0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.967+0000 7f20e6ffd700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f20dc0a2dc0 0x7f20dc0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.967+0000 7f20ed7fa700 1 -- 172.21.15.40:0/338794994 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f20e00ced50 con 0x7f20f01027a0 2024-06-06T03:38:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.968+0000 7f20e6ffd700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f20dc0a2dc0 0x7f20dc0a5280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f20f0100eb0 tx=0x7f20d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:12.971+0000 7f20ed7fa700 1 -- 172.21.15.40:0/338794994 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f20e001f020 con 0x7f20f01027a0 2024-06-06T03:38:13.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:12 smithi040 ceph-mon[32220]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:38:13.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:13.227+0000 7f20f5ec1700 1 -- 172.21.15.40:0/338794994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20f0069180 con 0x7f20f01027a0 2024-06-06T03:38:13.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:13.228+0000 7f20ed7fa700 1 -- 172.21.15.40:0/338794994 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11933 (secure 0 0 0) 0x7f20e009bd70 con 0x7f20f01027a0 2024-06-06T03:38:13.227 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:38:13.228 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:35:38.053505+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24398,24403,24425,24443,24470]},"id":1}]} 2024-06-06T03:38:13.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:13.230+0000 7f20f5ec1700 1 -- 172.21.15.40:0/338794994 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f20dc0a2dc0 msgr2=0x7f20dc0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:13.230+0000 7f20f5ec1700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f20dc0a2dc0 0x7f20dc0a5280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f20f0100eb0 tx=0x7f20d8005fd0 comp rx=0 tx=0).stop 2024-06-06T03:38:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:13.230+0000 7f20f5ec1700 1 -- 172.21.15.40:0/338794994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f01027a0 msgr2=0x7f20f0198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:13.230+0000 7f20f5ec1700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f01027a0 0x7f20f0198d30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f20e000b530 tx=0x7f20e000b8f0 comp rx=0 tx=0).stop 2024-06-06T03:38:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:13.230+0000 7f20f5ec1700 1 -- 172.21.15.40:0/338794994 shutdown_connections 2024-06-06T03:38:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:13.230+0000 7f20f5ec1700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f20dc0a2dc0 0x7f20dc0a5280 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:13.230+0000 7f20f5ec1700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20f0107870 0x7f20f019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:13.230+0000 7f20f5ec1700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f01027a0 0x7f20f0198d30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:13.230+0000 7f20f5ec1700 1 --2- 172.21.15.40:0/338794994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20f0103150 0x7f20f0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:13.230+0000 7f20f5ec1700 1 -- 172.21.15.40:0/338794994 >> 172.21.15.40:0/338794994 conn(0x7f20f0079570 msgr2=0x7f20f0101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:13.230+0000 7f20f5ec1700 1 -- 172.21.15.40:0/338794994 shutdown_connections 2024-06-06T03:38:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:13.230+0000 7f20f5ec1700 1 -- 172.21.15.40:0/338794994 wait complete. 2024-06-06T03:38:13.231 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 28 2024-06-06T03:38:13.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:12 smithi131 ceph-mon[36687]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:38:13.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:12 smithi104 ceph-mon[36569]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:38:13.848 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T03:38:14.098 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:14.221 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:13 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/338794994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:38:14.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:13 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/338794994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:38:14.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:13 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/338794994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:38:15.266 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:14 smithi040 ceph-mon[32220]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:38:15.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:14 smithi131 ceph-mon[36687]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:38:15.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:14 smithi104 ceph-mon[36569]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:38:15.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.802+0000 7fbb1649e700 1 -- 172.21.15.40:0/1069134247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb10107400 msgr2=0x7fbb101077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:15.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.802+0000 7fbb1649e700 1 --2- 172.21.15.40:0/1069134247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb10107400 0x7fbb101077e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009990 tx=0x7fbb00009ca0 comp rx=0 tx=0).stop 2024-06-06T03:38:15.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.802+0000 7fbb1649e700 1 -- 172.21.15.40:0/1069134247 shutdown_connections 2024-06-06T03:38:15.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.802+0000 7fbb1649e700 1 --2- 172.21.15.40:0/1069134247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb10101d50 0x7fbb1010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:15.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.802+0000 7fbb1649e700 1 --2- 172.21.15.40:0/1069134247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb10107400 0x7fbb101077e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.802+0000 7fbb1649e700 1 --2- 172.21.15.40:0/1069134247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb10101390 0x7fbb10101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.802+0000 7fbb1649e700 1 -- 172.21.15.40:0/1069134247 >> 172.21.15.40:0/1069134247 conn(0x7fbb10078340 msgr2=0x7fbb10078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.802+0000 7fbb1649e700 1 -- 172.21.15.40:0/1069134247 shutdown_connections 2024-06-06T03:38:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.803+0000 7fbb1649e700 1 -- 172.21.15.40:0/1069134247 wait complete. 2024-06-06T03:38:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.803+0000 7fbb1649e700 1 Processor -- start 2024-06-06T03:38:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.803+0000 7fbb1649e700 1 -- start start 2024-06-06T03:38:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.803+0000 7fbb1649e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb10101390 0x7fbb1019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.803+0000 7fbb1649e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb10101d50 0x7fbb1019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.803+0000 7fbb1649e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb10107400 0x7fbb10197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.803+0000 7fbb1649e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbb101116a0 con 0x7fbb10101d50 2024-06-06T03:38:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.803+0000 7fbb1649e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbb10111520 con 0x7fbb10101390 2024-06-06T03:38:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.803+0000 7fbb1649e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbb10111820 con 0x7fbb10107400 2024-06-06T03:38:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb07fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb10101d50 0x7fbb1019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb07fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb10101d50 0x7fbb1019dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55990/0 (socket says 172.21.15.40:55990) 2024-06-06T03:38:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb07fff700 1 -- 172.21.15.40:0/754432585 learned_addr learned my addr 172.21.15.40:0/754432585 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb0ffff700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb10101390 0x7fbb1019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb14a3b700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb10107400 0x7fbb10197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb07fff700 1 -- 172.21.15.40:0/754432585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb10107400 msgr2=0x7fbb10197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb07fff700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb10107400 0x7fbb10197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb07fff700 1 -- 172.21.15.40:0/754432585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb10101390 msgr2=0x7fbb1019d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb07fff700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb10101390 0x7fbb1019d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb07fff700 1 -- 172.21.15.40:0/754432585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb101980e0 con 0x7fbb10101d50 2024-06-06T03:38:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb14a3b700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb10107400 0x7fbb10197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb07fff700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb10101d50 0x7fbb1019dd00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800d950 tx=0x7fbaf800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb0dffb700 1 -- 172.21.15.40:0/754432585 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8016070 con 0x7fbb10101d50 2024-06-06T03:38:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb1649e700 1 -- 172.21.15.40:0/754432585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb101983d0 con 0x7fbb10101d50 2024-06-06T03:38:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb0dffb700 1 -- 172.21.15.40:0/754432585 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf8009e20 con 0x7fbb10101d50 2024-06-06T03:38:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb0dffb700 1 -- 172.21.15.40:0/754432585 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf800f620 con 0x7fbb10101d50 2024-06-06T03:38:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.804+0000 7fbb1649e700 1 -- 172.21.15.40:0/754432585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb101a4690 con 0x7fbb10101d50 2024-06-06T03:38:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.805+0000 7fbb1649e700 1 -- 172.21.15.40:0/754432585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb10069180 con 0x7fbb10101d50 2024-06-06T03:38:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.807+0000 7fbb0dffb700 1 -- 172.21.15.40:0/754432585 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbaf8019070 con 0x7fbb10101d50 2024-06-06T03:38:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.808+0000 7fbb0dffb700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbafc0a2c20 0x7fbafc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.808+0000 7fbb0ffff700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbafc0a2c20 0x7fbafc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.808+0000 7fbb0dffb700 1 -- 172.21.15.40:0/754432585 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fbaf80ceb40 con 0x7fbb10101d50 2024-06-06T03:38:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.808+0000 7fbb0ffff700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbafc0a2c20 0x7fbafc0a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb10198ac0 tx=0x7fbb00005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:15.812+0000 7fbb0dffb700 1 -- 172.21.15.40:0/754432585 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbaf809bb60 con 0x7fbb10101d50 2024-06-06T03:38:16.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:16.028+0000 7fbb1649e700 1 -- 172.21.15.40:0/754432585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb1002dca0 con 0x7fbb10101d50 2024-06-06T03:38:16.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:16.030+0000 7fbb0dffb700 1 -- 172.21.15.40:0/754432585 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22534 (secure 0 0 0) 0x7fbaf809b460 con 0x7fbb10101d50 2024-06-06T03:38:16.030 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:38:16.030 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"fsid":"77c25d68-23b4-11ef-bc9b-c7b262605968","created":"2024-06-06T03:26:53.165489+0000","modified":"2024-06-06T03:35:48.980461+0000","last_up_change":"2024-06-06T03:34:07.114448+0000","last_in_change":"2024-06-06T03:33:46.152948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T03:30:14.102785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T03:35:08.521025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T03:35:12.369035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63c0a320-a2f2-4b84-87e7-68501d335285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6803","nonce":3453526793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6805","nonce":3453526793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6809","nonce":3453526793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6807","nonce":3453526793}]},"public_addr":"172.21.15.40:6803/3453526793","cluster_addr":"172.21.15.40:6805/3453526793","heartbeat_back_addr":"172.21.15.40:6809/3453526793","heartbeat_front_addr":"172.21.15.40:6807/3453526793","state":["exists","up"]},{"osd":1,"uuid":"70a31534-06a1-4ffa-a54d-577d45bccf98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6801","nonce":405854252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6803","nonce":405854252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6807","nonce":405854252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6805","nonce":405854252}]},"public_addr":"172.21.15.104:6801/405854252","cluster_addr":"172.21.15.104:6803/405854252","heartbeat_back_addr":"172.21.15.104:6807/405854252","heartbeat_front_addr":"172.21.15.104:6805/405854252","state":["exists","up"]},{"osd":2,"uuid":"f5a89f43-e533-45f0-a777-bd3e9bfa8072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6801","nonce":3919221141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6803","nonce":3919221141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6807","nonce":3919221141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6805","nonce":3919221141}]},"public_addr":"172.21.15.131:6801/3919221141","cluster_addr":"172.21.15.131:6803/3919221141","heartbeat_back_addr":"172.21.15.131:6807/3919221141","heartbeat_front_addr":"172.21.15.131:6805/3919221141","state":["exists","up"]},{"osd":3,"uuid":"4df19390-6378-4346-a2be-29bce1dd2380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6811","nonce":2008541522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6813","nonce":2008541522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6817","nonce":2008541522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6815","nonce":2008541522}]},"public_addr":"172.21.15.40:6811/2008541522","cluster_addr":"172.21.15.40:6813/2008541522","heartbeat_back_addr":"172.21.15.40:6817/2008541522","heartbeat_front_addr":"172.21.15.40:6815/2008541522","state":["exists","up"]},{"osd":4,"uuid":"508de408-1cb1-4423-8199-db6bde9034f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6809","nonce":1153394847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6811","nonce":1153394847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6815","nonce":1153394847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6813","nonce":1153394847}]},"public_addr":"172.21.15.104:6809/1153394847","cluster_addr":"172.21.15.104:6811/1153394847","heartbeat_back_addr":"172.21.15.104:6815/1153394847","heartbeat_front_addr":"172.21.15.104:6813/1153394847","state":["exists","up"]},{"osd":5,"uuid":"a8c0cec9-a8ad-4a5d-800b-8b891ca6d836","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6809","nonce":1051933303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6811","nonce":1051933303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6815","nonce":1051933303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6813","nonce":1051933303}]},"public_addr":"172.21.15.131:6809/1051933303","cluster_addr":"172.21.15.131:6811/1051933303","heartbeat_back_addr":"172.21.15.131:6815/1051933303","heartbeat_front_addr":"172.21.15.131:6813/1051933303","state":["exists","up"]},{"osd":6,"uuid":"5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6819","nonce":2670922011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6821","nonce":2670922011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6825","nonce":2670922011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6823","nonce":2670922011}]},"public_addr":"172.21.15.40:6819/2670922011","cluster_addr":"172.21.15.40:6821/2670922011","heartbeat_back_addr":"172.21.15.40:6825/2670922011","heartbeat_front_addr":"172.21.15.40:6823/2670922011","state":["exists","up"]},{"osd":7,"uuid":"bfc536bb-6ae4-459e-afd9-8197c7420bd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6817","nonce":2353571051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6819","nonce":2353571051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6823","nonce":2353571051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6821","nonce":2353571051}]},"public_addr":"172.21.15.104:6817/2353571051","cluster_addr":"172.21.15.104:6819/2353571051","heartbeat_back_addr":"172.21.15.104:6823/2353571051","heartbeat_front_addr":"172.21.15.104:6821/2353571051","state":["exists","up"]},{"osd":8,"uuid":"1fe214f2-5ea3-4fc5-83bf-d9bccafe1529","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6817","nonce":1364533794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6819","nonce":1364533794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6823","nonce":1364533794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6821","nonce":1364533794}]},"public_addr":"172.21.15.131:6817/1364533794","cluster_addr":"172.21.15.131:6819/1364533794","heartbeat_back_addr":"172.21.15.131:6823/1364533794","heartbeat_front_addr":"172.21.15.131:6821/1364533794","state":["exists","up"]},{"osd":9,"uuid":"4b2b3555-6870-4ead-be50-b635799265a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6827","nonce":2604133190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6829","nonce":2604133190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6833","nonce":2604133190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6831","nonce":2604133190}]},"public_addr":"172.21.15.40:6827/2604133190","cluster_addr":"172.21.15.40:6829/2604133190","heartbeat_back_addr":"172.21.15.40:6833/2604133190","heartbeat_front_addr":"172.21.15.40:6831/2604133190","state":["exists","up"]},{"osd":10,"uuid":"70049e24-94ce-458e-97e3-dbcf9de05444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6825","nonce":2231164832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6827","nonce":2231164832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6831","nonce":2231164832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6829","nonce":2231164832}]},"public_addr":"172.21.15.104:6825/2231164832","cluster_addr":"172.21.15.104:6827/2231164832","heartbeat_back_addr":"172.21.15.104:6831/2231164832","heartbeat_front_addr":"172.21.15.104:6829/2231164832","state":["exists","up"]},{"osd":11,"uuid":"80447548-1fbe-4892-ac13-eb4bb6acf10d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6825","nonce":4128185670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6827","nonce":4128185670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6831","nonce":4128185670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6829","nonce":4128185670}]},"public_addr":"172.21.15.131:6825/4128185670","cluster_addr":"172.21.15.131:6827/4128185670","heartbeat_back_addr":"172.21.15.131:6831/4128185670","heartbeat_front_addr":"172.21.15.131:6829/4128185670","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:23.289784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:47.271677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:10.866792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:36.781924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:01.995848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:26.904606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:52.997438+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:19.221567+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:44.543977+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:11.948545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:38.402943+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:34:04.712200+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/23942578":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2234980855":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6801/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6800/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2301695357":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/1676880727":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6801/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6800/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/1597665640":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/513375601":"2024-06-07T03:27:19.799499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T03:38:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:16.032+0000 7fbb1649e700 1 -- 172.21.15.40:0/754432585 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbafc0a2c20 msgr2=0x7fbafc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:16.032+0000 7fbb1649e700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbafc0a2c20 0x7fbafc0a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb10198ac0 tx=0x7fbb00005960 comp rx=0 tx=0).stop 2024-06-06T03:38:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:16.032+0000 7fbb1649e700 1 -- 172.21.15.40:0/754432585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb10101d50 msgr2=0x7fbb1019dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:16.032+0000 7fbb1649e700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb10101d50 0x7fbb1019dd00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800d950 tx=0x7fbaf800dc60 comp rx=0 tx=0).stop 2024-06-06T03:38:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:16.033+0000 7fbb1649e700 1 -- 172.21.15.40:0/754432585 shutdown_connections 2024-06-06T03:38:16.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:16.033+0000 7fbb1649e700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fbafc0a2c20 0x7fbafc0a50e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:16.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:16.033+0000 7fbb1649e700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb10107400 0x7fbb10197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:16.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:16.033+0000 7fbb1649e700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb10101390 0x7fbb1019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:16.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:16.033+0000 7fbb1649e700 1 --2- 172.21.15.40:0/754432585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb10101d50 0x7fbb1019dd00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:16.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:16.033+0000 7fbb1649e700 1 -- 172.21.15.40:0/754432585 >> 172.21.15.40:0/754432585 conn(0x7fbb10078340 msgr2=0x7fbb10100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:16.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:16.033+0000 7fbb1649e700 1 -- 172.21.15.40:0/754432585 shutdown_connections 2024-06-06T03:38:16.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:16.033+0000 7fbb1649e700 1 -- 172.21.15.40:0/754432585 wait complete. 2024-06-06T03:38:16.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-06T03:35:15.961853+0000', 'modified': '2024-06-06T03:35:38.053505+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24470, 'mds_1': 24443, 'mds_2': 24425, 'mds_3': 24403, 'mds_4': 24398}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14580': {'gid': 14580, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6839/1930069210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6838', 'nonce': 1930069210}, {'type': 'v1', 'addr': '172.21.15.40:6839', 'nonce': 1930069210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14586': {'gid': 14586, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6841/2460301066', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6840', 'nonce': 2460301066}, {'type': 'v1', 'addr': '172.21.15.40:6841', 'nonce': 2460301066}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24358': {'gid': 24358, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6833/1833802668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 1833802668}, {'type': 'v1', 'addr': '172.21.15.131:6833', 'nonce': 1833802668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24376': {'gid': 24376, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/1221528906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 1221528906}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1221528906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24388': {'gid': 24388, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6837/526159320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6836', 'nonce': 526159320}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 526159320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24398': {'gid': 24398, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.104:6833/1414464971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6832', 'nonce': 1414464971}, {'type': 'v1', 'addr': '172.21.15.104:6833', 'nonce': 1414464971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.131:6839/782268782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 782268782}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 782268782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.104:6835/75462915', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 75462915}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 75462915}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.104:6837/2498466212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 2498466212}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 2498466212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.104:6839/489480648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 489480648}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 489480648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24470, 'qdb_cluster': [24398, 24403, 24425, 24443, 24470]} 2024-06-06T03:38:16.631 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-06T03:38:16.631 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.k status 2024-06-06T03:38:16.885 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:17.123 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:16 smithi040 ceph-mon[32220]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:38:17.123 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:16 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/754432585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:38:17.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:16 smithi131 ceph-mon[36687]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:38:17.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:16 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/754432585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:38:17.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:16 smithi104 ceph-mon[36569]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:38:17.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:16 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/754432585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:38:18.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.428+0000 7fda419bd700 1 -- 172.21.15.40:0/2650224480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c1066f0 msgr2=0x7fda3c10e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:18.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.428+0000 7fda419bd700 1 --2- 172.21.15.40:0/2650224480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c1066f0 0x7fda3c10e2c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fda3400b0d0 tx=0x7fda3400b3e0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.428+0000 7fda419bd700 1 -- 172.21.15.40:0/2650224480 shutdown_connections 2024-06-06T03:38:18.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.428+0000 7fda419bd700 1 --2- 172.21.15.40:0/2650224480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3c101610 0x7fda3c1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.428+0000 7fda419bd700 1 --2- 172.21.15.40:0/2650224480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c1066f0 0x7fda3c10e2c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.428+0000 7fda419bd700 1 --2- 172.21.15.40:0/2650224480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda3c101fc0 0x7fda3c106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.428+0000 7fda419bd700 1 -- 172.21.15.40:0/2650224480 >> 172.21.15.40:0/2650224480 conn(0x7fda3c078430 msgr2=0x7fda3c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:18.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.428+0000 7fda419bd700 1 -- 172.21.15.40:0/2650224480 shutdown_connections 2024-06-06T03:38:18.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.428+0000 7fda419bd700 1 -- 172.21.15.40:0/2650224480 wait complete. 2024-06-06T03:38:18.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.429+0000 7fda419bd700 1 Processor -- start 2024-06-06T03:38:18.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.429+0000 7fda419bd700 1 -- start start 2024-06-06T03:38:18.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.429+0000 7fda419bd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c101610 0x7fda3c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda419bd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3c101fc0 0x7fda3c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda419bd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda3c1066f0 0x7fda3c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda419bd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c111700 con 0x7fda3c101610 2024-06-06T03:38:18.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda419bd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c111580 con 0x7fda3c1066f0 2024-06-06T03:38:18.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda419bd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c111880 con 0x7fda3c101fc0 2024-06-06T03:38:18.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda3affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c101610 0x7fda3c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda3affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c101610 0x7fda3c19ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56014/0 (socket says 172.21.15.40:56014) 2024-06-06T03:38:18.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda3affd700 1 -- 172.21.15.40:0/2733663792 learned_addr learned my addr 172.21.15.40:0/2733663792 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:18.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda3a7fc700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3c101fc0 0x7fda3c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda3b7fe700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda3c1066f0 0x7fda3c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda3affd700 1 -- 172.21.15.40:0/2733663792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3c101fc0 msgr2=0x7fda3c19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:18.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda3affd700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3c101fc0 0x7fda3c19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda3affd700 1 -- 172.21.15.40:0/2733663792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda3c1066f0 msgr2=0x7fda3c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:18.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda3affd700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda3c1066f0 0x7fda3c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda3affd700 1 -- 172.21.15.40:0/2733663792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c19fe40 con 0x7fda3c101610 2024-06-06T03:38:18.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda3b7fe700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda3c1066f0 0x7fda3c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:18.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda3affd700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c101610 0x7fda3c19ae30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00d980 tx=0x7fda2c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:18.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda409bb700 1 -- 172.21.15.40:0/2733663792 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c017070 con 0x7fda3c101610 2024-06-06T03:38:18.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda419bd700 1 -- 172.21.15.40:0/2733663792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c1a0130 con 0x7fda3c101610 2024-06-06T03:38:18.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda409bb700 1 -- 172.21.15.40:0/2733663792 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda2c005020 con 0x7fda3c101610 2024-06-06T03:38:18.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda409bb700 1 -- 172.21.15.40:0/2733663792 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c009bc0 con 0x7fda3c101610 2024-06-06T03:38:18.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.430+0000 7fda419bd700 1 -- 172.21.15.40:0/2733663792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c1a79b0 con 0x7fda3c101610 2024-06-06T03:38:18.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.433+0000 7fda409bb700 1 -- 172.21.15.40:0/2733663792 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fda2c0051d0 con 0x7fda3c101610 2024-06-06T03:38:18.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.433+0000 7fda409bb700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fda240a2c20 0x7fda240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.433+0000 7fda3a7fc700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fda240a2c20 0x7fda240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.433+0000 7fda409bb700 1 -- 172.21.15.40:0/2733663792 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fda2c0ceeb0 con 0x7fda3c101610 2024-06-06T03:38:18.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.434+0000 7fda3a7fc700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fda240a2c20 0x7fda240a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fda3c107c00 tx=0x7fda3000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:18.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.434+0000 7fda4c240b80 1 Processor -- start 2024-06-06T03:38:18.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.434+0000 7fda4c240b80 1 -- start start 2024-06-06T03:38:18.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.434+0000 7fda4c240b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc985f10 0x5647dc9862f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.434+0000 7fda4c240b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc986830 0x5647dc980210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.434+0000 7fda4c240b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc980750 0x5647dc980b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.434+0000 7fda4c240b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5647dc872f10 con 0x5647dc985f10 2024-06-06T03:38:18.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.434+0000 7fda4c240b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5647dc8d1800 con 0x5647dc980750 2024-06-06T03:38:18.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.434+0000 7fda4c240b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5647dc839410 con 0x5647dc986830 2024-06-06T03:38:18.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.434+0000 7fda3affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc985f10 0x5647dc9862f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.434+0000 7fda3affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc985f10 0x5647dc9862f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56022/0 (socket says 172.21.15.40:56022) 2024-06-06T03:38:18.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.434+0000 7fda3affd700 1 -- 172.21.15.40:0/1012378796 learned_addr learned my addr 172.21.15.40:0/1012378796 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:18.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.435+0000 7fda3b7fe700 1 --2- 172.21.15.40:0/1012378796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc980750 0x5647dc980b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.435+0000 7fda3a7fc700 1 --2- 172.21.15.40:0/1012378796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc986830 0x5647dc980210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.435+0000 7fda3affd700 1 -- 172.21.15.40:0/1012378796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc986830 msgr2=0x5647dc980210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:18.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.435+0000 7fda3affd700 1 --2- 172.21.15.40:0/1012378796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc986830 0x5647dc980210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.435+0000 7fda3affd700 1 -- 172.21.15.40:0/1012378796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc980750 msgr2=0x5647dc980b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:18.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.435+0000 7fda3affd700 1 --2- 172.21.15.40:0/1012378796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc980750 0x5647dc980b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.435+0000 7fda3affd700 1 -- 172.21.15.40:0/1012378796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647dc820880 con 0x5647dc985f10 2024-06-06T03:38:18.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.435+0000 7fda3a7fc700 1 --2- 172.21.15.40:0/1012378796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc986830 0x5647dc980210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:38:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.435+0000 7fda3b7fe700 1 --2- 172.21.15.40:0/1012378796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc980750 0x5647dc980b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.435+0000 7fda3affd700 1 --2- 172.21.15.40:0/1012378796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc985f10 0x5647dc9862f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fda2c097c30 tx=0x7fda2c09b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2b0c29c1ac1432 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.435+0000 7fda21ffb700 1 -- 172.21.15.40:0/1012378796 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0a4440 con 0x5647dc985f10 2024-06-06T03:38:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.435+0000 7fda21ffb700 1 -- 172.21.15.40:0/1012378796 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda2c099ad0 con 0x5647dc985f10 2024-06-06T03:38:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.435+0000 7fda21ffb700 1 -- 172.21.15.40:0/1012378796 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0a4210 con 0x5647dc985f10 2024-06-06T03:38:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.436+0000 7fda4c240b80 1 -- 172.21.15.40:0/1012378796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc985f10 msgr2=0x5647dc9862f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.436+0000 7fda4c240b80 1 --2- 172.21.15.40:0/1012378796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc985f10 0x5647dc9862f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fda2c097c30 tx=0x7fda2c09b7d0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.436+0000 7fda4c240b80 1 -- 172.21.15.40:0/1012378796 shutdown_connections 2024-06-06T03:38:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.436+0000 7fda4c240b80 1 --2- 172.21.15.40:0/1012378796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc980750 0x5647dc980b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.436+0000 7fda4c240b80 1 --2- 172.21.15.40:0/1012378796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc985f10 0x5647dc9862f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.436+0000 7fda4c240b80 1 --2- 172.21.15.40:0/1012378796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc986830 0x5647dc980210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.436+0000 7fda4c240b80 1 -- 172.21.15.40:0/1012378796 >> 172.21.15.40:0/1012378796 conn(0x5647dc8ad030 msgr2=0x5647dc97b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.436+0000 7fda4c240b80 1 -- 172.21.15.40:0/1012378796 shutdown_connections 2024-06-06T03:38:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.436+0000 7fda4c240b80 1 -- 172.21.15.40:0/1012378796 wait complete. 2024-06-06T03:38:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.436+0000 7fda4c240b80 1 -- start start 2024-06-06T03:38:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.436+0000 7fda4c240b80 1 Processor -- start 2024-06-06T03:38:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda4c240b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc980750 0x5647dc98c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda4c240b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc985f10 0x5647dc989880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda4c240b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc986830 0x5647dc989dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda4c240b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5647dc871f70 con 0x5647dc985f10 2024-06-06T03:38:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda4c240b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5647dc8cd7c0 con 0x5647dc980750 2024-06-06T03:38:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda4c240b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5647dc98d1b0 con 0x5647dc986830 2024-06-06T03:38:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda4c240b80 10 client.0 _ll_register_callbacks cb 0x5647dc867560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:38:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda3a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc985f10 0x5647dc989880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda3a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc985f10 0x5647dc989880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56034/0 (socket says 172.21.15.40:56034) 2024-06-06T03:38:18.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda3a7fc700 1 -- 172.21.15.40:0/1720924434 learned_addr learned my addr 172.21.15.40:0/1720924434 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:18.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda3affd700 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc980750 0x5647dc98c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda3b7fe700 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc986830 0x5647dc989dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda3a7fc700 1 -- 172.21.15.40:0/1720924434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc986830 msgr2=0x5647dc989dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:18.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda3a7fc700 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc986830 0x5647dc989dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda3a7fc700 1 -- 172.21.15.40:0/1720924434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc980750 msgr2=0x5647dc98c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:18.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda3a7fc700 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc980750 0x5647dc98c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda3a7fc700 1 -- 172.21.15.40:0/1720924434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647dc897700 con 0x5647dc985f10 2024-06-06T03:38:18.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda3b7fe700 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc986830 0x5647dc989dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:38:18.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda3affd700 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc980750 0x5647dc98c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:18.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda3a7fc700 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc985f10 0x5647dc989880 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fda30011000 tx=0x7fda300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba25cd2b26f3c67 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:18.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.437+0000 7fda1affd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:38:18.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.438+0000 7fda1affd700 1 -- 172.21.15.40:0/1720924434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30027670 con 0x5647dc985f10 2024-06-06T03:38:18.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.438+0000 7fda1affd700 1 -- 172.21.15.40:0/1720924434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fda30027810 con 0x5647dc985f10 2024-06-06T03:38:18.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.438+0000 7fda4c240b80 1 -- 172.21.15.40:0/1720924434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda2c099ad0 con 0x5647dc985f10 2024-06-06T03:38:18.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.438+0000 7fda1affd700 1 -- 172.21.15.40:0/1720924434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda30027a10 con 0x5647dc985f10 2024-06-06T03:38:18.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.438+0000 7fda4c240b80 10 client.15090 fetch_fsmap learned FSMap version 28 2024-06-06T03:38:18.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.438+0000 7fda4c240b80 1 -- 172.21.15.40:0/1720924434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5647dc9978e0 con 0x5647dc985f10 2024-06-06T03:38:18.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.438+0000 7fda1affd700 1 -- 172.21.15.40:0/1720924434 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda2c099ad0 con 0x5647dc985f10 2024-06-06T03:38:18.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.438+0000 7fda1affd700 1 -- 172.21.15.40:0/1720924434 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fda30031070 con 0x5647dc985f10 2024-06-06T03:38:18.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.438+0000 7fda4c240b80 10 client.15090 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:38:18.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.438+0000 7fda4c240b80 10 client.15090 resolve_mds: resolved name 'k' to daemon mds.k 2024-06-06T03:38:18.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.438+0000 7fda4c240b80 20 client.15090 populate_metadata read hostname 'smithi040' 2024-06-06T03:38:18.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.438+0000 7fda4c240b80 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x5647dc999f50 0x5647dc99b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.438+0000 7fda4c240b80 4 client.15090 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:38:18.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.438+0000 7fda4c240b80 1 -- 172.21.15.40:0/1720924434 --> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647dc8ded00 con 0x5647dc999f50 2024-06-06T03:38:18.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.439+0000 7fda3affd700 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x5647dc999f50 0x5647dc99b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.439+0000 7fda3affd700 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x5647dc999f50 0x5647dc99b3c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=df8f3106657df52f server_cookie=f1b58beac5177d00 in_seq=0 out_seq=0 2024-06-06T03:38:18.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.439+0000 7fda1affd700 10 client.15090 ms_handle_connect on v2:172.21.15.104:6832/1414464971 2024-06-06T03:38:18.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda1affd700 1 -- 172.21.15.40:0/1720924434 <== mds.4 v2:172.21.15.104:6832/1414464971 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5647dc8ded00 con 0x5647dc999f50 2024-06-06T03:38:18.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda1affd700 10 client.15090 handle_command_reply: tid=0 2024-06-06T03:38:18.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda4c240b80 1 client.15090 shutdown 2024-06-06T03:38:18.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda4c240b80 1 -- 172.21.15.40:0/1720924434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc985f10 msgr2=0x5647dc989880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:18.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda4c240b80 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc985f10 0x5647dc989880 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fda30011000 tx=0x7fda300047c0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda4c240b80 1 -- 172.21.15.40:0/1720924434 shutdown_connections 2024-06-06T03:38:18.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda4c240b80 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x5647dc999f50 0x5647dc99b3c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda4c240b80 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc986830 0x5647dc989dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda4c240b80 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc980750 0x5647dc98c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda1affd700 0 client.15090 ms_handle_reset on v2:172.21.15.104:6832/1414464971 2024-06-06T03:38:18.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda4c240b80 1 --2- 172.21.15.40:0/1720924434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc985f10 0x5647dc989880 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda4c240b80 1 -- 172.21.15.40:0/1720924434 >> 172.21.15.40:0/1720924434 conn(0x5647dc8ad030 msgr2=0x5647dc97a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:18.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda4c240b80 1 -- 172.21.15.40:0/1720924434 shutdown_connections 2024-06-06T03:38:18.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda4c240b80 1 -- 172.21.15.40:0/1720924434 wait complete. 2024-06-06T03:38:18.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.440+0000 7fda4c240b80 20 client.15090 trim_cache size 0 max 16384 2024-06-06T03:38:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.461+0000 7fda4c240b80 1 Processor -- start 2024-06-06T03:38:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.461+0000 7fda4c240b80 1 -- start start 2024-06-06T03:38:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda4c240b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc980750 0x5647dc97ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda4c240b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc985f10 0x5647dc97b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda4c240b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc986830 0x5647dc986cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda4c240b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5647dc999700 con 0x5647dc986830 2024-06-06T03:38:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda4c240b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5647dc999dd0 con 0x5647dc985f10 2024-06-06T03:38:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda4c240b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5647dc99a4a0 con 0x5647dc980750 2024-06-06T03:38:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda3b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc986830 0x5647dc986cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda3b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc986830 0x5647dc986cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56048/0 (socket says 172.21.15.40:56048) 2024-06-06T03:38:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda3b7fe700 1 -- 172.21.15.40:0/245278389 learned_addr learned my addr 172.21.15.40:0/245278389 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda3a7fc700 1 --2- 172.21.15.40:0/245278389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc985f10 0x5647dc97b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda3affd700 1 --2- 172.21.15.40:0/245278389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc980750 0x5647dc97ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda3b7fe700 1 -- 172.21.15.40:0/245278389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc980750 msgr2=0x5647dc97ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda3b7fe700 1 --2- 172.21.15.40:0/245278389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc980750 0x5647dc97ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda3b7fe700 1 -- 172.21.15.40:0/245278389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc985f10 msgr2=0x5647dc97b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda3b7fe700 1 --2- 172.21.15.40:0/245278389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc985f10 0x5647dc97b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda3b7fe700 1 -- 172.21.15.40:0/245278389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647dc99a990 con 0x5647dc986830 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda3a7fc700 1 --2- 172.21.15.40:0/245278389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc985f10 0x5647dc97b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda3b7fe700 1 --2- 172.21.15.40:0/245278389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc986830 0x5647dc986cd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fda34009150 tx=0x7fda34009180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f040131c3c5b264 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.462+0000 7fda217fa700 1 -- 172.21.15.40:0/245278389 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34003c00 con 0x5647dc986830 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.463+0000 7fda217fa700 1 -- 172.21.15.40:0/245278389 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda34009450 con 0x5647dc986830 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.463+0000 7fda217fa700 1 -- 172.21.15.40:0/245278389 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34005420 con 0x5647dc986830 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.463+0000 7fda4c240b80 1 -- 172.21.15.40:0/245278389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc986830 msgr2=0x5647dc986cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.463+0000 7fda4c240b80 1 --2- 172.21.15.40:0/245278389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc986830 0x5647dc986cd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fda34009150 tx=0x7fda34009180 comp rx=0 tx=0).stop 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.463+0000 7fda4c240b80 1 -- 172.21.15.40:0/245278389 shutdown_connections 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.463+0000 7fda4c240b80 1 --2- 172.21.15.40:0/245278389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc980750 0x5647dc97ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.463+0000 7fda4c240b80 1 --2- 172.21.15.40:0/245278389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc986830 0x5647dc986cd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.463+0000 7fda4c240b80 1 --2- 172.21.15.40:0/245278389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc985f10 0x5647dc97b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.463+0000 7fda4c240b80 1 -- 172.21.15.40:0/245278389 >> 172.21.15.40:0/245278389 conn(0x5647dc8ad030 msgr2=0x5647dc988190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.463+0000 7fda4c240b80 1 -- 172.21.15.40:0/245278389 shutdown_connections 2024-06-06T03:38:18.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.463+0000 7fda4c240b80 1 -- 172.21.15.40:0/245278389 wait complete. 2024-06-06T03:38:18.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.464+0000 7fda4c240b80 1 -- start start 2024-06-06T03:38:18.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.464+0000 7fda4c240b80 1 Processor -- start 2024-06-06T03:38:18.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.464+0000 7fda4c240b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc980750 0x5647dc9a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.464+0000 7fda4c240b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc985f10 0x5647dc97a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.464+0000 7fda4c240b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc986830 0x5647dc97a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.464+0000 7fda4c240b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5647dc99b240 con 0x5647dc986830 2024-06-06T03:38:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.464+0000 7fda4c240b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5647dc99ab70 con 0x5647dc985f10 2024-06-06T03:38:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.464+0000 7fda4c240b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5647dc99b910 con 0x5647dc980750 2024-06-06T03:38:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.464+0000 7fda4c240b80 10 client.0 _ll_register_callbacks cb 0x5647dc867560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:38:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.464+0000 7fda3b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc986830 0x5647dc97a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.464+0000 7fda3b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc986830 0x5647dc97a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56056/0 (socket says 172.21.15.40:56056) 2024-06-06T03:38:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.464+0000 7fda3b7fe700 1 -- 172.21.15.40:0/842282544 learned_addr learned my addr 172.21.15.40:0/842282544 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.464+0000 7fda3affd700 1 --2- 172.21.15.40:0/842282544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc980750 0x5647dc9a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.464+0000 7fda3a7fc700 1 --2- 172.21.15.40:0/842282544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc985f10 0x5647dc97a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.465+0000 7fda3b7fe700 1 -- 172.21.15.40:0/842282544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc980750 msgr2=0x5647dc9a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.465+0000 7fda3b7fe700 1 --2- 172.21.15.40:0/842282544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc980750 0x5647dc9a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.465+0000 7fda3b7fe700 1 -- 172.21.15.40:0/842282544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc985f10 msgr2=0x5647dc97a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.465+0000 7fda3b7fe700 1 --2- 172.21.15.40:0/842282544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc985f10 0x5647dc97a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.465+0000 7fda3b7fe700 1 -- 172.21.15.40:0/842282544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647dc97ad00 con 0x5647dc986830 2024-06-06T03:38:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.465+0000 7fda3b7fe700 1 --2- 172.21.15.40:0/842282544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc986830 0x5647dc97a740 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fda3400b820 tx=0x7fda34006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896a6e79af4e741e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.465+0000 7fda1affd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:38:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.465+0000 7fda1affd700 1 -- 172.21.15.40:0/842282544 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34003e10 con 0x5647dc986830 2024-06-06T03:38:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.465+0000 7fda1affd700 1 -- 172.21.15.40:0/842282544 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fda3402a070 con 0x5647dc986830 2024-06-06T03:38:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.465+0000 7fda4c240b80 1 -- 172.21.15.40:0/842282544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda34009450 con 0x5647dc986830 2024-06-06T03:38:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.465+0000 7fda1affd700 1 -- 172.21.15.40:0/842282544 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda340074a0 con 0x5647dc986830 2024-06-06T03:38:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.465+0000 7fda4c240b80 10 client.15096 fetch_fsmap learned FSMap version 28 2024-06-06T03:38:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.465+0000 7fda4c240b80 1 -- 172.21.15.40:0/842282544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5647dc9a3640 con 0x5647dc986830 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.465+0000 7fda1affd700 1 -- 172.21.15.40:0/842282544 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda34009450 con 0x5647dc986830 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.466+0000 7fda1affd700 1 -- 172.21.15.40:0/842282544 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fda34011070 con 0x5647dc986830 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.466+0000 7fda4c240b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.466+0000 7fda4c240b80 10 client.15096 resolve_mds: resolved name 'k' to daemon mds.k 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.466+0000 7fda4c240b80 20 client.15096 populate_metadata read hostname 'smithi040' 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.466+0000 7fda4c240b80 1 --2- 172.21.15.40:0/842282544 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x5647dca09c70 0x5647dca08980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.466+0000 7fda4c240b80 4 client.15096 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.466+0000 7fda4c240b80 1 -- 172.21.15.40:0/842282544 --> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] -- command(tid 0: {"prefix": "status"}) -- 0x5647dc8d9280 con 0x5647dca09c70 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.466+0000 7fda3affd700 1 --2- 172.21.15.40:0/842282544 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x5647dca09c70 0x5647dca08980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.466+0000 7fda3affd700 1 --2- 172.21.15.40:0/842282544 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x5647dca09c70 0x5647dca08980 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6cd9cd7beb1d91be server_cookie=4c77b5c35c9eea0c in_seq=0 out_seq=0 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.466+0000 7fda1affd700 10 client.15096 ms_handle_connect on v2:172.21.15.104:6832/1414464971 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda1affd700 1 -- 172.21.15.40:0/842282544 <== mds.4 v2:172.21.15.104:6832/1414464971 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5647dc8d9280 con 0x5647dca09c70 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda1affd700 10 client.15096 handle_command_reply: tid=0 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda4c240b80 1 client.15096 shutdown 2024-06-06T03:38:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda4c240b80 1 -- 172.21.15.40:0/842282544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc986830 msgr2=0x5647dc97a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda4c240b80 1 --2- 172.21.15.40:0/842282544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc986830 0x5647dc97a740 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fda3400b820 tx=0x7fda34006210 comp rx=0 tx=0).stop 2024-06-06T03:38:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda4c240b80 1 -- 172.21.15.40:0/842282544 shutdown_connections 2024-06-06T03:38:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda4c240b80 1 --2- 172.21.15.40:0/842282544 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x5647dca09c70 0x5647dca08980 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda4c240b80 1 --2- 172.21.15.40:0/842282544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647dc980750 0x5647dc9a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda4c240b80 1 --2- 172.21.15.40:0/842282544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5647dc986830 0x5647dc97a740 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda4c240b80 1 --2- 172.21.15.40:0/842282544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647dc985f10 0x5647dc97a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda1affd700 0 client.15096 ms_handle_reset on v2:172.21.15.104:6832/1414464971 2024-06-06T03:38:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda4c240b80 1 -- 172.21.15.40:0/842282544 >> 172.21.15.40:0/842282544 conn(0x5647dc8ad030 msgr2=0x5647dc987d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda4c240b80 1 -- 172.21.15.40:0/842282544 shutdown_connections 2024-06-06T03:38:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda4c240b80 1 -- 172.21.15.40:0/842282544 wait complete. 2024-06-06T03:38:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.467+0000 7fda4c240b80 20 client.15096 trim_cache size 0 max 16384 2024-06-06T03:38:18.473 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:38:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:38:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 4, 2024-06-06T03:38:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24398, 2024-06-06T03:38:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:38:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:38:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:38:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 164.53818061199999, 2024-06-06T03:38:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 28, 2024-06-06T03:38:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 90, 2024-06-06T03:38:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 88, 2024-06-06T03:38:18.473 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 232.26974527300001 2024-06-06T03:38:18.473 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:38:18.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.468+0000 7fda419bd700 1 -- 172.21.15.40:0/2733663792 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fda240a2c20 msgr2=0x7fda240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:18.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.468+0000 7fda419bd700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fda240a2c20 0x7fda240a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fda3c107c00 tx=0x7fda3000b410 comp rx=0 tx=0).stop 2024-06-06T03:38:18.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.468+0000 7fda419bd700 1 -- 172.21.15.40:0/2733663792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c101610 msgr2=0x7fda3c19ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:18.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.468+0000 7fda419bd700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c101610 0x7fda3c19ae30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00d980 tx=0x7fda2c00dc90 comp rx=0 tx=0).stop 2024-06-06T03:38:18.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.468+0000 7fda419bd700 1 -- 172.21.15.40:0/2733663792 shutdown_connections 2024-06-06T03:38:18.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.468+0000 7fda419bd700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fda240a2c20 0x7fda240a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.468+0000 7fda419bd700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda3c1066f0 0x7fda3c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.468+0000 7fda419bd700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c101610 0x7fda3c19ae30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.468+0000 7fda419bd700 1 --2- 172.21.15.40:0/2733663792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3c101fc0 0x7fda3c19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:18.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.468+0000 7fda419bd700 1 -- 172.21.15.40:0/2733663792 >> 172.21.15.40:0/2733663792 conn(0x7fda3c078430 msgr2=0x7fda3c10d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:18.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.468+0000 7fda419bd700 1 -- 172.21.15.40:0/2733663792 shutdown_connections 2024-06-06T03:38:18.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:18.468+0000 7fda419bd700 1 -- 172.21.15.40:0/2733663792 wait complete. 2024-06-06T03:38:19.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:18 smithi040 ceph-mon[32220]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:38:19.241 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-06T03:38:19.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:18 smithi131 ceph-mon[36687]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:38:19.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:18 smithi104 ceph-mon[36569]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:38:19.500 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:21.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.052+0000 7fe71940f700 1 -- 172.21.15.40:0/1081209984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714101d40 msgr2=0x7fe71410e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:21.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.052+0000 7fe71940f700 1 --2- 172.21.15.40:0/1081209984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714101d40 0x7fe71410e200 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00b0a0 tx=0x7fe70c00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.052+0000 7fe71940f700 1 -- 172.21.15.40:0/1081209984 shutdown_connections 2024-06-06T03:38:21.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.052+0000 7fe71940f700 1 --2- 172.21.15.40:0/1081209984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7141073c0 0x7fe7141077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.052+0000 7fe71940f700 1 --2- 172.21.15.40:0/1081209984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714101d40 0x7fe71410e200 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.052+0000 7fe71940f700 1 --2- 172.21.15.40:0/1081209984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe714101380 0x7fe714101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.052+0000 7fe71940f700 1 -- 172.21.15.40:0/1081209984 >> 172.21.15.40:0/1081209984 conn(0x7fe714078330 msgr2=0x7fe714078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:21.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.052+0000 7fe71940f700 1 -- 172.21.15.40:0/1081209984 shutdown_connections 2024-06-06T03:38:21.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.052+0000 7fe71940f700 1 -- 172.21.15.40:0/1081209984 wait complete. 2024-06-06T03:38:21.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.053+0000 7fe71940f700 1 Processor -- start 2024-06-06T03:38:21.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.053+0000 7fe71940f700 1 -- start start 2024-06-06T03:38:21.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.053+0000 7fe71940f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714101380 0x7fe714103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.053+0000 7fe71940f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714101d40 0x7fe714103570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.053+0000 7fe71940f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7141073c0 0x7fe714103ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe71940f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe714111630 con 0x7fe714101380 2024-06-06T03:38:21.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe71940f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141114b0 con 0x7fe7141073c0 2024-06-06T03:38:21.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe71940f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141117b0 con 0x7fe714101d40 2024-06-06T03:38:21.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe70a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714101d40 0x7fe714103570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe7137fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7141073c0 0x7fe714103ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe70a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714101d40 0x7fe714103570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.40:43430/0 (socket says 172.21.15.40:43430) 2024-06-06T03:38:21.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe7137fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7141073c0 0x7fe714103ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:40784/0 (socket says 172.21.15.40:40784) 2024-06-06T03:38:21.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe70a7fc700 1 -- 172.21.15.40:0/4266463506 learned_addr learned my addr 172.21.15.40:0/4266463506 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:21.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe712ffd700 1 --2- 172.21.15.40:0/4266463506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714101380 0x7fe714103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe70a7fc700 1 -- 172.21.15.40:0/4266463506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7141073c0 msgr2=0x7fe714103ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:21.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe70a7fc700 1 --2- 172.21.15.40:0/4266463506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7141073c0 0x7fe714103ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe70a7fc700 1 -- 172.21.15.40:0/4266463506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714101380 msgr2=0x7fe714103030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:21.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe70a7fc700 1 --2- 172.21.15.40:0/4266463506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714101380 0x7fe714103030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe70a7fc700 1 -- 172.21.15.40:0/4266463506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7141a7080 con 0x7fe714101d40 2024-06-06T03:38:21.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe7137fe700 1 --2- 172.21.15.40:0/4266463506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7141073c0 0x7fe714103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:21.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe70a7fc700 1 --2- 172.21.15.40:0/4266463506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714101d40 0x7fe714103570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00e9a0 tx=0x7fe6fc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:21.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe710ff9700 1 -- 172.21.15.40:0/4266463506 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc00ca10 con 0x7fe714101d40 2024-06-06T03:38:21.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.054+0000 7fe710ff9700 1 -- 172.21.15.40:0/4266463506 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6fc00cbb0 con 0x7fe714101d40 2024-06-06T03:38:21.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.055+0000 7fe71940f700 1 -- 172.21.15.40:0/4266463506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7141a7370 con 0x7fe714101d40 2024-06-06T03:38:21.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.055+0000 7fe710ff9700 1 -- 172.21.15.40:0/4266463506 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc018680 con 0x7fe714101d40 2024-06-06T03:38:21.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.055+0000 7fe71940f700 1 -- 172.21.15.40:0/4266463506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7141a7900 con 0x7fe714101d40 2024-06-06T03:38:21.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.057+0000 7fe710ff9700 1 -- 172.21.15.40:0/4266463506 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe6fc010430 con 0x7fe714101d40 2024-06-06T03:38:21.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.058+0000 7fe710ff9700 1 --2- 172.21.15.40:0/4266463506 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe7000a2ca0 0x7fe7000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.058+0000 7fe710ff9700 1 -- 172.21.15.40:0/4266463506 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fe6fc0cf1c0 con 0x7fe714101d40 2024-06-06T03:38:21.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.058+0000 7fe712ffd700 1 --2- 172.21.15.40:0/4266463506 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe7000a2ca0 0x7fe7000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.058+0000 7fe712ffd700 1 --2- 172.21.15.40:0/4266463506 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe7000a2ca0 0x7fe7000a5160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe704009680 tx=0x7fe704009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:21.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.059+0000 7fe723c92b80 1 Processor -- start 2024-06-06T03:38:21.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.059+0000 7fe723c92b80 1 -- start start 2024-06-06T03:38:21.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.059+0000 7fe723c92b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc6f10 0x55fea2dc72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.059+0000 7fe723c92b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc7830 0x55fea2dc1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.059+0000 7fe723c92b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2dc1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.059+0000 7fe723c92b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fea2cb3f10 con 0x55fea2dc6f10 2024-06-06T03:38:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.059+0000 7fe723c92b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fea2d12800 con 0x55fea2dc7830 2024-06-06T03:38:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.059+0000 7fe723c92b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fea2c7a410 con 0x55fea2dc1750 2024-06-06T03:38:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.059+0000 7fe712ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc6f10 0x55fea2dc72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.059+0000 7fe712ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc6f10 0x55fea2dc72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49674/0 (socket says 172.21.15.40:49674) 2024-06-06T03:38:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.059+0000 7fe712ffd700 1 -- 172.21.15.40:0/466553780 learned_addr learned my addr 172.21.15.40:0/466553780 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.059+0000 7fe70a7fc700 1 --2- 172.21.15.40:0/466553780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc7830 0x55fea2dc1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.059+0000 7fe7137fe700 1 --2- 172.21.15.40:0/466553780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2dc1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.060+0000 7fe712ffd700 1 -- 172.21.15.40:0/466553780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 msgr2=0x55fea2dc1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.060+0000 7fe712ffd700 1 --2- 172.21.15.40:0/466553780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2dc1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.060+0000 7fe712ffd700 1 -- 172.21.15.40:0/466553780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc7830 msgr2=0x55fea2dc1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:21.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.060+0000 7fe712ffd700 1 --2- 172.21.15.40:0/466553780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc7830 0x55fea2dc1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.060+0000 7fe712ffd700 1 -- 172.21.15.40:0/466553780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea2c61880 con 0x55fea2dc6f10 2024-06-06T03:38:21.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.060+0000 7fe70a7fc700 1 --2- 172.21.15.40:0/466553780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc7830 0x55fea2dc1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:21.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.060+0000 7fe712ffd700 1 --2- 172.21.15.40:0/466553780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc6f10 0x55fea2dc72f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe704011d60 tx=0x7fe704014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4307d6f26343434e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:21.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.060+0000 7fe7097fa700 1 -- 172.21.15.40:0/466553780 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704011d90 con 0x55fea2dc6f10 2024-06-06T03:38:21.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.060+0000 7fe7097fa700 1 -- 172.21.15.40:0/466553780 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe704004020 con 0x55fea2dc6f10 2024-06-06T03:38:21.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.060+0000 7fe7097fa700 1 -- 172.21.15.40:0/466553780 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704003d40 con 0x55fea2dc6f10 2024-06-06T03:38:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.061+0000 7fe723c92b80 1 -- 172.21.15.40:0/466553780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc6f10 msgr2=0x55fea2dc72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.061+0000 7fe723c92b80 1 --2- 172.21.15.40:0/466553780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc6f10 0x55fea2dc72f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe704011d60 tx=0x7fe704014000 comp rx=0 tx=0).stop 2024-06-06T03:38:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.061+0000 7fe723c92b80 1 -- 172.21.15.40:0/466553780 shutdown_connections 2024-06-06T03:38:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.061+0000 7fe723c92b80 1 --2- 172.21.15.40:0/466553780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2dc1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.061+0000 7fe723c92b80 1 --2- 172.21.15.40:0/466553780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc6f10 0x55fea2dc72f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.061+0000 7fe723c92b80 1 --2- 172.21.15.40:0/466553780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc7830 0x55fea2dc1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.061+0000 7fe723c92b80 1 -- 172.21.15.40:0/466553780 >> 172.21.15.40:0/466553780 conn(0x55fea2cee030 msgr2=0x55fea2dbc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.061+0000 7fe723c92b80 1 -- 172.21.15.40:0/466553780 shutdown_connections 2024-06-06T03:38:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.061+0000 7fe723c92b80 1 -- 172.21.15.40:0/466553780 wait complete. 2024-06-06T03:38:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.061+0000 7fe723c92b80 1 -- start start 2024-06-06T03:38:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.061+0000 7fe723c92b80 1 Processor -- start 2024-06-06T03:38:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe723c92b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2dcd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe723c92b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 0x55fea2dca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe723c92b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dcad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe723c92b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fea2cb2f70 con 0x55fea2dc7830 2024-06-06T03:38:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe723c92b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fea2d0e7c0 con 0x55fea2dc6f10 2024-06-06T03:38:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe723c92b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fea2dce210 con 0x55fea2dc1750 2024-06-06T03:38:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe723c92b80 10 client.0 _ll_register_callbacks cb 0x55fea2ca8560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:38:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe7137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dcad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe712ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2dcd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe7137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dcad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49680/0 (socket says 172.21.15.40:49680) 2024-06-06T03:38:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe7137fe700 1 -- 172.21.15.40:0/3703504607 learned_addr learned my addr 172.21.15.40:0/3703504607 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe70a7fc700 1 --2- 172.21.15.40:0/3703504607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 0x55fea2dca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe7137fe700 1 -- 172.21.15.40:0/3703504607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 msgr2=0x55fea2dcd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:21.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe7137fe700 1 --2- 172.21.15.40:0/3703504607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2dcd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe7137fe700 1 -- 172.21.15.40:0/3703504607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 msgr2=0x55fea2dca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:21.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe7137fe700 1 --2- 172.21.15.40:0/3703504607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 0x55fea2dca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe7137fe700 1 -- 172.21.15.40:0/3703504607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea2cd8700 con 0x55fea2dc7830 2024-06-06T03:38:21.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.062+0000 7fe70a7fc700 1 --2- 172.21.15.40:0/3703504607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 0x55fea2dca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:21.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.063+0000 7fe7137fe700 1 --2- 172.21.15.40:0/3703504607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dcad90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe70c003c60 tx=0x7fe70c003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98680e90cd8ad0ca server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:21.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.063+0000 7fe6fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:38:21.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.063+0000 7fe6fa7fc700 1 -- 172.21.15.40:0/3703504607 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0078f0 con 0x55fea2dc7830 2024-06-06T03:38:21.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.063+0000 7fe6fa7fc700 1 -- 172.21.15.40:0/3703504607 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fe70c007a90 con 0x55fea2dc7830 2024-06-06T03:38:21.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.063+0000 7fe723c92b80 1 -- 172.21.15.40:0/3703504607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe704004020 con 0x55fea2dc7830 2024-06-06T03:38:21.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.063+0000 7fe6fa7fc700 1 -- 172.21.15.40:0/3703504607 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe70c02b030 con 0x55fea2dc7830 2024-06-06T03:38:21.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.063+0000 7fe723c92b80 10 client.15108 fetch_fsmap learned FSMap version 28 2024-06-06T03:38:21.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.063+0000 7fe723c92b80 1 -- 172.21.15.40:0/3703504607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fea2dd8940 con 0x55fea2dc7830 2024-06-06T03:38:21.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.063+0000 7fe6fa7fc700 1 -- 172.21.15.40:0/3703504607 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe704004020 con 0x55fea2dc7830 2024-06-06T03:38:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.063+0000 7fe6fa7fc700 1 -- 172.21.15.40:0/3703504607 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe70c023070 con 0x55fea2dc7830 2024-06-06T03:38:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.063+0000 7fe723c92b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:38:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.063+0000 7fe723c92b80 10 client.15108 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T03:38:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.064+0000 7fe723c92b80 20 client.15108 populate_metadata read hostname 'smithi040' 2024-06-06T03:38:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.064+0000 7fe723c92b80 1 --2- 172.21.15.40:0/3703504607 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x55fea2ddaf70 0x55fea2ddc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.064+0000 7fe723c92b80 4 client.15108 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:38:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.064+0000 7fe723c92b80 1 -- 172.21.15.40:0/3703504607 --> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fea2d1fd00 con 0x55fea2ddaf70 2024-06-06T03:38:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.064+0000 7fe712ffd700 1 --2- 172.21.15.40:0/3703504607 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x55fea2ddaf70 0x55fea2ddc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.064+0000 7fe712ffd700 1 --2- 172.21.15.40:0/3703504607 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x55fea2ddaf70 0x55fea2ddc3e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=761b0fc2fce518c2 server_cookie=5a4241fcfeeced50 in_seq=0 out_seq=0 2024-06-06T03:38:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.064+0000 7fe6fa7fc700 10 client.15108 ms_handle_connect on v2:172.21.15.131:6838/782268782 2024-06-06T03:38:21.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe6fa7fc700 1 -- 172.21.15.40:0/3703504607 <== mds.3 v2:172.21.15.131:6838/782268782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fea2d1fd00 con 0x55fea2ddaf70 2024-06-06T03:38:21.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe6fa7fc700 10 client.15108 handle_command_reply: tid=0 2024-06-06T03:38:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe723c92b80 1 client.15108 shutdown 2024-06-06T03:38:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe723c92b80 1 -- 172.21.15.40:0/3703504607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 msgr2=0x55fea2dcad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe723c92b80 1 --2- 172.21.15.40:0/3703504607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dcad90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe70c003c60 tx=0x7fe70c003c90 comp rx=0 tx=0).stop 2024-06-06T03:38:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe723c92b80 1 -- 172.21.15.40:0/3703504607 shutdown_connections 2024-06-06T03:38:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe723c92b80 1 --2- 172.21.15.40:0/3703504607 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x55fea2ddaf70 0x55fea2ddc3e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe723c92b80 1 --2- 172.21.15.40:0/3703504607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2dcd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe723c92b80 1 --2- 172.21.15.40:0/3703504607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dcad90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe723c92b80 1 --2- 172.21.15.40:0/3703504607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 0x55fea2dca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe723c92b80 1 -- 172.21.15.40:0/3703504607 >> 172.21.15.40:0/3703504607 conn(0x55fea2cee030 msgr2=0x55fea2dbb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe6fa7fc700 0 client.15108 ms_handle_reset on v2:172.21.15.131:6838/782268782 2024-06-06T03:38:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe723c92b80 1 -- 172.21.15.40:0/3703504607 shutdown_connections 2024-06-06T03:38:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe723c92b80 1 -- 172.21.15.40:0/3703504607 wait complete. 2024-06-06T03:38:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.065+0000 7fe723c92b80 20 client.15108 trim_cache size 0 max 16384 2024-06-06T03:38:21.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:20 smithi040 ceph-mon[32220]: pgmap v396: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:38:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:20 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:38:21.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.090+0000 7fe723c92b80 1 Processor -- start 2024-06-06T03:38:21.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.090+0000 7fe723c92b80 1 -- start start 2024-06-06T03:38:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe723c92b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2dc1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe723c92b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 0x55fea2dbbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe723c92b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dbc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe723c92b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fea2dda720 con 0x55fea2dc7830 2024-06-06T03:38:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe723c92b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fea2ddadf0 con 0x55fea2dc6f10 2024-06-06T03:38:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe723c92b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fea2ddb4c0 con 0x55fea2dc1750 2024-06-06T03:38:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe7137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dbc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe7137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dbc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49692/0 (socket says 172.21.15.40:49692) 2024-06-06T03:38:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe7137fe700 1 -- 172.21.15.40:0/1574835137 learned_addr learned my addr 172.21.15.40:0/1574835137 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe712ffd700 1 --2- 172.21.15.40:0/1574835137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2dc1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe70a7fc700 1 --2- 172.21.15.40:0/1574835137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 0x55fea2dbbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe7137fe700 1 -- 172.21.15.40:0/1574835137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 msgr2=0x55fea2dc1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe7137fe700 1 --2- 172.21.15.40:0/1574835137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2dc1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe7137fe700 1 -- 172.21.15.40:0/1574835137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 msgr2=0x55fea2dbbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe7137fe700 1 --2- 172.21.15.40:0/1574835137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 0x55fea2dbbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe7137fe700 1 -- 172.21.15.40:0/1574835137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea2dce210 con 0x55fea2dc7830 2024-06-06T03:38:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.091+0000 7fe712ffd700 1 --2- 172.21.15.40:0/1574835137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2dc1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.092+0000 7fe7137fe700 1 --2- 172.21.15.40:0/1574835137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dbc370 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009610 tx=0x7fe70c00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648fc591a3aa3ff4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.092+0000 7fe708ff9700 1 -- 172.21.15.40:0/1574835137 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c005c60 con 0x55fea2dc7830 2024-06-06T03:38:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.092+0000 7fe708ff9700 1 -- 172.21.15.40:0/1574835137 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70c005e00 con 0x55fea2dc7830 2024-06-06T03:38:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.092+0000 7fe708ff9700 1 -- 172.21.15.40:0/1574835137 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c025440 con 0x55fea2dc7830 2024-06-06T03:38:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.093+0000 7fe723c92b80 1 -- 172.21.15.40:0/1574835137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 msgr2=0x55fea2dbc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.093+0000 7fe723c92b80 1 --2- 172.21.15.40:0/1574835137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dbc370 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009610 tx=0x7fe70c00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.093+0000 7fe723c92b80 1 -- 172.21.15.40:0/1574835137 shutdown_connections 2024-06-06T03:38:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.093+0000 7fe723c92b80 1 --2- 172.21.15.40:0/1574835137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2dc1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.093+0000 7fe723c92b80 1 --2- 172.21.15.40:0/1574835137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dbc370 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.093+0000 7fe723c92b80 1 --2- 172.21.15.40:0/1574835137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 0x55fea2dbbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.093+0000 7fe723c92b80 1 -- 172.21.15.40:0/1574835137 >> 172.21.15.40:0/1574835137 conn(0x55fea2cee030 msgr2=0x55fea2dc9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.093+0000 7fe723c92b80 1 -- 172.21.15.40:0/1574835137 shutdown_connections 2024-06-06T03:38:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.093+0000 7fe723c92b80 1 -- 172.21.15.40:0/1574835137 wait complete. 2024-06-06T03:38:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.093+0000 7fe723c92b80 1 -- start start 2024-06-06T03:38:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.093+0000 7fe723c92b80 1 Processor -- start 2024-06-06T03:38:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe723c92b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2de5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe723c92b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 0x55fea2dbb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe723c92b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dbb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe723c92b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fea2ddc260 con 0x55fea2dc7830 2024-06-06T03:38:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe723c92b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fea2ddbb90 con 0x55fea2dc6f10 2024-06-06T03:38:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe723c92b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fea2ddc930 con 0x55fea2dc1750 2024-06-06T03:38:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe723c92b80 10 client.0 _ll_register_callbacks cb 0x55fea2ca8560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:38:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe7137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dbb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe7137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dbb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49700/0 (socket says 172.21.15.40:49700) 2024-06-06T03:38:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe70a7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 0x55fea2dbb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe7137fe700 1 -- 172.21.15.40:0/1734537668 learned_addr learned my addr 172.21.15.40:0/1734537668 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe712ffd700 1 --2- 172.21.15.40:0/1734537668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2de5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe7137fe700 1 -- 172.21.15.40:0/1734537668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 msgr2=0x55fea2de5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe7137fe700 1 --2- 172.21.15.40:0/1734537668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2de5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe7137fe700 1 -- 172.21.15.40:0/1734537668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 msgr2=0x55fea2dbb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe7137fe700 1 --2- 172.21.15.40:0/1734537668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 0x55fea2dbb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe7137fe700 1 -- 172.21.15.40:0/1734537668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea2dbbda0 con 0x55fea2dc7830 2024-06-06T03:38:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe70a7fc700 1 --2- 172.21.15.40:0/1734537668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 0x55fea2dbb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe712ffd700 1 --2- 172.21.15.40:0/1734537668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2de5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe7137fe700 1 --2- 172.21.15.40:0/1734537668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dbb760 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe70c005310 tx=0x7fe70c005220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce40c6a2c461f686 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe6fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:38:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.094+0000 7fe6fa7fc700 1 -- 172.21.15.40:0/1734537668 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c01f040 con 0x55fea2dc7830 2024-06-06T03:38:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.095+0000 7fe6fa7fc700 1 -- 172.21.15.40:0/1734537668 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fe70c02c070 con 0x55fea2dc7830 2024-06-06T03:38:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.095+0000 7fe723c92b80 1 -- 172.21.15.40:0/1734537668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe70c005e00 con 0x55fea2dc7830 2024-06-06T03:38:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.095+0000 7fe6fa7fc700 1 -- 172.21.15.40:0/1734537668 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe70c007900 con 0x55fea2dc7830 2024-06-06T03:38:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.095+0000 7fe723c92b80 10 client.15114 fetch_fsmap learned FSMap version 28 2024-06-06T03:38:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.095+0000 7fe723c92b80 1 -- 172.21.15.40:0/1734537668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fea2de4750 con 0x55fea2dc7830 2024-06-06T03:38:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.095+0000 7fe6fa7fc700 1 -- 172.21.15.40:0/1734537668 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70c005e00 con 0x55fea2dc7830 2024-06-06T03:38:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.095+0000 7fe6fa7fc700 1 -- 172.21.15.40:0/1734537668 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe70c023030 con 0x55fea2dc7830 2024-06-06T03:38:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.095+0000 7fe723c92b80 10 client.15114 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:38:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.095+0000 7fe723c92b80 10 client.15114 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T03:38:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.095+0000 7fe723c92b80 20 client.15114 populate_metadata read hostname 'smithi040' 2024-06-06T03:38:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.095+0000 7fe723c92b80 1 --2- 172.21.15.40:0/1734537668 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x55fea2e4af30 0x55fea2e49c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.095+0000 7fe723c92b80 4 client.15114 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:38:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.095+0000 7fe723c92b80 1 -- 172.21.15.40:0/1734537668 --> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] -- command(tid 0: {"prefix": "status"}) -- 0x55fea2d1a280 con 0x55fea2e4af30 2024-06-06T03:38:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe712ffd700 1 --2- 172.21.15.40:0/1734537668 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x55fea2e4af30 0x55fea2e49c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe712ffd700 1 --2- 172.21.15.40:0/1734537668 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x55fea2e4af30 0x55fea2e49c40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c2ad74bf08e4d464 server_cookie=97fd7b3e25590304 in_seq=0 out_seq=0 2024-06-06T03:38:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe6fa7fc700 10 client.15114 ms_handle_connect on v2:172.21.15.131:6838/782268782 2024-06-06T03:38:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe6fa7fc700 1 -- 172.21.15.40:0/1734537668 <== mds.3 v2:172.21.15.131:6838/782268782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fea2d1a280 con 0x55fea2e4af30 2024-06-06T03:38:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe6fa7fc700 10 client.15114 handle_command_reply: tid=0 2024-06-06T03:38:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe723c92b80 1 client.15114 shutdown 2024-06-06T03:38:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe723c92b80 1 -- 172.21.15.40:0/1734537668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 msgr2=0x55fea2dbb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe723c92b80 1 --2- 172.21.15.40:0/1734537668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dbb760 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe70c005310 tx=0x7fe70c005220 comp rx=0 tx=0).stop 2024-06-06T03:38:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe723c92b80 1 -- 172.21.15.40:0/1734537668 shutdown_connections 2024-06-06T03:38:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe723c92b80 1 --2- 172.21.15.40:0/1734537668 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x55fea2e4af30 0x55fea2e49c40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe6fa7fc700 0 client.15114 ms_handle_reset on v2:172.21.15.131:6838/782268782 2024-06-06T03:38:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe723c92b80 1 --2- 172.21.15.40:0/1734537668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fea2dc1750 0x55fea2de5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe723c92b80 1 --2- 172.21.15.40:0/1734537668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fea2dc7830 0x55fea2dbb760 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe723c92b80 1 --2- 172.21.15.40:0/1734537668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fea2dc6f10 0x55fea2dbb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.096+0000 7fe723c92b80 1 -- 172.21.15.40:0/1734537668 >> 172.21.15.40:0/1734537668 conn(0x55fea2cee030 msgr2=0x55fea2dc8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.097+0000 7fe723c92b80 1 -- 172.21.15.40:0/1734537668 shutdown_connections 2024-06-06T03:38:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.097+0000 7fe723c92b80 1 -- 172.21.15.40:0/1734537668 wait complete. 2024-06-06T03:38:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.097+0000 7fe723c92b80 20 client.15114 trim_cache size 0 max 16384 2024-06-06T03:38:21.102 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:38:21.102 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:38:21.102 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 3, 2024-06-06T03:38:21.102 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24403, 2024-06-06T03:38:21.102 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:38:21.102 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:38:21.102 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:38:21.102 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 168.17687402199999, 2024-06-06T03:38:21.102 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 28, 2024-06-06T03:38:21.102 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 90, 2024-06-06T03:38:21.102 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 88, 2024-06-06T03:38:21.102 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 212.698003446 2024-06-06T03:38:21.103 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:38:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.097+0000 7fe71940f700 1 -- 172.21.15.40:0/4266463506 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe7000a2ca0 msgr2=0x7fe7000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.097+0000 7fe71940f700 1 --2- 172.21.15.40:0/4266463506 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe7000a2ca0 0x7fe7000a5160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe704009680 tx=0x7fe704009260 comp rx=0 tx=0).stop 2024-06-06T03:38:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.097+0000 7fe71940f700 1 -- 172.21.15.40:0/4266463506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714101d40 msgr2=0x7fe714103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.097+0000 7fe71940f700 1 --2- 172.21.15.40:0/4266463506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714101d40 0x7fe714103570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00e9a0 tx=0x7fe6fc00ecb0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.098+0000 7fe71940f700 1 -- 172.21.15.40:0/4266463506 shutdown_connections 2024-06-06T03:38:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.098+0000 7fe71940f700 1 --2- 172.21.15.40:0/4266463506 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe7000a2ca0 0x7fe7000a5160 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.098+0000 7fe71940f700 1 --2- 172.21.15.40:0/4266463506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7141073c0 0x7fe714103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.098+0000 7fe71940f700 1 --2- 172.21.15.40:0/4266463506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714101380 0x7fe714103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.098+0000 7fe71940f700 1 --2- 172.21.15.40:0/4266463506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714101d40 0x7fe714103570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:21.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.098+0000 7fe71940f700 1 -- 172.21.15.40:0/4266463506 >> 172.21.15.40:0/4266463506 conn(0x7fe714078330 msgr2=0x7fe714100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:21.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.098+0000 7fe71940f700 1 -- 172.21.15.40:0/4266463506 shutdown_connections 2024-06-06T03:38:21.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:21.098+0000 7fe71940f700 1 -- 172.21.15.40:0/4266463506 wait complete. 2024-06-06T03:38:21.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:20 smithi131 ceph-mon[36687]: pgmap v396: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:38:21.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:20 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:38:21.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:20 smithi104 ceph-mon[36569]: pgmap v396: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T03:38:21.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:20 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:38:21.781 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-06-06T03:38:22.037 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:22.417 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:22 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:22 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:22 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:22 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:22 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:22 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:22 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:38:22.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:22 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:38:22.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:22 smithi040 ceph-mon[32220]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:22 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:22 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:22 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:22 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:22 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:22 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:22 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:38:22.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:22 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:38:22.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:22 smithi131 ceph-mon[36687]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:22 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:22 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:22 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:22 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:22 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:22 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:22.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:22 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:38:22.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:22 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:38:22.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:22 smithi104 ceph-mon[36569]: from='mgr.14152 172.21.15.40:0/165219287' entity='mgr.x' 2024-06-06T03:38:23.258 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:23 smithi040 ceph-mon[32220]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:38:23.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.483+0000 7f3096cce700 1 -- 172.21.15.40:0/353000108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3090106c00 msgr2=0x7f3090106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.483+0000 7f3096cce700 1 --2- 172.21.15.40:0/353000108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3090106c00 0x7f3090106fe0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f30840099c0 tx=0x7f3084009cd0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.483+0000 7f3096cce700 1 -- 172.21.15.40:0/353000108 shutdown_connections 2024-06-06T03:38:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.483+0000 7f3096cce700 1 --2- 172.21.15.40:0/353000108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3090101510 0x7f309010d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.483+0000 7f3096cce700 1 --2- 172.21.15.40:0/353000108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3090106c00 0x7f3090106fe0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.483+0000 7f3096cce700 1 --2- 172.21.15.40:0/353000108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3090100b50 0x7f3090100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.483+0000 7f3096cce700 1 -- 172.21.15.40:0/353000108 >> 172.21.15.40:0/353000108 conn(0x7f3090077990 msgr2=0x7f30900fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.483+0000 7f3096cce700 1 -- 172.21.15.40:0/353000108 shutdown_connections 2024-06-06T03:38:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.483+0000 7f3096cce700 1 -- 172.21.15.40:0/353000108 wait complete. 2024-06-06T03:38:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.483+0000 7f3096cce700 1 Processor -- start 2024-06-06T03:38:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.483+0000 7f3096cce700 1 -- start start 2024-06-06T03:38:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3096cce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3090100b50 0x7f309019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3096cce700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3090101510 0x7f309019dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3096cce700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3090106c00 0x7f3090197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3096cce700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f30901116f0 con 0x7f3090100b50 2024-06-06T03:38:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3096cce700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3090111570 con 0x7f3090101510 2024-06-06T03:38:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3096cce700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3090111870 con 0x7f3090106c00 2024-06-06T03:38:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3094a6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3090100b50 0x7f309019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3094a6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3090100b50 0x7f309019d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49722/0 (socket says 172.21.15.40:49722) 2024-06-06T03:38:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3094a6a700 1 -- 172.21.15.40:0/2208822188 learned_addr learned my addr 172.21.15.40:0/2208822188 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f308ffff700 1 --2- 172.21.15.40:0/2208822188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3090101510 0x7f309019dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f309526b700 1 --2- 172.21.15.40:0/2208822188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3090106c00 0x7f3090197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3094a6a700 1 -- 172.21.15.40:0/2208822188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3090106c00 msgr2=0x7f3090197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3094a6a700 1 --2- 172.21.15.40:0/2208822188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3090106c00 0x7f3090197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3094a6a700 1 -- 172.21.15.40:0/2208822188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3090101510 msgr2=0x7f309019dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3094a6a700 1 --2- 172.21.15.40:0/2208822188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3090101510 0x7f309019dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3094a6a700 1 -- 172.21.15.40:0/2208822188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3090197fd0 con 0x7f3090100b50 2024-06-06T03:38:23.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.484+0000 7f3094a6a700 1 --2- 172.21.15.40:0/2208822188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3090100b50 0x7f309019d6b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3084009fd0 tx=0x7f3084003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:23.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.485+0000 7f308dffb700 1 -- 172.21.15.40:0/2208822188 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084027070 con 0x7f3090100b50 2024-06-06T03:38:23.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.485+0000 7f3096cce700 1 -- 172.21.15.40:0/2208822188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3090198260 con 0x7f3090100b50 2024-06-06T03:38:23.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.485+0000 7f308dffb700 1 -- 172.21.15.40:0/2208822188 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30840040d0 con 0x7f3090100b50 2024-06-06T03:38:23.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.485+0000 7f308dffb700 1 -- 172.21.15.40:0/2208822188 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308400f460 con 0x7f3090100b50 2024-06-06T03:38:23.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.485+0000 7f3096cce700 1 -- 172.21.15.40:0/2208822188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f309010e390 con 0x7f3090100b50 2024-06-06T03:38:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.487+0000 7f308dffb700 1 -- 172.21.15.40:0/2208822188 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f308402a070 con 0x7f3090100b50 2024-06-06T03:38:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.488+0000 7f308dffb700 1 --2- 172.21.15.40:0/2208822188 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f30800a4e30 0x7f30800a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.488+0000 7f308ffff700 1 --2- 172.21.15.40:0/2208822188 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f30800a4e30 0x7f30800a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.488+0000 7f308dffb700 1 -- 172.21.15.40:0/2208822188 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f30840cff80 con 0x7f3090100b50 2024-06-06T03:38:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.488+0000 7f308ffff700 1 --2- 172.21.15.40:0/2208822188 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f30800a4e30 0x7f30800a72f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f30900fe5e0 tx=0x7f307c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.488+0000 7f30a1551b80 1 Processor -- start 2024-06-06T03:38:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f30a1551b80 1 -- start start 2024-06-06T03:38:23.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f30a1551b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedfe4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f30a1551b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfe5340 0x558eedfdeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f30a1551b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfdf2e0 0x558eedfdf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f30a1551b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558eedf2c7c0 con 0x558eedfe4a20 2024-06-06T03:38:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f30a1551b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558eeded46d0 con 0x558eedfdf2e0 2024-06-06T03:38:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f30a1551b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558eedf28780 con 0x558eedfe5340 2024-06-06T03:38:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f3094a6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedfe4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f3094a6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedfe4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49730/0 (socket says 172.21.15.40:49730) 2024-06-06T03:38:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f3094a6a700 1 -- 172.21.15.40:0/1754874516 learned_addr learned my addr 172.21.15.40:0/1754874516 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f309526b700 1 --2- 172.21.15.40:0/1754874516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfdf2e0 0x558eedfdf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f308ffff700 1 --2- 172.21.15.40:0/1754874516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfe5340 0x558eedfdeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f3094a6a700 1 -- 172.21.15.40:0/1754874516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfe5340 msgr2=0x558eedfdeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f3094a6a700 1 --2- 172.21.15.40:0/1754874516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfe5340 0x558eedfdeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f3094a6a700 1 -- 172.21.15.40:0/1754874516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfdf2e0 msgr2=0x558eedfdf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f3094a6a700 1 --2- 172.21.15.40:0/1754874516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfdf2e0 0x558eedfdf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f3094a6a700 1 -- 172.21.15.40:0/1754874516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eedecaf00 con 0x558eedfe4a20 2024-06-06T03:38:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.489+0000 7f3094a6a700 1 --2- 172.21.15.40:0/1754874516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedfe4e00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3084099300 tx=0x7f30840228f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74499e8a496a7310 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.490+0000 7f307affd700 1 -- 172.21.15.40:0/1754874516 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30840a0ce0 con 0x558eedfe4a20 2024-06-06T03:38:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.490+0000 7f307affd700 1 -- 172.21.15.40:0/1754874516 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30840a5510 con 0x558eedfe4a20 2024-06-06T03:38:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.490+0000 7f307affd700 1 -- 172.21.15.40:0/1754874516 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30840872f0 con 0x558eedfe4a20 2024-06-06T03:38:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.490+0000 7f30a1551b80 1 -- 172.21.15.40:0/1754874516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 msgr2=0x558eedfe4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.490+0000 7f30a1551b80 1 --2- 172.21.15.40:0/1754874516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedfe4e00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3084099300 tx=0x7f30840228f0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.490+0000 7f30a1551b80 1 -- 172.21.15.40:0/1754874516 shutdown_connections 2024-06-06T03:38:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.490+0000 7f30a1551b80 1 --2- 172.21.15.40:0/1754874516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfdf2e0 0x558eedfdf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.490+0000 7f30a1551b80 1 --2- 172.21.15.40:0/1754874516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedfe4e00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.490+0000 7f30a1551b80 1 --2- 172.21.15.40:0/1754874516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfe5340 0x558eedfdeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.490+0000 7f30a1551b80 1 -- 172.21.15.40:0/1754874516 >> 172.21.15.40:0/1754874516 conn(0x558eedf05c70 msgr2=0x558eedfd5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.490+0000 7f30a1551b80 1 -- 172.21.15.40:0/1754874516 shutdown_connections 2024-06-06T03:38:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.490+0000 7f30a1551b80 1 -- 172.21.15.40:0/1754874516 wait complete. 2024-06-06T03:38:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.491+0000 7f30a1551b80 1 -- start start 2024-06-06T03:38:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.491+0000 7f30a1551b80 1 Processor -- start 2024-06-06T03:38:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.491+0000 7f30a1551b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfdf2e0 0x558eedfeb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.491+0000 7f30a1551b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfe4a20 0x558eedfe8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.491+0000 7f30a1551b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 0x558eedfe89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.491+0000 7f30a1551b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558eedfebaa0 con 0x558eedfdf2e0 2024-06-06T03:38:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.491+0000 7f30a1551b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558eedeccba0 con 0x558eedfe5340 2024-06-06T03:38:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.491+0000 7f30a1551b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558eedfebc20 con 0x558eedfe4a20 2024-06-06T03:38:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.491+0000 7f3094a6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfdf2e0 0x558eedfeb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.491+0000 7f30a1551b80 10 client.0 _ll_register_callbacks cb 0x558eedec68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:38:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.491+0000 7f3094a6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfdf2e0 0x558eedfeb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49742/0 (socket says 172.21.15.40:49742) 2024-06-06T03:38:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.491+0000 7f3094a6a700 1 -- 172.21.15.40:0/1112353087 learned_addr learned my addr 172.21.15.40:0/1112353087 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f309526b700 1 --2- 172.21.15.40:0/1112353087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 0x558eedfe89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f308ffff700 1 --2- 172.21.15.40:0/1112353087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfe4a20 0x558eedfe8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f3094a6a700 1 -- 172.21.15.40:0/1112353087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfe4a20 msgr2=0x558eedfe8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f3094a6a700 1 --2- 172.21.15.40:0/1112353087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfe4a20 0x558eedfe8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f3094a6a700 1 -- 172.21.15.40:0/1112353087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 msgr2=0x558eedfe89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f3094a6a700 1 --2- 172.21.15.40:0/1112353087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 0x558eedfe89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f3094a6a700 1 -- 172.21.15.40:0/1112353087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eede7b880 con 0x558eedfdf2e0 2024-06-06T03:38:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f3094a6a700 1 --2- 172.21.15.40:0/1112353087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfdf2e0 0x558eedfeb180 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f308409c630 tx=0x7f308409ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b0d0315da84d72 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f306ffff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:38:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f306ffff700 1 -- 172.21.15.40:0/1112353087 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30840b9020 con 0x558eedfdf2e0 2024-06-06T03:38:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f306ffff700 1 -- 172.21.15.40:0/1112353087 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f308409cb90 con 0x558eedfdf2e0 2024-06-06T03:38:23.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f30a1551b80 1 -- 172.21.15.40:0/1112353087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30840a5510 con 0x558eedfdf2e0 2024-06-06T03:38:23.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f306ffff700 1 -- 172.21.15.40:0/1112353087 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f30840bb2c0 con 0x558eedfdf2e0 2024-06-06T03:38:23.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f30a1551b80 10 client.15126 fetch_fsmap learned FSMap version 28 2024-06-06T03:38:23.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.492+0000 7f30a1551b80 1 -- 172.21.15.40:0/1112353087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558eedfebaa0 con 0x558eedfdf2e0 2024-06-06T03:38:23.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.493+0000 7f306ffff700 1 -- 172.21.15.40:0/1112353087 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30840a5510 con 0x558eedfdf2e0 2024-06-06T03:38:23.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.493+0000 7f306ffff700 1 -- 172.21.15.40:0/1112353087 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3084022470 con 0x558eedfdf2e0 2024-06-06T03:38:23.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.493+0000 7f30a1551b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:38:23.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.493+0000 7f30a1551b80 10 client.15126 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-06T03:38:23.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.493+0000 7f30a1551b80 20 client.15126 populate_metadata read hostname 'smithi040' 2024-06-06T03:38:23.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.493+0000 7f30a1551b80 1 --2- 172.21.15.40:0/1112353087 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x558eedff8170 0x558eedff9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.493+0000 7f30a1551b80 4 client.15126 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:38:23.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.493+0000 7f30a1551b80 1 -- 172.21.15.40:0/1112353087 --> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558eedf39cc0 con 0x558eedff8170 2024-06-06T03:38:23.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.493+0000 7f308ffff700 1 --2- 172.21.15.40:0/1112353087 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x558eedff8170 0x558eedff9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.494+0000 7f308ffff700 1 --2- 172.21.15.40:0/1112353087 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x558eedff8170 0x558eedff9c40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d8aecb20148d797 server_cookie=61afd9d0394c1292 in_seq=0 out_seq=0 2024-06-06T03:38:23.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.494+0000 7f306ffff700 10 client.15126 ms_handle_connect on v2:172.21.15.104:6834/75462915 2024-06-06T03:38:23.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.494+0000 7f306ffff700 1 -- 172.21.15.40:0/1112353087 <== mds.2 v2:172.21.15.104:6834/75462915 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558eedf39cc0 con 0x558eedff8170 2024-06-06T03:38:23.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.494+0000 7f306ffff700 10 client.15126 handle_command_reply: tid=0 2024-06-06T03:38:23.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.494+0000 7f30a1551b80 1 client.15126 shutdown 2024-06-06T03:38:23.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.494+0000 7f30a1551b80 1 -- 172.21.15.40:0/1112353087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfdf2e0 msgr2=0x558eedfeb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:23.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.494+0000 7f30a1551b80 1 --2- 172.21.15.40:0/1112353087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfdf2e0 0x558eedfeb180 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f308409c630 tx=0x7f308409ca60 comp rx=0 tx=0).stop 2024-06-06T03:38:23.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.495+0000 7f30a1551b80 1 -- 172.21.15.40:0/1112353087 shutdown_connections 2024-06-06T03:38:23.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.495+0000 7f30a1551b80 1 --2- 172.21.15.40:0/1112353087 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x558eedff8170 0x558eedff9c40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.495+0000 7f30a1551b80 1 --2- 172.21.15.40:0/1112353087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 0x558eedfe89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.495+0000 7f30a1551b80 1 --2- 172.21.15.40:0/1112353087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfdf2e0 0x558eedfeb180 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.495+0000 7f306ffff700 0 client.15126 ms_handle_reset on v2:172.21.15.104:6834/75462915 2024-06-06T03:38:23.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.495+0000 7f30a1551b80 1 --2- 172.21.15.40:0/1112353087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfe4a20 0x558eedfe8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.495+0000 7f30a1551b80 1 -- 172.21.15.40:0/1112353087 >> 172.21.15.40:0/1112353087 conn(0x558eedf05c70 msgr2=0x558eedfd9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:23.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.495+0000 7f30a1551b80 1 -- 172.21.15.40:0/1112353087 shutdown_connections 2024-06-06T03:38:23.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.495+0000 7f30a1551b80 1 -- 172.21.15.40:0/1112353087 wait complete. 2024-06-06T03:38:23.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.495+0000 7f30a1551b80 20 client.15126 trim_cache size 0 max 16384 2024-06-06T03:38:23.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f30a1551b80 1 Processor -- start 2024-06-06T03:38:23.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f30a1551b80 1 -- start start 2024-06-06T03:38:23.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f30a1551b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfdf2e0 0x558eedfd56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f30a1551b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedfcf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f30a1551b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 0x558eedfe5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f30a1551b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558eedff7ff0 con 0x558eedfe4a20 2024-06-06T03:38:23.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f30a1551b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558eedff8650 con 0x558eedfe5340 2024-06-06T03:38:23.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f30a1551b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558eedff8d20 con 0x558eedfdf2e0 2024-06-06T03:38:23.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f308ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedfcf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f308ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedfcf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49756/0 (socket says 172.21.15.40:49756) 2024-06-06T03:38:23.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f308ffff700 1 -- 172.21.15.40:0/2022219872 learned_addr learned my addr 172.21.15.40:0/2022219872 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:23.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f3094a6a700 1 --2- 172.21.15.40:0/2022219872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfdf2e0 0x558eedfd56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f308ffff700 1 -- 172.21.15.40:0/2022219872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfdf2e0 msgr2=0x558eedfd56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:23.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f308ffff700 1 --2- 172.21.15.40:0/2022219872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfdf2e0 0x558eedfd56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f308ffff700 1 -- 172.21.15.40:0/2022219872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 msgr2=0x558eedfe5e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T03:38:23.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f308ffff700 1 --2- 172.21.15.40:0/2022219872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 0x558eedfe5e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f308ffff700 1 -- 172.21.15.40:0/2022219872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eedfe6a10 con 0x558eedfe4a20 2024-06-06T03:38:23.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.517+0000 7f308ffff700 1 --2- 172.21.15.40:0/2022219872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedfcf640 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f307c005c60 tx=0x7f307c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd14636e16e3826 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:23.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.518+0000 7f307a7fc700 1 -- 172.21.15.40:0/2022219872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c016070 con 0x558eedfe4a20 2024-06-06T03:38:23.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.518+0000 7f307a7fc700 1 -- 172.21.15.40:0/2022219872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f307c0107c0 con 0x558eedfe4a20 2024-06-06T03:38:23.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.518+0000 7f307a7fc700 1 -- 172.21.15.40:0/2022219872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c005440 con 0x558eedfe4a20 2024-06-06T03:38:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.518+0000 7f30a1551b80 1 -- 172.21.15.40:0/2022219872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 msgr2=0x558eedfcf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.518+0000 7f30a1551b80 1 --2- 172.21.15.40:0/2022219872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedfcf640 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f307c005c60 tx=0x7f307c012000 comp rx=0 tx=0).stop 2024-06-06T03:38:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.518+0000 7f30a1551b80 1 -- 172.21.15.40:0/2022219872 shutdown_connections 2024-06-06T03:38:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.518+0000 7f30a1551b80 1 --2- 172.21.15.40:0/2022219872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfdf2e0 0x558eedfd56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.518+0000 7f30a1551b80 1 --2- 172.21.15.40:0/2022219872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 0x558eedfe5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.518+0000 7f30a1551b80 1 --2- 172.21.15.40:0/2022219872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedfcf640 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.518+0000 7f30a1551b80 1 -- 172.21.15.40:0/2022219872 >> 172.21.15.40:0/2022219872 conn(0x558eedf05c70 msgr2=0x558eedfe5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.519+0000 7f30a1551b80 1 -- 172.21.15.40:0/2022219872 shutdown_connections 2024-06-06T03:38:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.519+0000 7f30a1551b80 1 -- 172.21.15.40:0/2022219872 wait complete. 2024-06-06T03:38:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.519+0000 7f30a1551b80 1 -- start start 2024-06-06T03:38:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.519+0000 7f30a1551b80 1 Processor -- start 2024-06-06T03:38:23.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.519+0000 7f30a1551b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfdf2e0 0x558eee0038d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.519+0000 7f30a1551b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedffcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.519+0000 7f30a1551b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 0x558eedffd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.519+0000 7f30a1551b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558eedff9ac0 con 0x558eedfe4a20 2024-06-06T03:38:23.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.519+0000 7f30a1551b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558eedff93f0 con 0x558eedfe5340 2024-06-06T03:38:23.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.519+0000 7f30a1551b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558eedffa190 con 0x558eedfdf2e0 2024-06-06T03:38:23.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.519+0000 7f30a1551b80 10 client.0 _ll_register_callbacks cb 0x558eedec68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:38:23.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f308ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedffcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f308ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedffcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49760/0 (socket says 172.21.15.40:49760) 2024-06-06T03:38:23.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f308ffff700 1 -- 172.21.15.40:0/3479138924 learned_addr learned my addr 172.21.15.40:0/3479138924 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:23.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f3094a6a700 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfdf2e0 0x558eee0038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f309526b700 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 0x558eedffd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f308ffff700 1 -- 172.21.15.40:0/3479138924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfdf2e0 msgr2=0x558eee0038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f308ffff700 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfdf2e0 0x558eee0038d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f308ffff700 1 -- 172.21.15.40:0/3479138924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 msgr2=0x558eedffd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f308ffff700 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 0x558eedffd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f308ffff700 1 -- 172.21.15.40:0/3479138924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eedffdaa0 con 0x558eedfe4a20 2024-06-06T03:38:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f3094a6a700 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfdf2e0 0x558eee0038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f309526b700 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 0x558eedffd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:38:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f308ffff700 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedffcc80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f307c010010 tx=0x7f307c010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc13cb227eeb779 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f306ffff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:38:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f306ffff700 1 -- 172.21.15.40:0/3479138924 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c004fb0 con 0x558eedfe4a20 2024-06-06T03:38:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f306ffff700 1 -- 172.21.15.40:0/3479138924 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f307c005150 con 0x558eedfe4a20 2024-06-06T03:38:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f30a1551b80 1 -- 172.21.15.40:0/3479138924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f307c0107c0 con 0x558eedfe4a20 2024-06-06T03:38:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.520+0000 7f306ffff700 1 -- 172.21.15.40:0/3479138924 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f307c005350 con 0x558eedfe4a20 2024-06-06T03:38:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.521+0000 7f30a1551b80 10 client.15132 fetch_fsmap learned FSMap version 28 2024-06-06T03:38:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.521+0000 7f30a1551b80 1 -- 172.21.15.40:0/3479138924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558eee064170 con 0x558eedfe4a20 2024-06-06T03:38:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.521+0000 7f306ffff700 1 -- 172.21.15.40:0/3479138924 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f307c040070 con 0x558eedfe4a20 2024-06-06T03:38:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.521+0000 7f306ffff700 1 -- 172.21.15.40:0/3479138924 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f307c043070 con 0x558eedfe4a20 2024-06-06T03:38:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.521+0000 7f30a1551b80 10 client.15132 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:38:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.521+0000 7f30a1551b80 10 client.15132 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-06T03:38:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.521+0000 7f30a1551b80 20 client.15132 populate_metadata read hostname 'smithi040' 2024-06-06T03:38:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.521+0000 7f30a1551b80 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x558eee068960 0x558eee067670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.521+0000 7f30a1551b80 4 client.15132 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:38:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.521+0000 7f30a1551b80 1 -- 172.21.15.40:0/3479138924 --> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] -- command(tid 0: {"prefix": "status"}) -- 0x7f307c0107c0 con 0x558eee068960 2024-06-06T03:38:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.521+0000 7f3094a6a700 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x558eee068960 0x558eee067670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f3094a6a700 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x558eee068960 0x558eee067670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb9091ac5258c2a9 server_cookie=63b2a21d3a9961e4 in_seq=0 out_seq=0 2024-06-06T03:38:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f306ffff700 10 client.15132 ms_handle_connect on v2:172.21.15.104:6834/75462915 2024-06-06T03:38:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f306ffff700 1 -- 172.21.15.40:0/3479138924 <== mds.2 v2:172.21.15.104:6834/75462915 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x7f307c0107c0 con 0x558eee068960 2024-06-06T03:38:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f306ffff700 10 client.15132 handle_command_reply: tid=0 2024-06-06T03:38:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f30a1551b80 1 client.15132 shutdown 2024-06-06T03:38:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f30a1551b80 1 -- 172.21.15.40:0/3479138924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 msgr2=0x558eedffcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f30a1551b80 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedffcc80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f307c010010 tx=0x7f307c010d90 comp rx=0 tx=0).stop 2024-06-06T03:38:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f30a1551b80 1 -- 172.21.15.40:0/3479138924 shutdown_connections 2024-06-06T03:38:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f30a1551b80 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x558eee068960 0x558eee067670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f30a1551b80 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558eedfdf2e0 0x558eee0038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f30a1551b80 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558eedfe5340 0x558eedffd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f306ffff700 0 client.15132 ms_handle_reset on v2:172.21.15.104:6834/75462915 2024-06-06T03:38:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f30a1551b80 1 --2- 172.21.15.40:0/3479138924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558eedfe4a20 0x558eedffcc80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f30a1551b80 1 -- 172.21.15.40:0/3479138924 >> 172.21.15.40:0/3479138924 conn(0x558eedf05c70 msgr2=0x558eedfcf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:23.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f30a1551b80 1 -- 172.21.15.40:0/3479138924 shutdown_connections 2024-06-06T03:38:23.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f30a1551b80 1 -- 172.21.15.40:0/3479138924 wait complete. 2024-06-06T03:38:23.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.522+0000 7f30a1551b80 20 client.15132 trim_cache size 0 max 16384 2024-06-06T03:38:23.528 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:38:23.528 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:38:23.528 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 2, 2024-06-06T03:38:23.528 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24425, 2024-06-06T03:38:23.528 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:38:23.528 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:38:23.528 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:38:23.529 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 171.61015617800001, 2024-06-06T03:38:23.529 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 28, 2024-06-06T03:38:23.529 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 90, 2024-06-06T03:38:23.529 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 88, 2024-06-06T03:38:23.529 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 226.55425420899999 2024-06-06T03:38:23.529 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:38:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.523+0000 7f3096cce700 1 -- 172.21.15.40:0/2208822188 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f30800a4e30 msgr2=0x7f30800a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.523+0000 7f3096cce700 1 --2- 172.21.15.40:0/2208822188 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f30800a4e30 0x7f30800a72f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f30900fe5e0 tx=0x7f307c00a300 comp rx=0 tx=0).stop 2024-06-06T03:38:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.523+0000 7f3096cce700 1 -- 172.21.15.40:0/2208822188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3090100b50 msgr2=0x7f309019d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.523+0000 7f3096cce700 1 --2- 172.21.15.40:0/2208822188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3090100b50 0x7f309019d6b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3084009fd0 tx=0x7f3084003680 comp rx=0 tx=0).stop 2024-06-06T03:38:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.523+0000 7f3096cce700 1 -- 172.21.15.40:0/2208822188 shutdown_connections 2024-06-06T03:38:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.523+0000 7f3096cce700 1 --2- 172.21.15.40:0/2208822188 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f30800a4e30 0x7f30800a72f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.523+0000 7f3096cce700 1 --2- 172.21.15.40:0/2208822188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3090106c00 0x7f3090197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.523+0000 7f3096cce700 1 --2- 172.21.15.40:0/2208822188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3090100b50 0x7f309019d6b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.523+0000 7f3096cce700 1 --2- 172.21.15.40:0/2208822188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3090101510 0x7f309019dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.523+0000 7f3096cce700 1 -- 172.21.15.40:0/2208822188 >> 172.21.15.40:0/2208822188 conn(0x7f3090077990 msgr2=0x7f30900feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.523+0000 7f3096cce700 1 -- 172.21.15.40:0/2208822188 shutdown_connections 2024-06-06T03:38:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:23.523+0000 7f3096cce700 1 -- 172.21.15.40:0/2208822188 wait complete. 2024-06-06T03:38:23.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:23 smithi131 ceph-mon[36687]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:38:23.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:23 smithi104 ceph-mon[36569]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T03:38:24.085 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-06T03:38:24.336 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:25.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:25 smithi040 ceph-mon[32220]: pgmap v398: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:38:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:25 smithi131 ceph-mon[36687]: pgmap v398: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:38:25.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:25 smithi104 ceph-mon[36569]: pgmap v398: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-06T03:38:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.925+0000 7f42c6e77700 1 -- 172.21.15.40:0/2267949599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42c0106750 msgr2=0x7f42c010e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.925+0000 7f42c6e77700 1 --2- 172.21.15.40:0/2267949599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42c0106750 0x7f42c010e2d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f42b80099c0 tx=0x7f42b8009cd0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.925+0000 7f42c6e77700 1 -- 172.21.15.40:0/2267949599 shutdown_connections 2024-06-06T03:38:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.925+0000 7f42c6e77700 1 --2- 172.21.15.40:0/2267949599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c0101df0 0x7f42c01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.925+0000 7f42c6e77700 1 --2- 172.21.15.40:0/2267949599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42c0106750 0x7f42c010e2d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.925+0000 7f42c6e77700 1 --2- 172.21.15.40:0/2267949599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42c01027a0 0x7f42c0106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.925+0000 7f42c6e77700 1 -- 172.21.15.40:0/2267949599 >> 172.21.15.40:0/2267949599 conn(0x7f42c0077a00 msgr2=0x7f42c00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:25.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.925+0000 7f42c6e77700 1 -- 172.21.15.40:0/2267949599 shutdown_connections 2024-06-06T03:38:25.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.925+0000 7f42c6e77700 1 -- 172.21.15.40:0/2267949599 wait complete. 2024-06-06T03:38:25.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.926+0000 7f42c6e77700 1 Processor -- start 2024-06-06T03:38:25.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.926+0000 7f42c6e77700 1 -- start start 2024-06-06T03:38:25.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.926+0000 7f42c6e77700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42c0101df0 0x7f42c00fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.926+0000 7f42c6e77700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42c01027a0 0x7f42c00fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.926+0000 7f42c6e77700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c0106750 0x7f42c00fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.926+0000 7f42c6e77700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0111700 con 0x7f42c01027a0 2024-06-06T03:38:25.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.926+0000 7f42c6e77700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0111580 con 0x7f42c0101df0 2024-06-06T03:38:25.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.926+0000 7f42c6e77700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0111880 con 0x7f42c0106750 2024-06-06T03:38:25.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.926+0000 7f42bffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42c01027a0 0x7f42c00fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.926+0000 7f42bffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42c01027a0 0x7f42c00fd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49774/0 (socket says 172.21.15.40:49774) 2024-06-06T03:38:25.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.926+0000 7f42bffff700 1 -- 172.21.15.40:0/4071665283 learned_addr learned my addr 172.21.15.40:0/4071665283 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:25.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.926+0000 7f42c4c13700 1 --2- 172.21.15.40:0/4071665283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42c0101df0 0x7f42c00fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.926+0000 7f42c5414700 1 --2- 172.21.15.40:0/4071665283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c0106750 0x7f42c00fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.927+0000 7f42bffff700 1 -- 172.21.15.40:0/4071665283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c0106750 msgr2=0x7f42c00fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:25.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.927+0000 7f42bffff700 1 --2- 172.21.15.40:0/4071665283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c0106750 0x7f42c00fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.927+0000 7f42bffff700 1 -- 172.21.15.40:0/4071665283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42c0101df0 msgr2=0x7f42c00fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:25.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.927+0000 7f42bffff700 1 --2- 172.21.15.40:0/4071665283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42c0101df0 0x7f42c00fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.927+0000 7f42bffff700 1 -- 172.21.15.40:0/4071665283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c01a70f0 con 0x7f42c01027a0 2024-06-06T03:38:25.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.927+0000 7f42bffff700 1 --2- 172.21.15.40:0/4071665283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42c01027a0 0x7f42c00fd3b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f42b000ba50 tx=0x7f42b000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:25.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.927+0000 7f42bdffb700 1 -- 172.21.15.40:0/4071665283 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0019070 con 0x7f42c01027a0 2024-06-06T03:38:25.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.927+0000 7f42bdffb700 1 -- 172.21.15.40:0/4071665283 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42b000ce10 con 0x7f42c01027a0 2024-06-06T03:38:25.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.927+0000 7f42c6e77700 1 -- 172.21.15.40:0/4071665283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c01a73e0 con 0x7f42c01027a0 2024-06-06T03:38:25.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.927+0000 7f42bdffb700 1 -- 172.21.15.40:0/4071665283 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0007390 con 0x7f42c01027a0 2024-06-06T03:38:25.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.927+0000 7f42c6e77700 1 -- 172.21.15.40:0/4071665283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c01a7970 con 0x7f42c01027a0 2024-06-06T03:38:25.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.929+0000 7f42bdffb700 1 -- 172.21.15.40:0/4071665283 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f42b001c070 con 0x7f42c01027a0 2024-06-06T03:38:25.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.930+0000 7f42bdffb700 1 --2- 172.21.15.40:0/4071665283 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42ac0a2ca0 0x7f42ac0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.931+0000 7f42c4c13700 1 --2- 172.21.15.40:0/4071665283 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42ac0a2ca0 0x7f42ac0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.931+0000 7f42bdffb700 1 -- 172.21.15.40:0/4071665283 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f42b00ce310 con 0x7f42c01027a0 2024-06-06T03:38:25.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.931+0000 7f42c4c13700 1 --2- 172.21.15.40:0/4071665283 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42ac0a2ca0 0x7f42ac0a5160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f42a8004630 tx=0x7f42a800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:25.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.931+0000 7f42d16fab80 1 Processor -- start 2024-06-06T03:38:25.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42d16fab80 1 -- start start 2024-06-06T03:38:25.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42d16fab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283d25f0 0x562e283d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42d16fab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42d16fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283cced0 0x562e283cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42d16fab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562e2831e7c0 con 0x562e283d2f10 2024-06-06T03:38:25.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42d16fab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562e282c66d0 con 0x562e283d25f0 2024-06-06T03:38:25.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42d16fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562e2831a780 con 0x562e283cced0 2024-06-06T03:38:25.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42bffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42bffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49790/0 (socket says 172.21.15.40:49790) 2024-06-06T03:38:25.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42bffff700 1 -- 172.21.15.40:0/2343433950 learned_addr learned my addr 172.21.15.40:0/2343433950 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:25.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42c5414700 1 --2- 172.21.15.40:0/2343433950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283cced0 0x562e283cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42c4c13700 1 --2- 172.21.15.40:0/2343433950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283d25f0 0x562e283d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42bffff700 1 -- 172.21.15.40:0/2343433950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283cced0 msgr2=0x562e283cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42bffff700 1 --2- 172.21.15.40:0/2343433950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283cced0 0x562e283cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42bffff700 1 -- 172.21.15.40:0/2343433950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283d25f0 msgr2=0x562e283d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42bffff700 1 --2- 172.21.15.40:0/2343433950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283d25f0 0x562e283d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.932+0000 7f42bffff700 1 -- 172.21.15.40:0/2343433950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e282bcf00 con 0x562e283d2f10 2024-06-06T03:38:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.933+0000 7f42bffff700 1 --2- 172.21.15.40:0/2343433950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283cc990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f42b0097560 tx=0x7f42b0004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc4492c6c324feb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.933+0000 7f42b6ffd700 1 -- 172.21.15.40:0/2343433950 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b009ab90 con 0x562e283d2f10 2024-06-06T03:38:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.933+0000 7f42b6ffd700 1 -- 172.21.15.40:0/2343433950 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42b009ef40 con 0x562e283d2f10 2024-06-06T03:38:25.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.933+0000 7f42b6ffd700 1 -- 172.21.15.40:0/2343433950 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b00a35b0 con 0x562e283d2f10 2024-06-06T03:38:25.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.933+0000 7f42d16fab80 1 -- 172.21.15.40:0/2343433950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 msgr2=0x562e283cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:25.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.933+0000 7f42d16fab80 1 --2- 172.21.15.40:0/2343433950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283cc990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f42b0097560 tx=0x7f42b0004be0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.934+0000 7f42d16fab80 1 -- 172.21.15.40:0/2343433950 shutdown_connections 2024-06-06T03:38:25.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.934+0000 7f42d16fab80 1 --2- 172.21.15.40:0/2343433950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283cced0 0x562e283cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.934+0000 7f42d16fab80 1 --2- 172.21.15.40:0/2343433950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283d25f0 0x562e283d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.934+0000 7f42d16fab80 1 --2- 172.21.15.40:0/2343433950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283cc990 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.934+0000 7f42d16fab80 1 -- 172.21.15.40:0/2343433950 >> 172.21.15.40:0/2343433950 conn(0x562e282f7c70 msgr2=0x562e283c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:25.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.934+0000 7f42d16fab80 1 -- 172.21.15.40:0/2343433950 shutdown_connections 2024-06-06T03:38:25.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.934+0000 7f42d16fab80 1 -- 172.21.15.40:0/2343433950 wait complete. 2024-06-06T03:38:25.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.934+0000 7f42d16fab80 1 -- start start 2024-06-06T03:38:25.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.934+0000 7f42d16fab80 1 Processor -- start 2024-06-06T03:38:25.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42d16fab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283cced0 0x562e283d8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42d16fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 0x562e283d6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42d16fab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283d6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42d16fab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562e283d9980 con 0x562e283d2f10 2024-06-06T03:38:25.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42d16fab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562e282beba0 con 0x562e283cced0 2024-06-06T03:38:25.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42d16fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562e283d9b00 con 0x562e283d25f0 2024-06-06T03:38:25.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42d16fab80 10 client.0 _ll_register_callbacks cb 0x562e282b88d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:38:25.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42c5414700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283d6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42c5414700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283d6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49798/0 (socket says 172.21.15.40:49798) 2024-06-06T03:38:25.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42c5414700 1 -- 172.21.15.40:0/3314194621 learned_addr learned my addr 172.21.15.40:0/3314194621 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42bffff700 1 --2- 172.21.15.40:0/3314194621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 0x562e283d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42c4c13700 1 --2- 172.21.15.40:0/3314194621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283cced0 0x562e283d8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42c5414700 1 -- 172.21.15.40:0/3314194621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 msgr2=0x562e283d6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42c5414700 1 --2- 172.21.15.40:0/3314194621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 0x562e283d6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42c5414700 1 -- 172.21.15.40:0/3314194621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283cced0 msgr2=0x562e283d8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42c5414700 1 --2- 172.21.15.40:0/3314194621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283cced0 0x562e283d8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42c5414700 1 -- 172.21.15.40:0/3314194621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e2826d880 con 0x562e283d2f10 2024-06-06T03:38:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42c4c13700 1 --2- 172.21.15.40:0/3314194621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283cced0 0x562e283d8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f42c5414700 1 --2- 172.21.15.40:0/3314194621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283d6550 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f42b8005cd0 tx=0x7f42b800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24a15aa5283fb64 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f429ffff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:38:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.935+0000 7f429ffff700 1 -- 172.21.15.40:0/3314194621 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8029060 con 0x562e283d2f10 2024-06-06T03:38:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.936+0000 7f429ffff700 1 -- 172.21.15.40:0/3314194621 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f42b8003b60 con 0x562e283d2f10 2024-06-06T03:38:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.936+0000 7f42d16fab80 1 -- 172.21.15.40:0/3314194621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b009ef40 con 0x562e283d2f10 2024-06-06T03:38:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.936+0000 7f429ffff700 1 -- 172.21.15.40:0/3314194621 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42b8003d60 con 0x562e283d2f10 2024-06-06T03:38:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.936+0000 7f42d16fab80 10 client.15144 fetch_fsmap learned FSMap version 28 2024-06-06T03:38:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.936+0000 7f42d16fab80 1 -- 172.21.15.40:0/3314194621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e283e4360 con 0x562e283d2f10 2024-06-06T03:38:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.936+0000 7f429ffff700 1 -- 172.21.15.40:0/3314194621 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42b009ef40 con 0x562e283d2f10 2024-06-06T03:38:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.936+0000 7f429ffff700 1 -- 172.21.15.40:0/3314194621 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f42b800f950 con 0x562e283d2f10 2024-06-06T03:38:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.936+0000 7f42d16fab80 10 client.15144 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:38:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.936+0000 7f42d16fab80 10 client.15144 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T03:38:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.936+0000 7f42d16fab80 20 client.15144 populate_metadata read hostname 'smithi040' 2024-06-06T03:38:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.936+0000 7f42d16fab80 1 --2- 172.21.15.40:0/3314194621 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x562e283e6970 0x562e283e7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.936+0000 7f42d16fab80 4 client.15144 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:38:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.936+0000 7f42d16fab80 1 -- 172.21.15.40:0/3314194621 --> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e2832bcc0 con 0x562e283e6970 2024-06-06T03:38:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.937+0000 7f42c4c13700 1 --2- 172.21.15.40:0/3314194621 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x562e283e6970 0x562e283e7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.937+0000 7f429ffff700 10 client.15144 ms_handle_connect on v2:172.21.15.104:6836/2498466212 2024-06-06T03:38:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.937+0000 7f42c4c13700 1 --2- 172.21.15.40:0/3314194621 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x562e283e6970 0x562e283e7de0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb9abb80c8b0b873 server_cookie=562df4126d2f63d8 in_seq=0 out_seq=0 2024-06-06T03:38:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.938+0000 7f429ffff700 1 -- 172.21.15.40:0/3314194621 <== mds.1 v2:172.21.15.104:6836/2498466212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562e2832bcc0 con 0x562e283e6970 2024-06-06T03:38:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.938+0000 7f429ffff700 10 client.15144 handle_command_reply: tid=0 2024-06-06T03:38:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.938+0000 7f42d16fab80 1 client.15144 shutdown 2024-06-06T03:38:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.938+0000 7f42d16fab80 1 -- 172.21.15.40:0/3314194621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 msgr2=0x562e283d6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.938+0000 7f42d16fab80 1 --2- 172.21.15.40:0/3314194621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283d6550 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f42b8005cd0 tx=0x7f42b800bf50 comp rx=0 tx=0).stop 2024-06-06T03:38:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.938+0000 7f42d16fab80 1 -- 172.21.15.40:0/3314194621 shutdown_connections 2024-06-06T03:38:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.938+0000 7f42d16fab80 1 --2- 172.21.15.40:0/3314194621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283d6550 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.938+0000 7f42d16fab80 1 --2- 172.21.15.40:0/3314194621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283cced0 0x562e283d8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.938+0000 7f42d16fab80 1 --2- 172.21.15.40:0/3314194621 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x562e283e6970 0x562e283e7de0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.938+0000 7f42d16fab80 1 --2- 172.21.15.40:0/3314194621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 0x562e283d6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.938+0000 7f429ffff700 0 client.15144 ms_handle_reset on v2:172.21.15.104:6836/2498466212 2024-06-06T03:38:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.938+0000 7f42d16fab80 1 -- 172.21.15.40:0/3314194621 >> 172.21.15.40:0/3314194621 conn(0x562e282f7c70 msgr2=0x562e283c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.939+0000 7f42d16fab80 1 -- 172.21.15.40:0/3314194621 shutdown_connections 2024-06-06T03:38:25.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.939+0000 7f42d16fab80 1 -- 172.21.15.40:0/3314194621 wait complete. 2024-06-06T03:38:25.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.939+0000 7f42d16fab80 20 client.15144 trim_cache size 0 max 16384 2024-06-06T03:38:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.960+0000 7f42d16fab80 1 Processor -- start 2024-06-06T03:38:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.960+0000 7f42d16fab80 1 -- start start 2024-06-06T03:38:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.960+0000 7f42d16fab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283cced0 0x562e283cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.960+0000 7f42d16fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 0x562e283d3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.960+0000 7f42d16fab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283d6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.960+0000 7f42d16fab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562e283e6120 con 0x562e283d2f10 2024-06-06T03:38:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.960+0000 7f42d16fab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562e283e67f0 con 0x562e283cced0 2024-06-06T03:38:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.960+0000 7f42d16fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562e283e6ec0 con 0x562e283d25f0 2024-06-06T03:38:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.961+0000 7f42c5414700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283d6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.961+0000 7f42c5414700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283d6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49812/0 (socket says 172.21.15.40:49812) 2024-06-06T03:38:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.961+0000 7f42c5414700 1 -- 172.21.15.40:0/3497611710 learned_addr learned my addr 172.21.15.40:0/3497611710 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.961+0000 7f42c4c13700 1 --2- 172.21.15.40:0/3497611710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283cced0 0x562e283cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.961+0000 7f42bffff700 1 --2- 172.21.15.40:0/3497611710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 0x562e283d3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.961+0000 7f42c5414700 1 -- 172.21.15.40:0/3497611710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 msgr2=0x562e283d3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.961+0000 7f42c5414700 1 --2- 172.21.15.40:0/3497611710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 0x562e283d3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.961+0000 7f42c5414700 1 -- 172.21.15.40:0/3497611710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283cced0 msgr2=0x562e283cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:25.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.961+0000 7f42c5414700 1 --2- 172.21.15.40:0/3497611710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283cced0 0x562e283cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.961+0000 7f42c5414700 1 -- 172.21.15.40:0/3497611710 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e283d6ac0 con 0x562e283d2f10 2024-06-06T03:38:25.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.961+0000 7f42c5414700 1 --2- 172.21.15.40:0/3497611710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283d6580 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f42b80098e0 tx=0x7f42b8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8601b7171d3206a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:25.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.961+0000 7f42b67fc700 1 -- 172.21.15.40:0/3497611710 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8034070 con 0x562e283d2f10 2024-06-06T03:38:25.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.961+0000 7f42b67fc700 1 -- 172.21.15.40:0/3497611710 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42b800b5c0 con 0x562e283d2f10 2024-06-06T03:38:25.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.961+0000 7f42b67fc700 1 -- 172.21.15.40:0/3497611710 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b800fae0 con 0x562e283d2f10 2024-06-06T03:38:25.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.962+0000 7f42d16fab80 1 -- 172.21.15.40:0/3497611710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 msgr2=0x562e283d6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:25.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.962+0000 7f42d16fab80 1 --2- 172.21.15.40:0/3497611710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283d6580 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f42b80098e0 tx=0x7f42b8009310 comp rx=0 tx=0).stop 2024-06-06T03:38:25.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.962+0000 7f42d16fab80 1 -- 172.21.15.40:0/3497611710 shutdown_connections 2024-06-06T03:38:25.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.962+0000 7f42d16fab80 1 --2- 172.21.15.40:0/3497611710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283d2f10 0x562e283d6580 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.962+0000 7f42d16fab80 1 --2- 172.21.15.40:0/3497611710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283cced0 0x562e283cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.962+0000 7f42d16fab80 1 --2- 172.21.15.40:0/3497611710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 0x562e283d3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.962+0000 7f42d16fab80 1 -- 172.21.15.40:0/3497611710 >> 172.21.15.40:0/3497611710 conn(0x562e282f7c70 msgr2=0x562e283d33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.962+0000 7f42d16fab80 1 -- 172.21.15.40:0/3497611710 shutdown_connections 2024-06-06T03:38:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.962+0000 7f42d16fab80 1 -- 172.21.15.40:0/3497611710 wait complete. 2024-06-06T03:38:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.962+0000 7f42d16fab80 1 -- start start 2024-06-06T03:38:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.962+0000 7f42d16fab80 1 Processor -- start 2024-06-06T03:38:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42d16fab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283cced0 0x562e283f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42d16fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 0x562e283ec210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42d16fab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283d2f10 0x562e283ec750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42d16fab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562e283e7c60 con 0x562e283cced0 2024-06-06T03:38:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42d16fab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562e283e7590 con 0x562e283d2f10 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42d16fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562e283e8330 con 0x562e283d25f0 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42d16fab80 10 client.0 _ll_register_callbacks cb 0x562e282b88d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42c4c13700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283cced0 0x562e283f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42c4c13700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283cced0 0x562e283f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49814/0 (socket says 172.21.15.40:49814) 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42c4c13700 1 -- 172.21.15.40:0/454447685 learned_addr learned my addr 172.21.15.40:0/454447685 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42bffff700 1 --2- 172.21.15.40:0/454447685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 0x562e283ec210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42c5414700 1 --2- 172.21.15.40:0/454447685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283d2f10 0x562e283ec750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42c4c13700 1 -- 172.21.15.40:0/454447685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 msgr2=0x562e283ec210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42c4c13700 1 --2- 172.21.15.40:0/454447685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 0x562e283ec210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42c4c13700 1 -- 172.21.15.40:0/454447685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283d2f10 msgr2=0x562e283ec750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42c4c13700 1 --2- 172.21.15.40:0/454447685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283d2f10 0x562e283ec750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42c4c13700 1 -- 172.21.15.40:0/454447685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e283ecd90 con 0x562e283cced0 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42c5414700 1 --2- 172.21.15.40:0/454447685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283d2f10 0x562e283ec750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f42c4c13700 1 --2- 172.21.15.40:0/454447685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283cced0 0x562e283f1750 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x562e283e8640 tx=0x7f42a8013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9e9ce06422983b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.963+0000 7f429ffff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:38:25.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.964+0000 7f429ffff700 1 -- 172.21.15.40:0/454447685 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a8011db0 con 0x562e283cced0 2024-06-06T03:38:25.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.964+0000 7f429ffff700 1 -- 172.21.15.40:0/454447685 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f42a8004830 con 0x562e283cced0 2024-06-06T03:38:25.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.964+0000 7f42d16fab80 1 -- 172.21.15.40:0/454447685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b800b5c0 con 0x562e283cced0 2024-06-06T03:38:25.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.964+0000 7f429ffff700 1 -- 172.21.15.40:0/454447685 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42a8004a30 con 0x562e283cced0 2024-06-06T03:38:25.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.964+0000 7f42d16fab80 10 client.15150 fetch_fsmap learned FSMap version 28 2024-06-06T03:38:25.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.964+0000 7f42d16fab80 1 -- 172.21.15.40:0/454447685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e28453090 con 0x562e283cced0 2024-06-06T03:38:25.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.964+0000 7f429ffff700 1 -- 172.21.15.40:0/454447685 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42b800b5c0 con 0x562e283cced0 2024-06-06T03:38:25.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.964+0000 7f429ffff700 1 -- 172.21.15.40:0/454447685 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f42a802e070 con 0x562e283cced0 2024-06-06T03:38:25.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.964+0000 7f42d16fab80 10 client.15150 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:38:25.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.964+0000 7f42d16fab80 10 client.15150 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T03:38:25.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.964+0000 7f42d16fab80 20 client.15150 populate_metadata read hostname 'smithi040' 2024-06-06T03:38:25.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.964+0000 7f42d16fab80 1 --2- 172.21.15.40:0/454447685 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x562e28457d70 0x562e28456a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:25.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.964+0000 7f42d16fab80 4 client.15150 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.964+0000 7f42d16fab80 1 -- 172.21.15.40:0/454447685 --> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] -- command(tid 0: {"prefix": "status"}) -- 0x562e283cd860 con 0x562e28457d70 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f42bffff700 1 --2- 172.21.15.40:0/454447685 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x562e28457d70 0x562e28456a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f42bffff700 1 --2- 172.21.15.40:0/454447685 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x562e28457d70 0x562e28456a80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8aab506f3fb066de server_cookie=d8b74c7c71234046 in_seq=0 out_seq=0 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f429ffff700 10 client.15150 ms_handle_connect on v2:172.21.15.104:6836/2498466212 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f429ffff700 1 -- 172.21.15.40:0/454447685 <== mds.1 v2:172.21.15.104:6836/2498466212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562e283cd860 con 0x562e28457d70 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f429ffff700 10 client.15150 handle_command_reply: tid=0 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f42d16fab80 1 client.15150 shutdown 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f42d16fab80 1 -- 172.21.15.40:0/454447685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283cced0 msgr2=0x562e283f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f42d16fab80 1 --2- 172.21.15.40:0/454447685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283cced0 0x562e283f1750 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x562e283e8640 tx=0x7f42a8013070 comp rx=0 tx=0).stop 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f42d16fab80 1 -- 172.21.15.40:0/454447685 shutdown_connections 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f42d16fab80 1 --2- 172.21.15.40:0/454447685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562e283d2f10 0x562e283ec750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f42d16fab80 1 --2- 172.21.15.40:0/454447685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562e283cced0 0x562e283f1750 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f42d16fab80 1 --2- 172.21.15.40:0/454447685 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x562e28457d70 0x562e28456a80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f42d16fab80 1 --2- 172.21.15.40:0/454447685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e283d25f0 0x562e283ec210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f42d16fab80 1 -- 172.21.15.40:0/454447685 >> 172.21.15.40:0/454447685 conn(0x562e282f7c70 msgr2=0x562e283d3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:25.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f429ffff700 0 client.15150 ms_handle_reset on v2:172.21.15.104:6836/2498466212 2024-06-06T03:38:25.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f42d16fab80 1 -- 172.21.15.40:0/454447685 shutdown_connections 2024-06-06T03:38:25.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.965+0000 7f42d16fab80 1 -- 172.21.15.40:0/454447685 wait complete. 2024-06-06T03:38:25.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.966+0000 7f42d16fab80 20 client.15150 trim_cache size 0 max 16384 2024-06-06T03:38:25.972 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:38:25.972 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:38:25.972 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 1, 2024-06-06T03:38:25.973 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24443, 2024-06-06T03:38:25.973 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:38:25.973 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:38:25.973 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:38:25.973 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 175.06834207099999, 2024-06-06T03:38:25.973 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 28, 2024-06-06T03:38:25.973 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 90, 2024-06-06T03:38:25.973 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 88, 2024-06-06T03:38:25.973 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 220.516820583 2024-06-06T03:38:25.973 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:38:25.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.966+0000 7f42c6e77700 1 -- 172.21.15.40:0/4071665283 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42ac0a2ca0 msgr2=0x7f42ac0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:25.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.966+0000 7f42c6e77700 1 --2- 172.21.15.40:0/4071665283 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42ac0a2ca0 0x7f42ac0a5160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f42a8004630 tx=0x7f42a800b3c0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.966+0000 7f42c6e77700 1 -- 172.21.15.40:0/4071665283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42c01027a0 msgr2=0x7f42c00fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:25.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.966+0000 7f42c6e77700 1 --2- 172.21.15.40:0/4071665283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42c01027a0 0x7f42c00fd3b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f42b000ba50 tx=0x7f42b000bd60 comp rx=0 tx=0).stop 2024-06-06T03:38:25.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.966+0000 7f42c6e77700 1 -- 172.21.15.40:0/4071665283 shutdown_connections 2024-06-06T03:38:25.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.966+0000 7f42c6e77700 1 --2- 172.21.15.40:0/4071665283 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42ac0a2ca0 0x7f42ac0a5160 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.966+0000 7f42c6e77700 1 --2- 172.21.15.40:0/4071665283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c0106750 0x7f42c00fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.966+0000 7f42c6e77700 1 --2- 172.21.15.40:0/4071665283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42c0101df0 0x7f42c00fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.966+0000 7f42c6e77700 1 --2- 172.21.15.40:0/4071665283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42c01027a0 0x7f42c00fd3b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:25.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.966+0000 7f42c6e77700 1 -- 172.21.15.40:0/4071665283 >> 172.21.15.40:0/4071665283 conn(0x7f42c0077a00 msgr2=0x7f42c0100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:25.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.966+0000 7f42c6e77700 1 -- 172.21.15.40:0/4071665283 shutdown_connections 2024-06-06T03:38:25.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:25.966+0000 7f42c6e77700 1 -- 172.21.15.40:0/4071665283 wait complete. 2024-06-06T03:38:26.485 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T03:38:26.735 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:27.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:27 smithi131 ceph-mon[36687]: pgmap v399: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:38:27.564 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:27 smithi040 ceph-mon[32220]: pgmap v399: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:38:27.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:27 smithi104 ceph-mon[36569]: pgmap v399: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T03:38:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.377+0000 7fa27816e700 1 -- 172.21.15.40:0/4281255921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa270107400 msgr2=0x7fa2701077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.377+0000 7fa27816e700 1 --2- 172.21.15.40:0/4281255921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa270107400 0x7fa2701077e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009990 tx=0x7fa26c009ca0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.377+0000 7fa27816e700 1 -- 172.21.15.40:0/4281255921 shutdown_connections 2024-06-06T03:38:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.377+0000 7fa27816e700 1 --2- 172.21.15.40:0/4281255921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa270101d50 0x7fa27010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.377+0000 7fa27816e700 1 --2- 172.21.15.40:0/4281255921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa270107400 0x7fa2701077e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.377+0000 7fa27816e700 1 --2- 172.21.15.40:0/4281255921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa270101390 0x7fa270101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.377+0000 7fa27816e700 1 -- 172.21.15.40:0/4281255921 >> 172.21.15.40:0/4281255921 conn(0x7fa270078340 msgr2=0x7fa270078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.377+0000 7fa27816e700 1 -- 172.21.15.40:0/4281255921 shutdown_connections 2024-06-06T03:38:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.377+0000 7fa27816e700 1 -- 172.21.15.40:0/4281255921 wait complete. 2024-06-06T03:38:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.378+0000 7fa27816e700 1 Processor -- start 2024-06-06T03:38:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.378+0000 7fa27816e700 1 -- start start 2024-06-06T03:38:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.378+0000 7fa27816e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa270101390 0x7fa27010ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.378+0000 7fa27816e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa270101d50 0x7fa27010f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.378+0000 7fa27816e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa270107400 0x7fa27010f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.378+0000 7fa27816e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa270112790 con 0x7fa270107400 2024-06-06T03:38:28.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.378+0000 7fa27816e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa270112610 con 0x7fa270101d50 2024-06-06T03:38:28.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.378+0000 7fa27816e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa270112910 con 0x7fa270101390 2024-06-06T03:38:28.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.378+0000 7fa27670b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa270107400 0x7fa27010f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.378+0000 7fa27670b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa270107400 0x7fa27010f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49826/0 (socket says 172.21.15.40:49826) 2024-06-06T03:38:28.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.378+0000 7fa27670b700 1 -- 172.21.15.40:0/3262746760 learned_addr learned my addr 172.21.15.40:0/3262746760 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:28.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.379+0000 7fa275709700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa270101d50 0x7fa27010f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.379+0000 7fa275f0a700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa270101390 0x7fa27010ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.379+0000 7fa27670b700 1 -- 172.21.15.40:0/3262746760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa270101390 msgr2=0x7fa27010ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:28.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.379+0000 7fa27670b700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa270101390 0x7fa27010ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.379+0000 7fa27670b700 1 -- 172.21.15.40:0/3262746760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa270101d50 msgr2=0x7fa27010f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:28.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.379+0000 7fa27670b700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa270101d50 0x7fa27010f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.379+0000 7fa27670b700 1 -- 172.21.15.40:0/3262746760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2701a70e0 con 0x7fa270107400 2024-06-06T03:38:28.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.379+0000 7fa275f0a700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa270101390 0x7fa27010ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:28.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.379+0000 7fa27670b700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa270107400 0x7fa27010f750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa268007d70 tx=0x7fa26800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:28.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.379+0000 7fa266ffd700 1 -- 172.21.15.40:0/3262746760 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268016020 con 0x7fa270107400 2024-06-06T03:38:28.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.379+0000 7fa266ffd700 1 -- 172.21.15.40:0/3262746760 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa26800ad40 con 0x7fa270107400 2024-06-06T03:38:28.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.379+0000 7fa27816e700 1 -- 172.21.15.40:0/3262746760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2701a73d0 con 0x7fa270107400 2024-06-06T03:38:28.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.379+0000 7fa266ffd700 1 -- 172.21.15.40:0/3262746760 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268011600 con 0x7fa270107400 2024-06-06T03:38:28.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.379+0000 7fa27816e700 1 -- 172.21.15.40:0/3262746760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2701a78c0 con 0x7fa270107400 2024-06-06T03:38:28.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.382+0000 7fa266ffd700 1 -- 172.21.15.40:0/3262746760 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa268019020 con 0x7fa270107400 2024-06-06T03:38:28.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.383+0000 7fa266ffd700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa25c0a2c20 0x7fa25c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.383+0000 7fa2829f1b80 1 Processor -- start 2024-06-06T03:38:28.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.383+0000 7fa275f0a700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa25c0a2c20 0x7fa25c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.383+0000 7fa266ffd700 1 -- 172.21.15.40:0/3262746760 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fa2680ceee0 con 0x7fa270107400 2024-06-06T03:38:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.383+0000 7fa2829f1b80 1 -- start start 2024-06-06T03:38:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa2829f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991eaf5f0 0x55c991eaf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa2829f1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaff10 0x55c991ea9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa2829f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991ea9ed0 0x55c991eaa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa2829f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c991dfb7c0 con 0x55c991eaf5f0 2024-06-06T03:38:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa2829f1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c991da36d0 con 0x55c991eaff10 2024-06-06T03:38:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa2829f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c991df7780 con 0x55c991ea9ed0 2024-06-06T03:38:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa275f0a700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa25c0a2c20 0x7fa25c0a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa26c006010 tx=0x7fa26c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa275f0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991eaf5f0 0x55c991eaf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa275f0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991eaf5f0 0x55c991eaf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49834/0 (socket says 172.21.15.40:49834) 2024-06-06T03:38:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa275f0a700 1 -- 172.21.15.40:0/714486736 learned_addr learned my addr 172.21.15.40:0/714486736 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa275709700 1 --2- 172.21.15.40:0/714486736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaff10 0x55c991ea9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa27670b700 1 --2- 172.21.15.40:0/714486736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991ea9ed0 0x55c991eaa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa275f0a700 1 -- 172.21.15.40:0/714486736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991ea9ed0 msgr2=0x55c991eaa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:28.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa275f0a700 1 --2- 172.21.15.40:0/714486736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991ea9ed0 0x55c991eaa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa275f0a700 1 -- 172.21.15.40:0/714486736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaff10 msgr2=0x55c991ea9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:28.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa275f0a700 1 --2- 172.21.15.40:0/714486736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaff10 0x55c991ea9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa275f0a700 1 -- 172.21.15.40:0/714486736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c991d99f00 con 0x55c991eaf5f0 2024-06-06T03:38:28.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa275f0a700 1 --2- 172.21.15.40:0/714486736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991eaf5f0 0x55c991eaf9d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00ba90 tx=0x7fa26c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92319e0fa676fba server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:28.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.384+0000 7fa25bfff700 1 -- 172.21.15.40:0/714486736 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c00f460 con 0x55c991eaf5f0 2024-06-06T03:38:28.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.385+0000 7fa25bfff700 1 -- 172.21.15.40:0/714486736 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa26c00fae0 con 0x55c991eaf5f0 2024-06-06T03:38:28.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.385+0000 7fa25bfff700 1 -- 172.21.15.40:0/714486736 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c024ab0 con 0x55c991eaf5f0 2024-06-06T03:38:28.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.385+0000 7fa2829f1b80 1 -- 172.21.15.40:0/714486736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991eaf5f0 msgr2=0x55c991eaf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:28.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.385+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/714486736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991eaf5f0 0x55c991eaf9d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00ba90 tx=0x7fa26c01f000 comp rx=0 tx=0).stop 2024-06-06T03:38:28.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.385+0000 7fa2829f1b80 1 -- 172.21.15.40:0/714486736 shutdown_connections 2024-06-06T03:38:28.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.385+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/714486736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991ea9ed0 0x55c991eaa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.385+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/714486736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991eaf5f0 0x55c991eaf9d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.385+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/714486736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaff10 0x55c991ea9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.385+0000 7fa2829f1b80 1 -- 172.21.15.40:0/714486736 >> 172.21.15.40:0/714486736 conn(0x55c991dd4c70 msgr2=0x55c991ea4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:28.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.385+0000 7fa2829f1b80 1 -- 172.21.15.40:0/714486736 shutdown_connections 2024-06-06T03:38:28.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.385+0000 7fa2829f1b80 1 -- 172.21.15.40:0/714486736 wait complete. 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.386+0000 7fa2829f1b80 1 -- start start 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.386+0000 7fa2829f1b80 1 Processor -- start 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.386+0000 7fa2829f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991eb5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.386+0000 7fa2829f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaf5f0 0x55c991eb2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.386+0000 7fa2829f1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaff10 0x55c991eb34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.386+0000 7fa2829f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c991eb6920 con 0x55c991ea9ed0 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.386+0000 7fa2829f1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c991d9bba0 con 0x55c991eaff10 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.386+0000 7fa2829f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c991eb6aa0 con 0x55c991eaf5f0 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.386+0000 7fa2829f1b80 10 client.0 _ll_register_callbacks cb 0x55c991d958d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.386+0000 7fa275f0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991eb5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.386+0000 7fa275f0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991eb5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49848/0 (socket says 172.21.15.40:49848) 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.386+0000 7fa275f0a700 1 -- 172.21.15.40:0/1152100198 learned_addr learned my addr 172.21.15.40:0/1152100198 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.386+0000 7fa275709700 1 --2- 172.21.15.40:0/1152100198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaf5f0 0x55c991eb2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa27670b700 1 --2- 172.21.15.40:0/1152100198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaff10 0x55c991eb34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa275f0a700 1 -- 172.21.15.40:0/1152100198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaf5f0 msgr2=0x55c991eb2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa275f0a700 1 --2- 172.21.15.40:0/1152100198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaf5f0 0x55c991eb2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa275f0a700 1 -- 172.21.15.40:0/1152100198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaff10 msgr2=0x55c991eb34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa275f0a700 1 --2- 172.21.15.40:0/1152100198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaff10 0x55c991eb34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa275f0a700 1 -- 172.21.15.40:0/1152100198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c991d4a880 con 0x55c991ea9ed0 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa27670b700 1 --2- 172.21.15.40:0/1152100198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaff10 0x55c991eb34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa275f0a700 1 --2- 172.21.15.40:0/1152100198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991eb5ca0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00ba60 tx=0x7fa26c0037e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6e8a3686af2f54 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa2597fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa2597fa700 1 -- 172.21.15.40:0/1152100198 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c034070 con 0x55c991ea9ed0 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa2829f1b80 1 -- 172.21.15.40:0/1152100198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa26c00fae0 con 0x55c991ea9ed0 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa2597fa700 1 -- 172.21.15.40:0/1152100198 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fa26c00f6d0 con 0x55c991ea9ed0 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa2597fa700 1 -- 172.21.15.40:0/1152100198 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa26c00f8d0 con 0x55c991ea9ed0 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa2829f1b80 10 client.15162 fetch_fsmap learned FSMap version 28 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.387+0000 7fa2829f1b80 1 -- 172.21.15.40:0/1152100198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c991ec1250 con 0x55c991ea9ed0 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.388+0000 7fa2597fa700 1 -- 172.21.15.40:0/1152100198 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa26c00fae0 con 0x55c991ea9ed0 2024-06-06T03:38:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.388+0000 7fa2597fa700 1 -- 172.21.15.40:0/1152100198 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa26c030900 con 0x55c991ea9ed0 2024-06-06T03:38:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.388+0000 7fa2829f1b80 10 client.15162 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:38:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.388+0000 7fa2829f1b80 10 client.15162 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T03:38:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.388+0000 7fa2829f1b80 20 client.15162 populate_metadata read hostname 'smithi040' 2024-06-06T03:38:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.388+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/1152100198 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c991ec25a0 0x55c991ec4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.388+0000 7fa2829f1b80 4 client.15162 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:38:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.388+0000 7fa2829f1b80 1 -- 172.21.15.40:0/1152100198 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c991e08cc0 con 0x55c991ec25a0 2024-06-06T03:38:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.388+0000 7fa275709700 1 --2- 172.21.15.40:0/1152100198 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c991ec25a0 0x55c991ec4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.388+0000 7fa2597fa700 10 client.15162 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:38:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.388+0000 7fa275709700 1 --2- 172.21.15.40:0/1152100198 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c991ec25a0 0x55c991ec4790 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40bd1084f61bdccd server_cookie=cb5a907b202015a2 in_seq=0 out_seq=0 2024-06-06T03:38:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.389+0000 7fa2597fa700 1 -- 172.21.15.40:0/1152100198 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c991e08cc0 con 0x55c991ec25a0 2024-06-06T03:38:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.389+0000 7fa2597fa700 10 client.15162 handle_command_reply: tid=0 2024-06-06T03:38:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.389+0000 7fa2829f1b80 1 client.15162 shutdown 2024-06-06T03:38:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.389+0000 7fa2829f1b80 1 -- 172.21.15.40:0/1152100198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 msgr2=0x55c991eb5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.389+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/1152100198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991eb5ca0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00ba60 tx=0x7fa26c0037e0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.390+0000 7fa2829f1b80 1 -- 172.21.15.40:0/1152100198 shutdown_connections 2024-06-06T03:38:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.390+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/1152100198 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c991ec25a0 0x55c991ec4790 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.390+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/1152100198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaff10 0x55c991eb34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.390+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/1152100198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991eb5ca0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.390+0000 7fa2597fa700 0 client.15162 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:38:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.390+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/1152100198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaf5f0 0x55c991eb2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.390+0000 7fa2829f1b80 1 -- 172.21.15.40:0/1152100198 >> 172.21.15.40:0/1152100198 conn(0x55c991dd4c70 msgr2=0x55c991ea8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.390+0000 7fa2829f1b80 1 -- 172.21.15.40:0/1152100198 shutdown_connections 2024-06-06T03:38:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.390+0000 7fa2829f1b80 1 -- 172.21.15.40:0/1152100198 wait complete. 2024-06-06T03:38:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.390+0000 7fa2829f1b80 20 client.15162 trim_cache size 0 max 16384 2024-06-06T03:38:28.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.411+0000 7fa2829f1b80 1 Processor -- start 2024-06-06T03:38:28.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.411+0000 7fa2829f1b80 1 -- start start 2024-06-06T03:38:28.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa2829f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991ea9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa2829f1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaf5f0 0x55c991eb0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa2829f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaff10 0x55c991ea49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa2829f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c991ec2ad0 con 0x55c991ea9ed0 2024-06-06T03:38:28.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa2829f1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c991ec31a0 con 0x55c991eaf5f0 2024-06-06T03:38:28.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa2829f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c991ec3870 con 0x55c991eaff10 2024-06-06T03:38:28.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa275f0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991ea9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa275f0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991ea9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49856/0 (socket says 172.21.15.40:49856) 2024-06-06T03:38:28.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa275f0a700 1 -- 172.21.15.40:0/3243352449 learned_addr learned my addr 172.21.15.40:0/3243352449 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:28.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa275709700 1 --2- 172.21.15.40:0/3243352449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaf5f0 0x55c991eb0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa275f0a700 1 -- 172.21.15.40:0/3243352449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaff10 msgr2=0x55c991ea49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:28.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa275f0a700 1 --2- 172.21.15.40:0/3243352449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaff10 0x55c991ea49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa275f0a700 1 -- 172.21.15.40:0/3243352449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaf5f0 msgr2=0x55c991eb0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:28.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa275f0a700 1 --2- 172.21.15.40:0/3243352449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaf5f0 0x55c991eb0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa275f0a700 1 -- 172.21.15.40:0/3243352449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c991eb3ed0 con 0x55c991ea9ed0 2024-06-06T03:38:28.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa275f0a700 1 --2- 172.21.15.40:0/3243352449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991ea9990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0098e0 tx=0x7fa26c00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8800075fd3ca5726 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:28.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.412+0000 7fa25b7fe700 1 -- 172.21.15.40:0/3243352449 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c034040 con 0x55c991ea9ed0 2024-06-06T03:38:28.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.413+0000 7fa25b7fe700 1 -- 172.21.15.40:0/3243352449 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa26c042020 con 0x55c991ea9ed0 2024-06-06T03:38:28.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.413+0000 7fa25b7fe700 1 -- 172.21.15.40:0/3243352449 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c017710 con 0x55c991ea9ed0 2024-06-06T03:38:28.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.413+0000 7fa2829f1b80 1 -- 172.21.15.40:0/3243352449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 msgr2=0x55c991ea9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:28.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.413+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/3243352449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991ea9990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0098e0 tx=0x7fa26c00bf80 comp rx=0 tx=0).stop 2024-06-06T03:38:28.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.413+0000 7fa2829f1b80 1 -- 172.21.15.40:0/3243352449 shutdown_connections 2024-06-06T03:38:28.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.413+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/3243352449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaff10 0x55c991ea49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.413+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/3243352449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991ea9990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.413+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/3243352449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaf5f0 0x55c991eb0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.413+0000 7fa2829f1b80 1 -- 172.21.15.40:0/3243352449 >> 172.21.15.40:0/3243352449 conn(0x55c991dd4c70 msgr2=0x55c991eb1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:28.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.413+0000 7fa2829f1b80 1 -- 172.21.15.40:0/3243352449 shutdown_connections 2024-06-06T03:38:28.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.413+0000 7fa2829f1b80 1 -- 172.21.15.40:0/3243352449 wait complete. 2024-06-06T03:38:28.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.414+0000 7fa2829f1b80 1 -- start start 2024-06-06T03:38:28.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.414+0000 7fa2829f1b80 1 Processor -- start 2024-06-06T03:38:28.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.414+0000 7fa2829f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991ecddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.414+0000 7fa2829f1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaf5f0 0x55c991ece330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.414+0000 7fa2829f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaff10 0x55c991ec8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.414+0000 7fa2829f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c991ec4610 con 0x55c991ea9ed0 2024-06-06T03:38:28.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.414+0000 7fa2829f1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c991ec3f40 con 0x55c991eaf5f0 2024-06-06T03:38:28.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.414+0000 7fa2829f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c991ec4ce0 con 0x55c991eaff10 2024-06-06T03:38:28.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.414+0000 7fa2829f1b80 10 client.0 _ll_register_callbacks cb 0x55c991d958d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:38:28.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.414+0000 7fa275f0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991ecddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa275709700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaf5f0 0x55c991ece330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa275f0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991ecddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49866/0 (socket says 172.21.15.40:49866) 2024-06-06T03:38:28.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa275f0a700 1 -- 172.21.15.40:0/2044144822 learned_addr learned my addr 172.21.15.40:0/2044144822 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:28.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa27670b700 1 --2- 172.21.15.40:0/2044144822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaff10 0x55c991ec8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa275f0a700 1 -- 172.21.15.40:0/2044144822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaff10 msgr2=0x55c991ec8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:28.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa275f0a700 1 --2- 172.21.15.40:0/2044144822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaff10 0x55c991ec8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa275f0a700 1 -- 172.21.15.40:0/2044144822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaf5f0 msgr2=0x55c991ece330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:38:28.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa275f0a700 1 --2- 172.21.15.40:0/2044144822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaf5f0 0x55c991ece330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa275f0a700 1 -- 172.21.15.40:0/2044144822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c991ec93d0 con 0x55c991ea9ed0 2024-06-06T03:38:28.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa27670b700 1 --2- 172.21.15.40:0/2044144822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaff10 0x55c991ec8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:28.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa275709700 1 --2- 172.21.15.40:0/2044144822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaf5f0 0x55c991ece330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:28.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa275f0a700 1 --2- 172.21.15.40:0/2044144822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991ecddf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0059b0 tx=0x7fa26c030da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a55abcdd3af24bf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:28.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa2597fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:38:28.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa2597fa700 1 -- 172.21.15.40:0/2044144822 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c04b070 con 0x55c991ea9ed0 2024-06-06T03:38:28.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa2829f1b80 1 -- 172.21.15.40:0/2044144822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa26c042020 con 0x55c991ea9ed0 2024-06-06T03:38:28.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa2597fa700 1 -- 172.21.15.40:0/2044144822 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fa26c043040 con 0x55c991ea9ed0 2024-06-06T03:38:28.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa2597fa700 1 -- 172.21.15.40:0/2044144822 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa26c057050 con 0x55c991ea9ed0 2024-06-06T03:38:28.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.415+0000 7fa2829f1b80 10 client.15168 fetch_fsmap learned FSMap version 28 2024-06-06T03:38:28.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.416+0000 7fa2829f1b80 1 -- 172.21.15.40:0/2044144822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c991f37ec0 con 0x55c991ea9ed0 2024-06-06T03:38:28.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.416+0000 7fa2597fa700 1 -- 172.21.15.40:0/2044144822 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa26c042020 con 0x55c991ea9ed0 2024-06-06T03:38:28.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.416+0000 7fa2597fa700 1 -- 172.21.15.40:0/2044144822 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa26c047830 con 0x55c991ea9ed0 2024-06-06T03:38:28.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.416+0000 7fa2829f1b80 10 client.15168 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T03:38:28.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.416+0000 7fa2829f1b80 10 client.15168 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T03:38:28.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.416+0000 7fa2829f1b80 20 client.15168 populate_metadata read hostname 'smithi040' 2024-06-06T03:38:28.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.416+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/2044144822 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c991ec25a0 0x55c991f3b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.416+0000 7fa2829f1b80 4 client.15168 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.416+0000 7fa2829f1b80 1 -- 172.21.15.40:0/2044144822 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "status"}) -- 0x55c991e03240 con 0x55c991ec25a0 2024-06-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.416+0000 7fa275709700 1 --2- 172.21.15.40:0/2044144822 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c991ec25a0 0x55c991f3b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2597fa700 10 client.15168 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa275709700 1 --2- 172.21.15.40:0/2044144822 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c991ec25a0 0x55c991f3b490 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b1b45320a4c5cf9 server_cookie=9e0236f14ec3e3ee in_seq=0 out_seq=0 2024-06-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2597fa700 1 -- 172.21.15.40:0/2044144822 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c991e03240 con 0x55c991ec25a0 2024-06-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2597fa700 10 client.15168 handle_command_reply: tid=0 2024-06-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2829f1b80 1 client.15168 shutdown 2024-06-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2829f1b80 1 -- 172.21.15.40:0/2044144822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 msgr2=0x55c991ecddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/2044144822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991ecddf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0059b0 tx=0x7fa26c030da0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2829f1b80 1 -- 172.21.15.40:0/2044144822 shutdown_connections 2024-06-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/2044144822 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c991ec25a0 0x55c991f3b490 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/2044144822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c991eaff10 0x55c991ec8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/2044144822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c991ea9ed0 0x55c991ecddf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2829f1b80 1 --2- 172.21.15.40:0/2044144822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c991eaf5f0 0x55c991ece330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2597fa700 0 client.15168 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2829f1b80 1 -- 172.21.15.40:0/2044144822 >> 172.21.15.40:0/2044144822 conn(0x55c991dd4c70 msgr2=0x55c991eb1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2829f1b80 1 -- 172.21.15.40:0/2044144822 shutdown_connections 2024-06-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.417+0000 7fa2829f1b80 1 -- 172.21.15.40:0/2044144822 wait complete. 2024-06-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.418+0000 7fa2829f1b80 20 client.15168 trim_cache size 0 max 16384 2024-06-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-06-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24470, 2024-06-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 191.55872730600001, 2024-06-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 28, 2024-06-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 90, 2024-06-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 88, 2024-06-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 208.863888183 2024-06-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.418+0000 7fa27816e700 1 -- 172.21.15.40:0/3262746760 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa25c0a2c20 msgr2=0x7fa25c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.418+0000 7fa27816e700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa25c0a2c20 0x7fa25c0a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa26c006010 tx=0x7fa26c00b540 comp rx=0 tx=0).stop 2024-06-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.418+0000 7fa27816e700 1 -- 172.21.15.40:0/3262746760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa270107400 msgr2=0x7fa27010f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.418+0000 7fa27816e700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa270107400 0x7fa27010f750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa268007d70 tx=0x7fa26800a340 comp rx=0 tx=0).stop 2024-06-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.418+0000 7fa27816e700 1 -- 172.21.15.40:0/3262746760 shutdown_connections 2024-06-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.418+0000 7fa27816e700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fa25c0a2c20 0x7fa25c0a50e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.418+0000 7fa27816e700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa270101390 0x7fa27010ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.418+0000 7fa27816e700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa270107400 0x7fa27010f750 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.418+0000 7fa27816e700 1 --2- 172.21.15.40:0/3262746760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa270101d50 0x7fa27010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:28.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.418+0000 7fa27816e700 1 -- 172.21.15.40:0/3262746760 >> 172.21.15.40:0/3262746760 conn(0x7fa270078340 msgr2=0x7fa2701000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:28.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.418+0000 7fa27816e700 1 -- 172.21.15.40:0/3262746760 shutdown_connections 2024-06-06T03:38:28.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:28.419+0000 7fa27816e700 1 -- 172.21.15.40:0/3262746760 wait complete. 2024-06-06T03:38:29.004 INFO:teuthology.run_tasks:Running task sequential... 2024-06-06T03:38:29.013 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-06T03:38:29.013 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-06T03:38:29.014 INFO:teuthology.task.print:Enabling mgr modules 2024-06-06T03:38:29.014 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-06-06T03:38:29.015 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T03:38:29.015 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi040.front.sepia.ceph.com 2024-06-06T03:38:29.015 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-06-06T03:38:29.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.125+0000 7fc295a76640 1 -- 172.21.15.40:0/626437302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc290101ba0 msgr2=0x7fc290102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:29.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.125+0000 7fc295a76640 1 --2- 172.21.15.40:0/626437302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc290101ba0 0x7fc290102020 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc284009900 tx=0x7fc28402a410 comp rx=0 tx=0).stop 2024-06-06T03:38:29.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.125+0000 7fc295a76640 1 -- 172.21.15.40:0/626437302 shutdown_connections 2024-06-06T03:38:29.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.125+0000 7fc295a76640 1 --2- 172.21.15.40:0/626437302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc290102560 0x7fc290108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:29.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.125+0000 7fc295a76640 1 --2- 172.21.15.40:0/626437302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc290101ba0 0x7fc290102020 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:29.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.125+0000 7fc295a76640 1 --2- 172.21.15.40:0/626437302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2901009b0 0x7fc290100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:29.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.125+0000 7fc295a76640 1 -- 172.21.15.40:0/626437302 >> 172.21.15.40:0/626437302 conn(0x7fc2900fbf20 msgr2=0x7fc2900fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:29.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.126+0000 7fc295a76640 1 -- 172.21.15.40:0/626437302 shutdown_connections 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.126+0000 7fc295a76640 1 -- 172.21.15.40:0/626437302 wait complete. 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.126+0000 7fc295a76640 1 Processor -- start 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.126+0000 7fc295a76640 1 -- start start 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.126+0000 7fc295a76640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2901009b0 0x7fc290197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.126+0000 7fc295a76640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc290101ba0 0x7fc290198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.126+0000 7fc295a76640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc290102560 0x7fc29019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.126+0000 7fc295a76640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc29010c5c0 con 0x7fc290101ba0 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.126+0000 7fc295a76640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc29010c440 con 0x7fc2901009b0 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.126+0000 7fc295a76640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc29010c740 con 0x7fc290102560 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc290101ba0 0x7fc290198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc290101ba0 0x7fc290198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35778/0 (socket says 172.21.15.40:35778) 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28effd640 1 -- 172.21.15.40:0/1935927932 learned_addr learned my addr 172.21.15.40:0/1935927932 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28ffff640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc290102560 0x7fc29019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28f7fe640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2901009b0 0x7fc290197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28effd640 1 -- 172.21.15.40:0/1935927932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc290102560 msgr2=0x7fc29019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28effd640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc290102560 0x7fc29019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28effd640 1 -- 172.21.15.40:0/1935927932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2901009b0 msgr2=0x7fc290197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28effd640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2901009b0 0x7fc290197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28effd640 1 -- 172.21.15.40:0/1935927932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29019f900 con 0x7fc290101ba0 2024-06-06T03:38:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28ffff640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc290102560 0x7fc29019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28f7fe640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2901009b0 0x7fc290197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28effd640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc290101ba0 0x7fc290198200 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc284006010 tx=0x7fc284031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28cff9640 1 -- 172.21.15.40:0/1935927932 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28403f030 con 0x7fc290101ba0 2024-06-06T03:38:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc295a76640 1 -- 172.21.15.40:0/1935927932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29019fb90 con 0x7fc290101ba0 2024-06-06T03:38:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28cff9640 1 -- 172.21.15.40:0/1935927932 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc284003ad0 con 0x7fc290101ba0 2024-06-06T03:38:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc28cff9640 1 -- 172.21.15.40:0/1935927932 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284035070 con 0x7fc290101ba0 2024-06-06T03:38:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.127+0000 7fc295a76640 1 -- 172.21.15.40:0/1935927932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2901a0080 con 0x7fc290101ba0 2024-06-06T03:38:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.128+0000 7fc295a76640 1 -- 172.21.15.40:0/1935927932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2901a0250 con 0x7fc290101ba0 2024-06-06T03:38:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.130+0000 7fc28cff9640 1 -- 172.21.15.40:0/1935927932 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc284042030 con 0x7fc290101ba0 2024-06-06T03:38:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.130+0000 7fc28cff9640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc2780a2c30 0x7fc2780a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.130+0000 7fc28f7fe640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc2780a2c30 0x7fc2780a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.130+0000 7fc28cff9640 1 -- 172.21.15.40:0/1935927932 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fc2840efb50 con 0x7fc290101ba0 2024-06-06T03:38:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.131+0000 7fc28f7fe640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc2780a2c30 0x7fc2780a50f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc280005fd0 tx=0x7fc280005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:29.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.134+0000 7fc28cff9640 1 -- 172.21.15.40:0/1935927932 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc2840bcb70 con 0x7fc290101ba0 2024-06-06T03:38:29.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:29 smithi040 ceph-mon[32220]: pgmap v400: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:38:29.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:29.370+0000 7fc295a76640 1 -- 172.21.15.40:0/1935927932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fc2901a05e0 con 0x7fc290101ba0 2024-06-06T03:38:29.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:29 smithi131 ceph-mon[36687]: pgmap v400: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:38:29.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:29 smithi104 ceph-mon[36569]: pgmap v400: 97 pgs: 97 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:38:30.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:30.181+0000 7fc28cff9640 1 -- 172.21.15.40:0/1935927932 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fc2840bc470 con 0x7fc290101ba0 2024-06-06T03:38:30.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:30.184+0000 7fc295a76640 1 -- 172.21.15.40:0/1935927932 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc2780a2c30 msgr2=0x7fc2780a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:30.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:30.184+0000 7fc295a76640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc2780a2c30 0x7fc2780a50f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc280005fd0 tx=0x7fc280005850 comp rx=0 tx=0).stop 2024-06-06T03:38:30.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:30.184+0000 7fc295a76640 1 -- 172.21.15.40:0/1935927932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc290101ba0 msgr2=0x7fc290198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:30.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:30.184+0000 7fc295a76640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc290101ba0 0x7fc290198200 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc284006010 tx=0x7fc284031b60 comp rx=0 tx=0).stop 2024-06-06T03:38:30.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:30.185+0000 7fc295a76640 1 -- 172.21.15.40:0/1935927932 shutdown_connections 2024-06-06T03:38:30.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:30.185+0000 7fc295a76640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc2780a2c30 0x7fc2780a50f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:30.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:30.185+0000 7fc295a76640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc290102560 0x7fc29019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:30.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:30.185+0000 7fc295a76640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc290101ba0 0x7fc290198200 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:30.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:30.185+0000 7fc295a76640 1 --2- 172.21.15.40:0/1935927932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2901009b0 0x7fc290197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:30.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:30.185+0000 7fc295a76640 1 -- 172.21.15.40:0/1935927932 >> 172.21.15.40:0/1935927932 conn(0x7fc2900fbf20 msgr2=0x7fc2900fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:30.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:30.185+0000 7fc295a76640 1 -- 172.21.15.40:0/1935927932 shutdown_connections 2024-06-06T03:38:30.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:30.185+0000 7fc295a76640 1 -- 172.21.15.40:0/1935927932 wait complete. 2024-06-06T03:38:30.202 INFO:teuthology.run_tasks:Running task kclient... 2024-06-06T03:38:30.227 INFO:tasks.kclient:Mounting kernel clients... 2024-06-06T03:38:30.227 INFO:tasks.kclient:config is {'client.0': {}} 2024-06-06T03:38:30.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T03:38:30.228 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-06-06T03:38:30.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T03:38:30.246 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-06-06T03:38:30.314 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2024-06-06T03:38:30.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:38:30.316 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-06-06T03:38:30.316 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-06T03:38:30.316 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-06T03:38:30.316 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-06T03:38:30.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:30 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1935927932' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-06T03:38:30.532 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:30 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: ignoring --setuser ceph since I am not root 2024-06-06T03:38:30.532 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:30 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: ignoring --setgroup ceph since I am not root 2024-06-06T03:38:30.532 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:30 smithi131 ceph-mgr[38491]: -- 172.21.15.131:0/3815623431 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c6d74c85a0 con 0x55c6d749c800 2024-06-06T03:38:30.532 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:30 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:30.364+0000 7f8d8d068200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T03:38:30.542 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:30 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1935927932' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-06T03:38:30.542 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:30 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: ignoring --setuser ceph since I am not root 2024-06-06T03:38:30.543 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:30 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: ignoring --setgroup ceph since I am not root 2024-06-06T03:38:30.543 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:30 smithi104 ceph-mgr[37947]: -- 172.21.15.104:0/283234554 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559d729a45a0 con 0x559d72978c00 2024-06-06T03:38:30.543 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:30 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:30.360+0000 7f413419b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T03:38:30.543 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:30 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:30.542+0000 7f413419b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:38:30.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:30 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1935927932' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-06T03:38:30.580 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:30 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: ignoring --setuser ceph since I am not root 2024-06-06T03:38:30.580 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:30 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: ignoring --setgroup ceph since I am not root 2024-06-06T03:38:30.580 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:30 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:30.376+0000 7f6880605200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T03:38:30.617 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:30.840 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:30 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:30.645+0000 7f413419b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T03:38:30.853 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:30 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:30.584+0000 7f6880605200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:38:30.853 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:30 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:30.704+0000 7f6880605200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T03:38:30.957 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:30 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:30.547+0000 7f8d8d068200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:38:30.957 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:30 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:30.649+0000 7f8d8d068200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T03:38:31.130 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:31 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:31.040+0000 7f6880605200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T03:38:31.131 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:31 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:31.130+0000 7f6880605200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T03:38:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:31 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1935927932' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-06T03:38:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:31 smithi131 ceph-mon[36687]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-06-06T03:38:31.282 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:30 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:30.956+0000 7f8d8d068200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T03:38:31.282 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:31 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:31.043+0000 7f8d8d068200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T03:38:31.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:31 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1935927932' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-06T03:38:31.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:31 smithi104 ceph-mon[36569]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-06-06T03:38:31.340 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:30 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:30.954+0000 7f413419b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T03:38:31.340 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:31 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:31.040+0000 7f413419b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T03:38:31.508 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:31 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1935927932' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-06T03:38:31.508 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:31 smithi040 ceph-mon[32220]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-06-06T03:38:32.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.388+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/2817397321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf406aa00 msgr2=0x7f0cf406ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:32.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.388+0000 7f0cfa7ac700 1 --2- 172.21.15.40:0/2817397321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf406aa00 0x7f0cf406ae60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009990 tx=0x7f0ce8009ca0 comp rx=0 tx=0).stop 2024-06-06T03:38:32.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.388+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/2817397321 shutdown_connections 2024-06-06T03:38:32.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.388+0000 7f0cfa7ac700 1 --2- 172.21.15.40:0/2817397321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf41084d0 0x7f0cf41088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:32.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.388+0000 7f0cfa7ac700 1 --2- 172.21.15.40:0/2817397321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cf406b3a0 0x7f0cf410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:32.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.388+0000 7f0cfa7ac700 1 --2- 172.21.15.40:0/2817397321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf406aa00 0x7f0cf406ae60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:32.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.388+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/2817397321 >> 172.21.15.40:0/2817397321 conn(0x7f0cf4078330 msgr2=0x7f0cf4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:32.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.388+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/2817397321 shutdown_connections 2024-06-06T03:38:32.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.388+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/2817397321 wait complete. 2024-06-06T03:38:32.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.388+0000 7f0cfa7ac700 1 Processor -- start 2024-06-06T03:38:32.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cfa7ac700 1 -- start start 2024-06-06T03:38:32.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cfa7ac700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cf406aa00 0x7f0cf4101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:32.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cfa7ac700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf406b3a0 0x7f0cf4101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:32.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cfa7ac700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf41084d0 0x7f0cf4101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:32.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cfa7ac700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf4112730 con 0x7f0cf406b3a0 2024-06-06T03:38:32.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cfa7ac700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41125b0 con 0x7f0cf406aa00 2024-06-06T03:38:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cfa7ac700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41128b0 con 0x7f0cf41084d0 2024-06-06T03:38:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cf37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf406b3a0 0x7f0cf4101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cf37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf406b3a0 0x7f0cf4101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35802/0 (socket says 172.21.15.40:35802) 2024-06-06T03:38:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cf37fe700 1 -- 172.21.15.40:0/469896248 learned_addr learned my addr 172.21.15.40:0/469896248 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cf8d49700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf41084d0 0x7f0cf4101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cf3fff700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cf406aa00 0x7f0cf4101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cf3fff700 1 -- 172.21.15.40:0/469896248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf41084d0 msgr2=0x7f0cf4101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cf3fff700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf41084d0 0x7f0cf4101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cf3fff700 1 -- 172.21.15.40:0/469896248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf406b3a0 msgr2=0x7f0cf4101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cf3fff700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf406b3a0 0x7f0cf4101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cf3fff700 1 -- 172.21.15.40:0/469896248 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf41a7100 con 0x7f0cf406aa00 2024-06-06T03:38:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.389+0000 7f0cf8d49700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf41084d0 0x7f0cf4101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.390+0000 7f0cf3fff700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cf406aa00 0x7f0cf4101450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400e9f0 tx=0x7f0ce400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.390+0000 7f0cf17fa700 1 -- 172.21.15.40:0/469896248 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4004220 con 0x7f0cf406aa00 2024-06-06T03:38:32.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.390+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/469896248 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf41a73f0 con 0x7f0cf406aa00 2024-06-06T03:38:32.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.390+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/469896248 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf41a7930 con 0x7f0cf406aa00 2024-06-06T03:38:32.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.390+0000 7f0cf17fa700 1 -- 172.21.15.40:0/469896248 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce4004d10 con 0x7f0cf406aa00 2024-06-06T03:38:32.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.390+0000 7f0cf17fa700 1 -- 172.21.15.40:0/469896248 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4005020 con 0x7f0cf406aa00 2024-06-06T03:38:32.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.391+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/469896248 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf4103b80 con 0x7f0cf406aa00 2024-06-06T03:38:32.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.392+0000 7f0cf17fa700 1 -- 172.21.15.40:0/469896248 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f0ce4042af0 con 0x7f0cf406aa00 2024-06-06T03:38:32.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.393+0000 7f0cf17fa700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0cdc0a2cf0 0x7f0cdc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:32.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.393+0000 7f0cf37fe700 1 -- 172.21.15.40:0/469896248 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0cdc0a2cf0 msgr2=0x7f0cdc0a51b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/869712339 2024-06-06T03:38:32.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.393+0000 7f0cf17fa700 1 -- 172.21.15.40:0/469896248 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f0ce40cee30 con 0x7f0cf406aa00 2024-06-06T03:38:32.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.393+0000 7f0cf37fe700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0cdc0a2cf0 0x7f0cdc0a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T03:38:32.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.396+0000 7f0cf17fa700 1 -- 172.21.15.40:0/469896248 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0ce401f3a0 con 0x7f0cf406aa00 2024-06-06T03:38:32.531 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:32 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:32.266+0000 7f8d8d068200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T03:38:32.532 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:32 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:32.440+0000 7f8d8d068200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T03:38:32.552 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:32 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:32.214+0000 7f413419b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T03:38:32.552 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:32 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:32.383+0000 7f413419b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T03:38:32.552 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:32 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:32.552+0000 7f413419b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T03:38:32.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.594+0000 7f0cf37fe700 1 -- 172.21.15.40:0/469896248 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0cdc0a2cf0 msgr2=0x7f0cdc0a51b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/869712339 2024-06-06T03:38:32.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.594+0000 7f0cf37fe700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0cdc0a2cf0 0x7f0cdc0a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T03:38:32.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.652+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/469896248 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0cf4102930 con 0x7f0cf406aa00 2024-06-06T03:38:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.652+0000 7f0cf17fa700 1 -- 172.21.15.40:0/469896248 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f0ce409bdb0 con 0x7f0cf406aa00 2024-06-06T03:38:32.652 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-06-06T03:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.655+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/469896248 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0cdc0a2cf0 msgr2=0x7f0cdc0a51b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.655+0000 7f0cfa7ac700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0cdc0a2cf0 0x7f0cdc0a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.655+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/469896248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cf406aa00 msgr2=0x7f0cf4101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.655+0000 7f0cfa7ac700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cf406aa00 0x7f0cf4101450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400e9f0 tx=0x7f0ce400ed00 comp rx=0 tx=0).stop 2024-06-06T03:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.655+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/469896248 shutdown_connections 2024-06-06T03:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.655+0000 7f0cfa7ac700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f0cdc0a2cf0 0x7f0cdc0a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.655+0000 7f0cfa7ac700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf41084d0 0x7f0cf4101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.655+0000 7f0cfa7ac700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cf406aa00 0x7f0cf4101450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.655+0000 7f0cfa7ac700 1 --2- 172.21.15.40:0/469896248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf406b3a0 0x7f0cf4101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.655+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/469896248 >> 172.21.15.40:0/469896248 conn(0x7f0cf4078330 msgr2=0x7f0cf40ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.655+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/469896248 shutdown_connections 2024-06-06T03:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:32.655+0000 7f0cfa7ac700 1 -- 172.21.15.40:0/469896248 wait complete. 2024-06-06T03:38:32.667 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:32 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:32.412+0000 7f6880605200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T03:38:32.667 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:32 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:32.593+0000 7f6880605200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T03:38:32.839 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:32 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/469896248' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T03:38:32.903 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:32 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/469896248' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T03:38:32.904 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:32 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:32.615+0000 7f8d8d068200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T03:38:32.942 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:32 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/469896248' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T03:38:32.943 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:32 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:32.775+0000 7f6880605200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T03:38:33.090 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:32 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:32.840+0000 7f413419b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T03:38:33.090 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:32 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:32.932+0000 7f413419b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T03:38:33.199 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:32 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:32.903+0000 7f8d8d068200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T03:38:33.199 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:32 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:32.996+0000 7f8d8d068200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T03:38:33.213 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:33 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:33.084+0000 7f6880605200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T03:38:33.213 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:33 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:33.183+0000 7f6880605200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T03:38:33.214 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-06T03:38:33.214 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-06T03:38:33.214 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-06-06T03:38:33.214 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-06T03:38:33.214 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-06T03:38:33.214 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-06T03:38:33.214 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-06T03:38:33.215 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-06T03:38:33.215 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-06T03:38:33.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T03:38:33.215 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-06-06T03:38:33.232 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-06T03:38:33.232 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-06T03:38:33.233 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-06-06T03:38:33.233 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-06-06T03:38:33.233 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-06-06T03:38:33.233 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-06-06T03:38:33.233 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T03:38:33.233 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-06-06T03:38:33.233 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-06-06T03:38:33.233 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T03:38:33.233 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-06-06T03:38:33.233 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-06-06T03:38:33.233 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-06T03:38:33.233 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-06-06T03:38:33.233 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-06-06T03:38:33.233 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-06-06T03:38:33.234 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42019sec preferred_lft 42019sec 2024-06-06T03:38:33.234 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-06-06T03:38:33.234 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-06-06T03:38:33.234 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T03:38:33.234 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-06-06T03:38:33.234 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-06-06T03:38:33.234 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-06T03:38:33.234 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T03:38:33.235 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-06-06T03:38:33.235 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2024-06-06T03:38:33.235 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2024-06-06T03:38:33.235 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2024-06-06T03:38:33.235 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-06T03:38:33.235 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-06-06T03:38:33.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T03:38:33.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T03:38:33.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T03:38:33.450 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-06T03:38:33.465 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:33 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:33.401+0000 7f6880605200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T03:38:33.476 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-06-06T03:38:33.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T03:38:33.479 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-06-06T03:38:33.531 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:33 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:33.199+0000 7f8d8d068200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T03:38:33.532 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:33 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:33.391+0000 7f8d8d068200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:38:33.554 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-06-06T03:38:33.554 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-06T03:38:33.555 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-06T03:38:33.555 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-06T03:38:33.555 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-06T03:38:33.555 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T03:38:33.556 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-06-06T03:38:33.556 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-06T03:38:33.556 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-06T03:38:33.556 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-06T03:38:33.556 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-06-06T03:38:33.590 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:33 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:33.135+0000 7f413419b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T03:38:33.590 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:33 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:33.326+0000 7f413419b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:38:33.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T03:38:33.743 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:33 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:33.619+0000 7f6880605200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T03:38:33.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T03:38:33.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T03:38:33.842 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-06-06T03:38:33.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T03:38:33.861 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2024-06-06T03:38:33.917 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T03:38:33.917 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-06-06T03:38:33.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T03:38:33.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-06T03:38:33.917 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-06-06T03:38:34.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T03:38:34.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T03:38:34.040 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-06T03:38:34.040 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T03:38:34.040 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-06-06T03:38:34.041 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-06T03:38:34.041 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-06T03:38:34.041 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-06T03:38:34.041 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-06T03:38:34.041 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-06T03:38:34.041 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-06-06T03:38:34.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T03:38:34.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T03:38:34.180 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T03:38:34.180 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-06-06T03:38:34.180 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-06-06T03:38:34.180 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2024-06-06T03:38:34.180 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-06-06T03:38:34.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T03:38:34.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T03:38:34.262 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T03:38:34.262 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-06T03:38:34.279 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-06T03:38:34.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T03:38:34.281 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-06T03:38:34.282 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:33 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:33.991+0000 7f8d8d068200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T03:38:34.336 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-06-06T03:38:34.336 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-06-06T03:38:34.336 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/5cc6513e-156c-4d04-8f86-bd2c9b139dd9 2024-06-06T03:38:34.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T03:38:34.337 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/5cc6513e-156c-4d04-8f86-bd2c9b139dd9 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-06-06T03:38:34.340 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:33 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:33.923+0000 7f413419b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T03:38:34.440 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-06-06T03:38:34.440 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-06-06T03:38:34.566 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:34 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:34.291+0000 7f6880605200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T03:38:34.587 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-06-06T03:38:34.588 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-06-06T03:38:34.588 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.104:3300,172.21.15.131:3300" 2024-06-06T03:38:34.588 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@77c25d68-23b4-11ef-bc9b-c7b262605968.cephfs=/volumes/qa/sv_1/5cc6513e-156c-4d04-8f86-bd2c9b139dd9 2024-06-06T03:38:34.588 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.40:3300/172.21.15.104:3300/172.21.15.131:3300" will pass to kernel 2024-06-06T03:38:34.589 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-06-06T03:38:34.589 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-06-06T03:38:34.589 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-06-06T03:38:34.589 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-06-06T03:38:34.589 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-06-06T03:38:34.590 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-06T03:38:34.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T03:38:34.590 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-06T03:38:34.621 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-06T03:38:34.643 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:34 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:34.345+0000 7f413419b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T03:38:34.643 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:34 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:34.441+0000 7f413419b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T03:38:34.643 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:34 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:34.643+0000 7f413419b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T03:38:34.704 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:34 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:34.409+0000 7f8d8d068200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T03:38:34.704 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:34 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:34.505+0000 7f8d8d068200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T03:38:34.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-06T03:38:34.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T03:38:34.720 INFO:teuthology.orchestra.run.smithi040.stdout:client24911 2024-06-06T03:38:34.720 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:38:34.720 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T03:38:34.739 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-06T03:38:34.830 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:34 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:34.757+0000 7f6880605200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T03:38:34.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-06T03:38:34.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T03:38:34.839 INFO:teuthology.orchestra.run.smithi040.stdout:client24911 2024-06-06T03:38:34.839 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-06T03:38:34.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-06T03:38:34.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T03:38:34.897 INFO:teuthology.orchestra.run.smithi040.stdout:client24911 2024-06-06T03:38:34.898 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968.client24911/status') 2024-06-06T03:38:34.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T03:38:34.960 INFO:teuthology.orchestra.run.smithi040.stdout:instance: client.24911 (3)172.21.15.40:0/211787955 2024-06-06T03:38:34.960 INFO:teuthology.orchestra.run.smithi040.stdout:blocklisted: false 2024-06-06T03:38:34.960 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2024-06-06T03:38:34.960 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2024-06-06T03:38:34.960 INFO:teuthology.orchestra.run.smithi040.stderr:70 bytes copied, 2.5088e-05 s, 2.8 MB/s 2024-06-06T03:38:34.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T03:38:34.967 INFO:teuthology.run_tasks:Running task exec... 2024-06-06T03:38:34.976 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T03:38:34.976 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi040.front.sepia.ceph.com 2024-06-06T03:38:34.976 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-06-06T03:38:35.032 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:34 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:34.704+0000 7f8d8d068200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T03:38:35.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.085+0000 7f42b13c8640 1 -- 172.21.15.40:0/2179309578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42ac100900 msgr2=0x7f42ac100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:35.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.085+0000 7f42b13c8640 1 --2- 172.21.15.40:0/2179309578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42ac100900 0x7f42ac100d20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f429c009900 tx=0x7f429c02a410 comp rx=0 tx=0).stop 2024-06-06T03:38:35.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.085+0000 7f42b13c8640 1 -- 172.21.15.40:0/2179309578 shutdown_connections 2024-06-06T03:38:35.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.085+0000 7f42b13c8640 1 --2- 172.21.15.40:0/2179309578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42ac1024b0 0x7f42ac108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.085+0000 7f42b13c8640 1 --2- 172.21.15.40:0/2179309578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42ac101af0 0x7f42ac101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.085+0000 7f42b13c8640 1 --2- 172.21.15.40:0/2179309578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42ac100900 0x7f42ac100d20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.085+0000 7f42b13c8640 1 -- 172.21.15.40:0/2179309578 >> 172.21.15.40:0/2179309578 conn(0x7f42ac0fbe90 msgr2=0x7f42ac0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.085+0000 7f42b13c8640 1 -- 172.21.15.40:0/2179309578 shutdown_connections 2024-06-06T03:38:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.085+0000 7f42b13c8640 1 -- 172.21.15.40:0/2179309578 wait complete. 2024-06-06T03:38:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.086+0000 7f42b13c8640 1 Processor -- start 2024-06-06T03:38:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.086+0000 7f42b13c8640 1 -- start start 2024-06-06T03:38:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.086+0000 7f42b13c8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42ac100900 0x7f42ac197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.086+0000 7f42b13c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42ac101af0 0x7f42ac1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.086+0000 7f42b13c8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42ac1024b0 0x7f42ac19f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.086+0000 7f42b13c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac10c570 con 0x7f42ac101af0 2024-06-06T03:38:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.086+0000 7f42b13c8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac10c3f0 con 0x7f42ac100900 2024-06-06T03:38:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.086+0000 7f42b13c8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac10c6f0 con 0x7f42ac1024b0 2024-06-06T03:38:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.086+0000 7f42a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42ac101af0 0x7f42ac1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.086+0000 7f42aaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42ac100900 0x7f42ac197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.086+0000 7f42a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42ac101af0 0x7f42ac1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35820/0 (socket says 172.21.15.40:35820) 2024-06-06T03:38:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.086+0000 7f42a3fff640 1 -- 172.21.15.40:0/1246820273 learned_addr learned my addr 172.21.15.40:0/1246820273 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.086+0000 7f42ab7fe640 1 --2- 172.21.15.40:0/1246820273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42ac1024b0 0x7f42ac19f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.087+0000 7f42aaffd640 1 -- 172.21.15.40:0/1246820273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42ac1024b0 msgr2=0x7f42ac19f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.087+0000 7f42aaffd640 1 --2- 172.21.15.40:0/1246820273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42ac1024b0 0x7f42ac19f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.087+0000 7f42aaffd640 1 -- 172.21.15.40:0/1246820273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42ac101af0 msgr2=0x7f42ac1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.087+0000 7f42aaffd640 1 --2- 172.21.15.40:0/1246820273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42ac101af0 0x7f42ac1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.087+0000 7f42aaffd640 1 -- 172.21.15.40:0/1246820273 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac19f8b0 con 0x7f42ac100900 2024-06-06T03:38:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.087+0000 7f42a3fff640 1 --2- 172.21.15.40:0/1246820273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42ac101af0 0x7f42ac1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.087+0000 7f42aaffd640 1 --2- 172.21.15.40:0/1246820273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42ac100900 0x7f42ac197c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f429c009230 tx=0x7f429c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.087+0000 7f42a8ff9640 1 -- 172.21.15.40:0/1246820273 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c03f070 con 0x7f42ac100900 2024-06-06T03:38:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.087+0000 7f42b13c8640 1 -- 172.21.15.40:0/1246820273 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ac19fb40 con 0x7f42ac100900 2024-06-06T03:38:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.087+0000 7f42a8ff9640 1 -- 172.21.15.40:0/1246820273 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f429c003cd0 con 0x7f42ac100900 2024-06-06T03:38:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.087+0000 7f42a8ff9640 1 -- 172.21.15.40:0/1246820273 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c035070 con 0x7f42ac100900 2024-06-06T03:38:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.087+0000 7f42b13c8640 1 -- 172.21.15.40:0/1246820273 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ac1a0050 con 0x7f42ac100900 2024-06-06T03:38:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.088+0000 7f42b13c8640 1 -- 172.21.15.40:0/1246820273 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42ac1a0220 con 0x7f42ac100900 2024-06-06T03:38:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.089+0000 7f42a8ff9640 1 -- 172.21.15.40:0/1246820273 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f429c03b460 con 0x7f42ac100900 2024-06-06T03:38:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.090+0000 7f42a8ff9640 1 --2- 172.21.15.40:0/1246820273 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42980a2c80 0x7f42980a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.090+0000 7f42a3fff640 1 -- 172.21.15.40:0/1246820273 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42980a2c80 msgr2=0x7f42980a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/869712339 2024-06-06T03:38:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.090+0000 7f42a8ff9640 1 -- 172.21.15.40:0/1246820273 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f429c0ef620 con 0x7f42ac100900 2024-06-06T03:38:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.090+0000 7f42a3fff640 1 --2- 172.21.15.40:0/1246820273 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42980a2c80 0x7f42980a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T03:38:35.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.093+0000 7f42a8ff9640 1 -- 172.21.15.40:0/1246820273 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f429c0bbf50 con 0x7f42ac100900 2024-06-06T03:38:35.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.291+0000 7f42a3fff640 1 -- 172.21.15.40:0/1246820273 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42980a2c80 msgr2=0x7f42980a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/869712339 2024-06-06T03:38:35.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.291+0000 7f42a3fff640 1 --2- 172.21.15.40:0/1246820273 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42980a2c80 0x7f42980a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T03:38:35.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.321+0000 7f42b13c8640 1 -- 172.21.15.40:0/1246820273 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f42ac1a03f0 con 0x7f42ac100900 2024-06-06T03:38:35.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:34 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:34.863+0000 7f6880605200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T03:38:35.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:35 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:35.086+0000 7f6880605200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T03:38:35.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.377+0000 7f42a8ff9640 1 -- 172.21.15.40:0/1246820273 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7f429c0c0250 con 0x7f42ac100900 2024-06-06T03:38:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.379+0000 7f42b13c8640 1 -- 172.21.15.40:0/1246820273 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42980a2c80 msgr2=0x7f42980a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:38:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.379+0000 7f42b13c8640 1 --2- 172.21.15.40:0/1246820273 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42980a2c80 0x7f42980a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.379+0000 7f42b13c8640 1 -- 172.21.15.40:0/1246820273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42ac100900 msgr2=0x7f42ac197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.379+0000 7f42b13c8640 1 --2- 172.21.15.40:0/1246820273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42ac100900 0x7f42ac197c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f429c009230 tx=0x7f429c004270 comp rx=0 tx=0).stop 2024-06-06T03:38:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.379+0000 7f42b13c8640 1 -- 172.21.15.40:0/1246820273 shutdown_connections 2024-06-06T03:38:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.379+0000 7f42b13c8640 1 --2- 172.21.15.40:0/1246820273 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f42980a2c80 0x7f42980a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.379+0000 7f42b13c8640 1 --2- 172.21.15.40:0/1246820273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42ac1024b0 0x7f42ac19f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.379+0000 7f42b13c8640 1 --2- 172.21.15.40:0/1246820273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42ac101af0 0x7f42ac1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.379+0000 7f42b13c8640 1 --2- 172.21.15.40:0/1246820273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42ac100900 0x7f42ac197c70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.379+0000 7f42b13c8640 1 -- 172.21.15.40:0/1246820273 >> 172.21.15.40:0/1246820273 conn(0x7f42ac0fbe90 msgr2=0x7f42ac0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.380+0000 7f42b13c8640 1 -- 172.21.15.40:0/1246820273 shutdown_connections 2024-06-06T03:38:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.380+0000 7f42b13c8640 1 -- 172.21.15.40:0/1246820273 wait complete. 2024-06-06T03:38:35.392 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-06T03:38:35.403 INFO:teuthology.run_tasks:Running task exec... 2024-06-06T03:38:35.457 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T03:38:35.457 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi040.front.sepia.ceph.com 2024-06-06T03:38:35.458 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-06-06T03:38:35.499 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:35 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1246820273' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-06T03:38:35.499 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:35 smithi104 ceph-mon[36569]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-06T03:38:35.499 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:35 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:35.164+0000 7f413419b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T03:38:35.499 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:35 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:35.254+0000 7f413419b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T03:38:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:35 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1246820273' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-06T03:38:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:35 smithi131 ceph-mon[36687]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-06T03:38:35.532 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:35.229+0000 7f8d8d068200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T03:38:35.532 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:35.322+0000 7f8d8d068200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T03:38:35.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.564+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3867090923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c8104e60 msgr2=0x7fe3c8107250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:35.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.564+0000 7fe3cd2e7640 1 --2- 172.21.15.40:0/3867090923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c8104e60 0x7fe3c8107250 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009900 tx=0x7fe3bc02a410 comp rx=0 tx=0).stop 2024-06-06T03:38:35.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.564+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3867090923 shutdown_connections 2024-06-06T03:38:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.564+0000 7fe3cd2e7640 1 --2- 172.21.15.40:0/3867090923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3c8107790 0x7fe3c8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.564+0000 7fe3cd2e7640 1 --2- 172.21.15.40:0/3867090923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c8104e60 0x7fe3c8107250 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.564+0000 7fe3cd2e7640 1 --2- 172.21.15.40:0/3867090923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3c8102530 0x7fe3c8104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.564+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3867090923 >> 172.21.15.40:0/3867090923 conn(0x7fe3c80fbf20 msgr2=0x7fe3c80fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.564+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3867090923 shutdown_connections 2024-06-06T03:38:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.564+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3867090923 wait complete. 2024-06-06T03:38:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.565+0000 7fe3cd2e7640 1 Processor -- start 2024-06-06T03:38:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.565+0000 7fe3cd2e7640 1 -- start start 2024-06-06T03:38:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.565+0000 7fe3cd2e7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3c8102530 0x7fe3c8101ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.565+0000 7fe3cd2e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c8104e60 0x7fe3c81004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.565+0000 7fe3cd2e7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3c8107790 0x7fe3c8100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.565+0000 7fe3cd2e7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c810d6d0 con 0x7fe3c8104e60 2024-06-06T03:38:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.565+0000 7fe3cd2e7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c810d550 con 0x7fe3c8102530 2024-06-06T03:38:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.565+0000 7fe3cd2e7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c810d850 con 0x7fe3c8107790 2024-06-06T03:38:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.565+0000 7fe3c67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c8104e60 0x7fe3c81004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.565+0000 7fe3c67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c8104e60 0x7fe3c81004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35840/0 (socket says 172.21.15.40:35840) 2024-06-06T03:38:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.565+0000 7fe3c67fc640 1 -- 172.21.15.40:0/3739237784 learned_addr learned my addr 172.21.15.40:0/3739237784 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.565+0000 7fe3c6ffd640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3c8102530 0x7fe3c8101ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.566+0000 7fe3c77fe640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3c8107790 0x7fe3c8100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.566+0000 7fe3c67fc640 1 -- 172.21.15.40:0/3739237784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3c8107790 msgr2=0x7fe3c8100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.566+0000 7fe3c67fc640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3c8107790 0x7fe3c8100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.566+0000 7fe3c67fc640 1 -- 172.21.15.40:0/3739237784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3c8102530 msgr2=0x7fe3c8101ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.566+0000 7fe3c67fc640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3c8102530 0x7fe3c8101ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.566+0000 7fe3c67fc640 1 -- 172.21.15.40:0/3739237784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c81012b0 con 0x7fe3c8104e60 2024-06-06T03:38:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.566+0000 7fe3c6ffd640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3c8102530 0x7fe3c8101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.566+0000 7fe3c67fc640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c8104e60 0x7fe3c81004e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc031610 tx=0x7fe3bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.566+0000 7fe3affff640 1 -- 172.21.15.40:0/3739237784 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc03f030 con 0x7fe3c8104e60 2024-06-06T03:38:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.566+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3739237784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c81a1d40 con 0x7fe3c8104e60 2024-06-06T03:38:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.566+0000 7fe3affff640 1 -- 172.21.15.40:0/3739237784 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3bc003ad0 con 0x7fe3c8104e60 2024-06-06T03:38:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.566+0000 7fe3affff640 1 -- 172.21.15.40:0/3739237784 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc035070 con 0x7fe3c8104e60 2024-06-06T03:38:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.566+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3739237784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c81a2220 con 0x7fe3c8104e60 2024-06-06T03:38:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.567+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3739237784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c80fda10 con 0x7fe3c8104e60 2024-06-06T03:38:35.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.569+0000 7fe3affff640 1 -- 172.21.15.40:0/3739237784 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fe3bc042030 con 0x7fe3c8104e60 2024-06-06T03:38:35.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.569+0000 7fe3affff640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe3b00a2c80 0x7fe3b00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:35.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.569+0000 7fe3c6ffd640 1 -- 172.21.15.40:0/3739237784 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe3b00a2c80 msgr2=0x7fe3b00a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/869712339 2024-06-06T03:38:35.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.569+0000 7fe3affff640 1 -- 172.21.15.40:0/3739237784 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fe3bc0ef330 con 0x7fe3c8104e60 2024-06-06T03:38:35.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.569+0000 7fe3c6ffd640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe3b00a2c80 0x7fe3b00a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T03:38:35.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.578+0000 7fe3affff640 1 -- 172.21.15.40:0/3739237784 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe3bc0bbd00 con 0x7fe3c8104e60 2024-06-06T03:38:35.662 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:35 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1246820273' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-06T03:38:35.662 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:35 smithi040 ceph-mon[32220]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-06T03:38:35.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.770+0000 7fe3c6ffd640 1 -- 172.21.15.40:0/3739237784 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe3b00a2c80 msgr2=0x7fe3b00a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/869712339 2024-06-06T03:38:35.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.770+0000 7fe3c6ffd640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe3b00a2c80 0x7fe3b00a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T03:38:35.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.797+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3739237784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fe3c80625a0 con 0x7fe3c8104e60 2024-06-06T03:38:35.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.801+0000 7fe3affff640 1 -- 172.21.15.40:0/3739237784 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v30)=0 v30) ==== 129+0+0 (secure 0 0 0) 0x7fe3bc0c0000 con 0x7fe3c8104e60 2024-06-06T03:38:35.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.803+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3739237784 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe3b00a2c80 msgr2=0x7fe3b00a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:38:35.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.803+0000 7fe3cd2e7640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe3b00a2c80 0x7fe3b00a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.803+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3739237784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c8104e60 msgr2=0x7fe3c81004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:35.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.803+0000 7fe3cd2e7640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c8104e60 0x7fe3c81004e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc031610 tx=0x7fe3bc004270 comp rx=0 tx=0).stop 2024-06-06T03:38:35.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.804+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3739237784 shutdown_connections 2024-06-06T03:38:35.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.804+0000 7fe3cd2e7640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fe3b00a2c80 0x7fe3b00a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.804+0000 7fe3cd2e7640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3c8107790 0x7fe3c8100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.804+0000 7fe3cd2e7640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c8104e60 0x7fe3c81004e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.804+0000 7fe3cd2e7640 1 --2- 172.21.15.40:0/3739237784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3c8102530 0x7fe3c8101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.804+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3739237784 >> 172.21.15.40:0/3739237784 conn(0x7fe3c80fbf20 msgr2=0x7fe3c8102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:35.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.804+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3739237784 shutdown_connections 2024-06-06T03:38:35.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.804+0000 7fe3cd2e7640 1 -- 172.21.15.40:0/3739237784 wait complete. 2024-06-06T03:38:35.815 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-06-06T03:38:35.840 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:35 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:35.499+0000 7f413419b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T03:38:35.840 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:35 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:35.632+0000 7f413419b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T03:38:35.840 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:35 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:35.727+0000 7f413419b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T03:38:35.914 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:35.568+0000 7f8d8d068200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T03:38:35.914 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:35.700+0000 7f8d8d068200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T03:38:35.914 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:35.795+0000 7f8d8d068200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T03:38:35.915 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:35.914+0000 7f8d8d068200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T03:38:35.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.922+0000 7f1e09177640 1 -- 172.21.15.40:0/2328060148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e041024b0 msgr2=0x7f1e04108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:35.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.922+0000 7f1e09177640 1 --2- 172.21.15.40:0/2328060148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e041024b0 0x7f1e04108d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009ca0 tx=0x7f1dfc02a8c0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.922+0000 7f1e09177640 1 -- 172.21.15.40:0/2328060148 shutdown_connections 2024-06-06T03:38:35.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.922+0000 7f1e09177640 1 --2- 172.21.15.40:0/2328060148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e041024b0 0x7f1e04108d40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.922+0000 7f1e09177640 1 --2- 172.21.15.40:0/2328060148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e04101af0 0x7f1e04101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.922+0000 7f1e09177640 1 --2- 172.21.15.40:0/2328060148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e04100900 0x7f1e04100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.922+0000 7f1e09177640 1 -- 172.21.15.40:0/2328060148 >> 172.21.15.40:0/2328060148 conn(0x7f1e040fbe70 msgr2=0x7f1e040fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:35.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.922+0000 7f1e09177640 1 -- 172.21.15.40:0/2328060148 shutdown_connections 2024-06-06T03:38:35.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.922+0000 7f1e09177640 1 -- 172.21.15.40:0/2328060148 wait complete. 2024-06-06T03:38:35.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.923+0000 7f1e09177640 1 Processor -- start 2024-06-06T03:38:35.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.923+0000 7f1e09177640 1 -- start start 2024-06-06T03:38:35.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.923+0000 7f1e09177640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e04100900 0x7f1e04104890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:35.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.923+0000 7f1e09177640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e04101af0 0x7f1e04102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:35.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.923+0000 7f1e09177640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e041024b0 0x7f1e04103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:35.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.923+0000 7f1e09177640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0410c560 con 0x7f1e04101af0 2024-06-06T03:38:35.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.923+0000 7f1e09177640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0410c3e0 con 0x7f1e04100900 2024-06-06T03:38:35.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.923+0000 7f1e09177640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0410c6e0 con 0x7f1e041024b0 2024-06-06T03:38:35.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1e0259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e04101af0 0x7f1e04102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:35.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1e0259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e04101af0 0x7f1e04102ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35854/0 (socket says 172.21.15.40:35854) 2024-06-06T03:38:35.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1e0259c640 1 -- 172.21.15.40:0/3522908767 learned_addr learned my addr 172.21.15.40:0/3522908767 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:35.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1e02d9d640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e04100900 0x7f1e04104890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:35.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1e0359e640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e041024b0 0x7f1e04103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:35.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1e0259c640 1 -- 172.21.15.40:0/3522908767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e041024b0 msgr2=0x7f1e04103410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:35.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1e0259c640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e041024b0 0x7f1e04103410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1e0259c640 1 -- 172.21.15.40:0/3522908767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e04100900 msgr2=0x7f1e04104890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:35.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1e0259c640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e04100900 0x7f1e04104890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:35.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1e0259c640 1 -- 172.21.15.40:0/3522908767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e04103ca0 con 0x7f1e04101af0 2024-06-06T03:38:35.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1e02d9d640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e04100900 0x7f1e04104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:35.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1e0259c640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e04101af0 0x7f1e04102ed0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9f0 tx=0x7f1df800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:35.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1debfff640 1 -- 172.21.15.40:0/3522908767 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df800cd80 con 0x7f1e04101af0 2024-06-06T03:38:35.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1debfff640 1 -- 172.21.15.40:0/3522908767 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1df8010910 con 0x7f1e04101af0 2024-06-06T03:38:35.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1e09177640 1 -- 172.21.15.40:0/3522908767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e041a1d80 con 0x7f1e04101af0 2024-06-06T03:38:35.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1debfff640 1 -- 172.21.15.40:0/3522908767 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8018990 con 0x7f1e04101af0 2024-06-06T03:38:35.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.924+0000 7f1e09177640 1 -- 172.21.15.40:0/3522908767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e041a2210 con 0x7f1e04101af0 2024-06-06T03:38:35.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.925+0000 7f1e09177640 1 -- 172.21.15.40:0/3522908767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e041a24a0 con 0x7f1e04101af0 2024-06-06T03:38:35.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.927+0000 7f1debfff640 1 -- 172.21.15.40:0/3522908767 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f1df8010ac0 con 0x7f1e04101af0 2024-06-06T03:38:35.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.927+0000 7f1debfff640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1dec0a2c80 0x7f1dec0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:35.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.927+0000 7f1e02d9d640 1 -- 172.21.15.40:0/3522908767 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1dec0a2c80 msgr2=0x7f1dec0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/869712339 2024-06-06T03:38:35.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.928+0000 7f1debfff640 1 -- 172.21.15.40:0/3522908767 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f1df80cf6b0 con 0x7f1e04101af0 2024-06-06T03:38:35.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.928+0000 7f1e02d9d640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1dec0a2c80 0x7f1dec0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T03:38:35.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:35.935+0000 7f1debfff640 1 -- 172.21.15.40:0/3522908767 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1df809c080 con 0x7f1e04101af0 2024-06-06T03:38:36.043 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:35 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:35.662+0000 7f6880605200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T03:38:36.043 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:35 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:35.767+0000 7f6880605200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T03:38:36.043 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:36 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:36.043+0000 7f6880605200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T03:38:36.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.123+0000 7f1e09177640 1 -- 172.21.15.40:0/3522908767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f1e041a2670 con 0x7f1e04101af0 2024-06-06T03:38:36.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.127+0000 7f1debfff640 1 -- 172.21.15.40:0/3522908767 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7f1df80a0380 con 0x7f1e04101af0 2024-06-06T03:38:36.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.129+0000 7f1e02d9d640 1 -- 172.21.15.40:0/3522908767 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1dec0a2c80 msgr2=0x7f1dec0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/869712339 2024-06-06T03:38:36.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.129+0000 7f1e02d9d640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1dec0a2c80 0x7f1dec0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T03:38:36.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.129+0000 7f1e09177640 1 -- 172.21.15.40:0/3522908767 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1dec0a2c80 msgr2=0x7f1dec0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:38:36.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.129+0000 7f1e09177640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1dec0a2c80 0x7f1dec0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.129+0000 7f1e09177640 1 -- 172.21.15.40:0/3522908767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e04101af0 msgr2=0x7f1e04102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:36.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.129+0000 7f1e09177640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e04101af0 0x7f1e04102ed0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9f0 tx=0x7f1df800eeb0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.129+0000 7f1e09177640 1 -- 172.21.15.40:0/3522908767 shutdown_connections 2024-06-06T03:38:36.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.129+0000 7f1e09177640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f1dec0a2c80 0x7f1dec0a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.129+0000 7f1e09177640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e041024b0 0x7f1e04103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.129+0000 7f1e09177640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e04101af0 0x7f1e04102ed0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.129+0000 7f1e09177640 1 --2- 172.21.15.40:0/3522908767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e04100900 0x7f1e04104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.129+0000 7f1e09177640 1 -- 172.21.15.40:0/3522908767 >> 172.21.15.40:0/3522908767 conn(0x7f1e040fbe70 msgr2=0x7f1e040fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:36.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.130+0000 7f1e09177640 1 -- 172.21.15.40:0/3522908767 shutdown_connections 2024-06-06T03:38:36.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.130+0000 7f1e09177640 1 -- 172.21.15.40:0/3522908767 wait complete. 2024-06-06T03:38:36.140 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-06-06T03:38:36.143 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:35 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:35.843+0000 7f413419b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T03:38:36.143 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:36 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:36.143+0000 7f413419b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T03:38:36.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.247+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/3747132433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5f01024b0 msgr2=0x7fc5f0108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:36.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.247+0000 7fc5f4eb2640 1 --2- 172.21.15.40:0/3747132433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5f01024b0 0x7fc5f0108d40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009ca0 tx=0x7fc5e802a8c0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.247+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/3747132433 shutdown_connections 2024-06-06T03:38:36.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.247+0000 7fc5f4eb2640 1 --2- 172.21.15.40:0/3747132433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5f01024b0 0x7fc5f0108d40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.247+0000 7fc5f4eb2640 1 --2- 172.21.15.40:0/3747132433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f0101af0 0x7fc5f0101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.247+0000 7fc5f4eb2640 1 --2- 172.21.15.40:0/3747132433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5f0100900 0x7fc5f0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.247+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/3747132433 >> 172.21.15.40:0/3747132433 conn(0x7fc5f00fbe70 msgr2=0x7fc5f00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:36.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.247+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/3747132433 shutdown_connections 2024-06-06T03:38:36.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.247+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/3747132433 wait complete. 2024-06-06T03:38:36.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.248+0000 7fc5f4eb2640 1 Processor -- start 2024-06-06T03:38:36.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.248+0000 7fc5f4eb2640 1 -- start start 2024-06-06T03:38:36.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.248+0000 7fc5f4eb2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f0100900 0x7fc5f0069bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.248+0000 7fc5f4eb2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5f0101af0 0x7fc5f019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.248+0000 7fc5f4eb2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5f01024b0 0x7fc5f019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.248+0000 7fc5f4eb2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f010c580 con 0x7fc5f01024b0 2024-06-06T03:38:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.248+0000 7fc5f4eb2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f010c400 con 0x7fc5f0101af0 2024-06-06T03:38:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.248+0000 7fc5f4eb2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f010c700 con 0x7fc5f0100900 2024-06-06T03:38:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.248+0000 7fc5e7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5f0101af0 0x7fc5f019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5eed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5f01024b0 0x7fc5f019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5ee59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f0100900 0x7fc5f0069bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5e7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5f0101af0 0x7fc5f019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:46990/0 (socket says 172.21.15.40:46990) 2024-06-06T03:38:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5e7fff640 1 -- 172.21.15.40:0/1693012838 learned_addr learned my addr 172.21.15.40:0/1693012838 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5e7fff640 1 -- 172.21.15.40:0/1693012838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f0100900 msgr2=0x7fc5f0069bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5e7fff640 1 --2- 172.21.15.40:0/1693012838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f0100900 0x7fc5f0069bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5e7fff640 1 -- 172.21.15.40:0/1693012838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5f01024b0 msgr2=0x7fc5f019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5e7fff640 1 --2- 172.21.15.40:0/1693012838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5f01024b0 0x7fc5f019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5e7fff640 1 -- 172.21.15.40:0/1693012838 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f019f8b0 con 0x7fc5f0101af0 2024-06-06T03:38:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5e7fff640 1 --2- 172.21.15.40:0/1693012838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5f0101af0 0x7fc5f019c980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800e9f0 tx=0x7fc5d800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5e77fe640 1 -- 172.21.15.40:0/1693012838 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800cd80 con 0x7fc5f0101af0 2024-06-06T03:38:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5e77fe640 1 -- 172.21.15.40:0/1693012838 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d8005500 con 0x7fc5f0101af0 2024-06-06T03:38:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/1693012838 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f019fb40 con 0x7fc5f0101af0 2024-06-06T03:38:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5e77fe640 1 -- 172.21.15.40:0/1693012838 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8010690 con 0x7fc5f0101af0 2024-06-06T03:38:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.249+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/1693012838 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f01a0040 con 0x7fc5f0101af0 2024-06-06T03:38:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.250+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/1693012838 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f0107bc0 con 0x7fc5f0101af0 2024-06-06T03:38:36.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.251+0000 7fc5e77fe640 1 -- 172.21.15.40:0/1693012838 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fc5d80046c0 con 0x7fc5f0101af0 2024-06-06T03:38:36.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.252+0000 7fc5e77fe640 1 --2- 172.21.15.40:0/1693012838 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc5dc0a2c80 0x7fc5dc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:36.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.252+0000 7fc5ee59c640 1 -- 172.21.15.40:0/1693012838 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc5dc0a2c80 msgr2=0x7fc5dc0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/869712339 2024-06-06T03:38:36.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.252+0000 7fc5e77fe640 1 -- 172.21.15.40:0/1693012838 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7fc5d80cf460 con 0x7fc5f0101af0 2024-06-06T03:38:36.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.252+0000 7fc5ee59c640 1 --2- 172.21.15.40:0/1693012838 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc5dc0a2c80 0x7fc5dc0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T03:38:36.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.255+0000 7fc5e77fe640 1 -- 172.21.15.40:0/1693012838 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc5d809c3e0 con 0x7fc5f0101af0 2024-06-06T03:38:36.295 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:36 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:36.190+0000 7f6880605200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T03:38:36.295 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:36 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:36.295+0000 7f6880605200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T03:38:36.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.446+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/1693012838 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fc5f002d2e0 con 0x7fc5f0101af0 2024-06-06T03:38:36.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.453+0000 7fc5ee59c640 1 -- 172.21.15.40:0/1693012838 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc5dc0a2c80 msgr2=0x7fc5dc0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/869712339 2024-06-06T03:38:36.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.453+0000 7fc5ee59c640 1 --2- 172.21.15.40:0/1693012838 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc5dc0a2c80 0x7fc5dc0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T03:38:36.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.453+0000 7fc5e77fe640 1 -- 172.21.15.40:0/1693012838 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7fc5d809bce0 con 0x7fc5f0101af0 2024-06-06T03:38:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.455+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/1693012838 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc5dc0a2c80 msgr2=0x7fc5dc0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:38:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.455+0000 7fc5f4eb2640 1 --2- 172.21.15.40:0/1693012838 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc5dc0a2c80 0x7fc5dc0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.455+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/1693012838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5f0101af0 msgr2=0x7fc5f019c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.455+0000 7fc5f4eb2640 1 --2- 172.21.15.40:0/1693012838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5f0101af0 0x7fc5f019c980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800e9f0 tx=0x7fc5d800edf0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.455+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/1693012838 shutdown_connections 2024-06-06T03:38:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.455+0000 7fc5f4eb2640 1 --2- 172.21.15.40:0/1693012838 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7fc5dc0a2c80 0x7fc5dc0a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.455+0000 7fc5f4eb2640 1 --2- 172.21.15.40:0/1693012838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5f01024b0 0x7fc5f019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.455+0000 7fc5f4eb2640 1 --2- 172.21.15.40:0/1693012838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5f0101af0 0x7fc5f019c980 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.455+0000 7fc5f4eb2640 1 --2- 172.21.15.40:0/1693012838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f0100900 0x7fc5f0069bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.455+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/1693012838 >> 172.21.15.40:0/1693012838 conn(0x7fc5f00fbe70 msgr2=0x7fc5f0104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.455+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/1693012838 shutdown_connections 2024-06-06T03:38:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.455+0000 7fc5f4eb2640 1 -- 172.21.15.40:0/1693012838 wait complete. 2024-06-06T03:38:36.466 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-06-06T03:38:36.532 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:36 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:36.218+0000 7f8d8d068200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T03:38:36.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:36 smithi131 ceph-mon[36687]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-06T03:38:36.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:36 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:38:36.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:36 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3739237784' entity='client.admin' 2024-06-06T03:38:36.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:36 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3522908767' entity='client.admin' 2024-06-06T03:38:36.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.576+0000 7f5819f63640 1 -- 172.21.15.40:0/1644090345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5814101570 msgr2=0x7f58141019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:36.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.576+0000 7f5819f63640 1 --2- 172.21.15.40:0/1644090345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5814101570 0x7f58141019f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f580c009ca0 tx=0x7f580c02a8c0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.576+0000 7f5819f63640 1 -- 172.21.15.40:0/1644090345 shutdown_connections 2024-06-06T03:38:36.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.576+0000 7f5819f63640 1 --2- 172.21.15.40:0/1644090345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5814101570 0x7f58141019f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.576+0000 7f5819f63640 1 --2- 172.21.15.40:0/1644090345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5814100bd0 0x7f5814101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.576+0000 7f5819f63640 1 --2- 172.21.15.40:0/1644090345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5814100270 0x7f5814100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.576+0000 7f5819f63640 1 -- 172.21.15.40:0/1644090345 >> 172.21.15.40:0/1644090345 conn(0x7f58140fbe70 msgr2=0x7f58140fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:36.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.576+0000 7f5819f63640 1 -- 172.21.15.40:0/1644090345 shutdown_connections 2024-06-06T03:38:36.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.576+0000 7f5819f63640 1 -- 172.21.15.40:0/1644090345 wait complete. 2024-06-06T03:38:36.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.577+0000 7f5819f63640 1 Processor -- start 2024-06-06T03:38:36.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.577+0000 7f5819f63640 1 -- start start 2024-06-06T03:38:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.577+0000 7f5819f63640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5814100270 0x7f5814199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.577+0000 7f5819f63640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5814100bd0 0x7f581419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.577+0000 7f5819f63640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5814101570 0x7f58141a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.577+0000 7f5819f63640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f581410f850 con 0x7f5814100270 2024-06-06T03:38:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.577+0000 7f5819f63640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f581410f6d0 con 0x7f5814101570 2024-06-06T03:38:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.577+0000 7f5819f63640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f581410f9d0 con 0x7f5814100bd0 2024-06-06T03:38:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f58137fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5814100270 0x7f5814199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f58137fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5814100270 0x7f5814199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35880/0 (socket says 172.21.15.40:35880) 2024-06-06T03:38:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f5812ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5814100bd0 0x7f581419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f58137fe640 1 -- 172.21.15.40:0/2733733436 learned_addr learned my addr 172.21.15.40:0/2733733436 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f5813fff640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5814101570 0x7f58141a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f5812ffd640 1 -- 172.21.15.40:0/2733733436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5814101570 msgr2=0x7f58141a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f5812ffd640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5814101570 0x7f58141a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f5812ffd640 1 -- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5814100270 msgr2=0x7f5814199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f5812ffd640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5814100270 0x7f5814199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f5812ffd640 1 -- 172.21.15.40:0/2733733436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58141a19f0 con 0x7f5814100bd0 2024-06-06T03:38:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f5813fff640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5814101570 0x7f58141a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f5812ffd640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5814100bd0 0x7f581419a310 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00e9f0 tx=0x7f57fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f5810ff9640 1 -- 172.21.15.40:0/2733733436 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc00cd80 con 0x7f5814100bd0 2024-06-06T03:38:36.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f5819f63640 1 -- 172.21.15.40:0/2733733436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58141a1ce0 con 0x7f5814100bd0 2024-06-06T03:38:36.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f5810ff9640 1 -- 172.21.15.40:0/2733733436 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57fc0035f0 con 0x7f5814100bd0 2024-06-06T03:38:36.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.578+0000 7f5810ff9640 1 -- 172.21.15.40:0/2733733436 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc010670 con 0x7f5814100bd0 2024-06-06T03:38:36.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.579+0000 7f5819f63640 1 -- 172.21.15.40:0/2733733436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58141a21d0 con 0x7f5814100bd0 2024-06-06T03:38:36.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.579+0000 7f5819f63640 1 -- 172.21.15.40:0/2733733436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f581410afc0 con 0x7f5814100bd0 2024-06-06T03:38:36.581 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:36 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:36.421+0000 7f6880605200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T03:38:36.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:36 smithi040 ceph-mon[32220]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-06T03:38:36.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:36 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:38:36.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:36 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3739237784' entity='client.admin' 2024-06-06T03:38:36.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:36 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3522908767' entity='client.admin' 2024-06-06T03:38:36.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.582+0000 7f5810ff9640 1 -- 172.21.15.40:0/2733733436 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f57fc0037a0 con 0x7f5814100bd0 2024-06-06T03:38:36.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.582+0000 7f5810ff9640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58000a2c80 0x7f58000a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:36.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.582+0000 7f58137fe640 1 -- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58000a2c80 msgr2=0x7f58000a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/869712339 2024-06-06T03:38:36.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.582+0000 7f5810ff9640 1 -- 172.21.15.40:0/2733733436 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8124+0+0 (secure 0 0 0) 0x7f57fc0cf3e0 con 0x7f5814100bd0 2024-06-06T03:38:36.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.582+0000 7f58137fe640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58000a2c80 0x7f58000a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T03:38:36.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.589+0000 7f5810ff9640 1 -- 172.21.15.40:0/2733733436 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f57fc09c3f0 con 0x7f5814100bd0 2024-06-06T03:38:36.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:36 smithi104 ceph-mon[36569]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-06T03:38:36.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:36 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:38:36.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:36 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3739237784' entity='client.admin' 2024-06-06T03:38:36.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:36 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3522908767' entity='client.admin' 2024-06-06T03:38:36.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.783+0000 7f58137fe640 1 -- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58000a2c80 msgr2=0x7f58000a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/869712339 2024-06-06T03:38:36.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.783+0000 7f58137fe640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58000a2c80 0x7f58000a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T03:38:36.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:36.797+0000 7f5819f63640 1 -- 172.21.15.40:0/2733733436 --> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f5814061de0 con 0x7f58000a2c80 2024-06-06T03:38:37.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:36 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:36.740+0000 7f6880605200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T03:38:37.090 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:36 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:36.743+0000 7f413419b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T03:38:37.090 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:38:36 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y[37943]: 2024-06-06T03:38:36.961+0000 7f413419b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T03:38:37.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:37.184+0000 7f58137fe640 1 -- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58000a2c80 msgr2=0x7f58000a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/869712339 2024-06-06T03:38:37.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:37.184+0000 7f58137fe640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58000a2c80 0x7f58000a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-06T03:38:37.282 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:36 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:36.827+0000 7f8d8d068200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T03:38:37.282 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:38:37 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z[38459]: 2024-06-06T03:38:37.044+0000 7f8d8d068200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T03:38:37.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:37.464+0000 7f5810ff9640 1 -- 172.21.15.40:0/2733733436 <== mon.1 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 20) ==== 137850+0+0 (secure 0 0 0) 0x7f57fc098360 con 0x7f5814100bd0 2024-06-06T03:38:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:37.636+0000 7f5810ff9640 1 -- 172.21.15.40:0/2733733436 <== mon.1 v2:172.21.15.131:3300/0 8 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f57fc0ccf80 con 0x7f5814100bd0 2024-06-06T03:38:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:37.637+0000 7f5810ff9640 1 -- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58000a2c80 msgr2=0x7f58000a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:38:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:37.637+0000 7f5810ff9640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:6800/869712339,v1:172.21.15.40:6801/869712339] conn(0x7f58000a2c80 0x7f58000a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:37.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:37 smithi131 ceph-mon[36687]: from='client.? ' entity='client.admin' 2024-06-06T03:38:37.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:37 smithi131 ceph-mon[36687]: Standby manager daemon y restarted 2024-06-06T03:38:37.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:37 smithi131 ceph-mon[36687]: Standby manager daemon y started 2024-06-06T03:38:37.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:37 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.104:0/1977243809' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T03:38:37.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:37 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.104:0/1977243809' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T03:38:37.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:37 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.104:0/1977243809' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T03:38:37.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:37 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.104:0/1977243809' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T03:38:37.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:37 smithi131 ceph-mon[36687]: Standby manager daemon z restarted 2024-06-06T03:38:37.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:37 smithi131 ceph-mon[36687]: Standby manager daemon z started 2024-06-06T03:38:37.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:37 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.131:0/1334861459' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T03:38:37.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:37 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.131:0/1334861459' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T03:38:37.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:37 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.131:0/1334861459' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T03:38:37.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:37 smithi131 ceph-mon[36687]: from='mgr.? 172.21.15.131:0/1334861459' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T03:38:37.785 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:37.384+0000 7f6880605200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T03:38:37.785 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:37.618+0000 7f6880605200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T03:38:37.786 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-mon[32220]: from='client.? ' entity='client.admin' 2024-06-06T03:38:37.786 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-mon[32220]: Standby manager daemon y restarted 2024-06-06T03:38:37.786 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-mon[32220]: Standby manager daemon y started 2024-06-06T03:38:37.786 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.104:0/1977243809' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T03:38:37.786 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.104:0/1977243809' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T03:38:37.786 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.104:0/1977243809' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T03:38:37.786 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.104:0/1977243809' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T03:38:37.786 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-mon[32220]: Standby manager daemon z restarted 2024-06-06T03:38:37.786 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-mon[32220]: Standby manager daemon z started 2024-06-06T03:38:37.786 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.131:0/1334861459' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T03:38:37.786 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.131:0/1334861459' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T03:38:37.786 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.131:0/1334861459' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T03:38:37.787 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-mon[32220]: from='mgr.? 172.21.15.131:0/1334861459' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T03:38:37.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:37 smithi104 ceph-mon[36569]: from='client.? ' entity='client.admin' 2024-06-06T03:38:37.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:37 smithi104 ceph-mon[36569]: Standby manager daemon y restarted 2024-06-06T03:38:37.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:37 smithi104 ceph-mon[36569]: Standby manager daemon y started 2024-06-06T03:38:37.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:37 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.104:0/1977243809' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T03:38:37.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:37 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.104:0/1977243809' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T03:38:37.788 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:37 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.104:0/1977243809' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T03:38:37.788 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:37 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.104:0/1977243809' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T03:38:37.788 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:37 smithi104 ceph-mon[36569]: Standby manager daemon z restarted 2024-06-06T03:38:37.788 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:37 smithi104 ceph-mon[36569]: Standby manager daemon z started 2024-06-06T03:38:37.788 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:37 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.131:0/1334861459' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T03:38:37.788 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:37 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.131:0/1334861459' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T03:38:37.788 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:37 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.131:0/1334861459' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T03:38:37.788 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:37 smithi104 ceph-mon[36569]: from='mgr.? 172.21.15.131:0/1334861459' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T03:38:38.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:37.881+0000 7f67dfebf700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:38:38.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:37.881+0000 7f67dfebf700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:38:38.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:37.881+0000 7f67dfebf700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:38:38.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:37.881+0000 7f67dfebf700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:38:38.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:37.881+0000 7f67dfebf700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:38:38.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:37.882+0000 7f67e16c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:38:38.080 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:37.882+0000 7f67e16c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:38:38.081 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:37.882+0000 7f67e16c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:38:38.081 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:37.882+0000 7f67e16c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:38:38.081 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:38:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:38:37.882+0000 7f67e16c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:38:38.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.639+0000 7f5810ff9640 1 -- 172.21.15.40:0/2733733436 <== mon.1 v2:172.21.15.131:3300/0 9 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f57fc0d0210 con 0x7f5814100bd0 2024-06-06T03:38:38.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.639+0000 7f5810ff9640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f58000358c0 0x7f5800035ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:38.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.639+0000 7f5810ff9640 1 -- 172.21.15.40:0/2733733436 --> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f5814061de0 con 0x7f58000358c0 2024-06-06T03:38:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.644+0000 7f58137fe640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f58000358c0 0x7f5800035ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:38.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.646+0000 7f58137fe640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f58000358c0 0x7f5800035ca0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f58040098b0 tx=0x7f5804005f90 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:38.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.714+0000 7f5810ff9640 1 -- 172.21.15.40:0/2733733436 <== mgr.24790 v2:172.21.15.40:6800/1568651190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5814061de0 con 0x7f58000358c0 2024-06-06T03:38:38.714 INFO:teuthology.orchestra.run.smithi040.stdout:Schedule set for path / 2024-06-06T03:38:38.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.716+0000 7f5819f63640 1 -- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f58000358c0 msgr2=0x7f5800035ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:38.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.716+0000 7f5819f63640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f58000358c0 0x7f5800035ca0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f58040098b0 tx=0x7f5804005f90 comp rx=0 tx=0).stop 2024-06-06T03:38:38.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.716+0000 7f5819f63640 1 -- 172.21.15.40:0/2733733436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5814100bd0 msgr2=0x7f581419a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:38.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.716+0000 7f5819f63640 1 --2- 172.21.15.40:0/2733733436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5814100bd0 0x7f581419a310 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00e9f0 tx=0x7f57fc00eeb0 comp rx=0 tx=0).stop 2024-06-06T03:38:38.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.716+0000 7f5813fff640 1 -- 172.21.15.40:0/2733733436 reap_dead start 2024-06-06T03:38:38.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.716+0000 7f5819f63640 1 -- 172.21.15.40:0/2733733436 shutdown_connections 2024-06-06T03:38:38.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.716+0000 7f5819f63640 1 -- 172.21.15.40:0/2733733436 >> 172.21.15.40:0/2733733436 conn(0x7f58140fbe70 msgr2=0x7f58140fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:38.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.716+0000 7f5819f63640 1 -- 172.21.15.40:0/2733733436 shutdown_connections 2024-06-06T03:38:38.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.716+0000 7f5819f63640 1 -- 172.21.15.40:0/2733733436 wait complete. 2024-06-06T03:38:38.727 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-06-06T03:38:38.727 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: Active manager daemon x restarted 2024-06-06T03:38:38.727 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: Activating manager daemon x 2024-06-06T03:38:38.727 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T03:38:38.727 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: mgrmap e21: x(active, starting, since 0.0151292s), standbys: z, y 2024-06-06T03:38:38.727 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:38:38.727 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:38:38.727 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:38:38.727 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T03:38:38.727 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T03:38:38.728 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T03:38:38.728 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T03:38:38.728 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T03:38:38.728 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T03:38:38.728 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T03:38:38.728 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T03:38:38.728 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T03:38:38.728 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T03:38:38.728 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T03:38:38.728 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T03:38:38.728 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T03:38:38.728 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T03:38:38.728 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T03:38:38.729 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T03:38:38.730 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: Manager daemon x is now available 2024-06-06T03:38:38.730 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:38:38.730 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T03:38:38.730 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:38 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T03:38:38.731 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-06-06T03:38:38.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-06-06T03:38:38.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: Active manager daemon x restarted 2024-06-06T03:38:38.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: Activating manager daemon x 2024-06-06T03:38:38.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T03:38:38.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: mgrmap e21: x(active, starting, since 0.0151292s), standbys: z, y 2024-06-06T03:38:38.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:38:38.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:38:38.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:38:38.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T03:38:38.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T03:38:38.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T03:38:38.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T03:38:38.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T03:38:38.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T03:38:38.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T03:38:38.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:38:38.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:38:38.785 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:38:38.785 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:38:38.785 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:38:38.785 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:38:38.785 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T03:38:38.785 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T03:38:38.785 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T03:38:38.785 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: Manager daemon x is now available 2024-06-06T03:38:38.785 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:38:38.785 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T03:38:38.785 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:38 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T03:38:38.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-06-06T03:38:38.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: Active manager daemon x restarted 2024-06-06T03:38:38.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: Activating manager daemon x 2024-06-06T03:38:38.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T03:38:38.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: mgrmap e21: x(active, starting, since 0.0151292s), standbys: z, y 2024-06-06T03:38:38.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T03:38:38.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T03:38:38.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T03:38:38.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T03:38:38.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T03:38:38.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T03:38:38.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T03:38:38.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T03:38:38.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T03:38:38.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T03:38:38.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T03:38:38.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T03:38:38.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T03:38:38.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T03:38:38.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T03:38:38.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T03:38:38.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T03:38:38.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T03:38:38.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T03:38:38.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T03:38:38.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T03:38:38.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T03:38:38.842 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T03:38:38.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T03:38:38.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T03:38:38.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T03:38:38.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T03:38:38.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T03:38:38.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T03:38:38.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T03:38:38.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T03:38:38.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T03:38:38.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T03:38:38.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: Manager daemon x is now available 2024-06-06T03:38:38.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:38:38.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T03:38:38.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:38 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T03:38:38.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.841+0000 7f75596bd640 1 -- 172.21.15.40:0/1796452218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7554102530 msgr2=0x7f7554104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:38.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.841+0000 7f75596bd640 1 --2- 172.21.15.40:0/1796452218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7554102530 0x7f7554104920 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7544009ca0 tx=0x7f754402a8c0 comp rx=0 tx=0).stop 2024-06-06T03:38:38.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.841+0000 7f75596bd640 1 -- 172.21.15.40:0/1796452218 shutdown_connections 2024-06-06T03:38:38.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.841+0000 7f75596bd640 1 --2- 172.21.15.40:0/1796452218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7554107790 0x7f7554109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:38.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.841+0000 7f75596bd640 1 --2- 172.21.15.40:0/1796452218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7554104e60 0x7f7554107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:38.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.841+0000 7f75596bd640 1 --2- 172.21.15.40:0/1796452218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7554102530 0x7f7554104920 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:38.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.841+0000 7f75596bd640 1 -- 172.21.15.40:0/1796452218 >> 172.21.15.40:0/1796452218 conn(0x7f75540fbf20 msgr2=0x7f75540fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:38.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.841+0000 7f75596bd640 1 -- 172.21.15.40:0/1796452218 shutdown_connections 2024-06-06T03:38:38.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.842+0000 7f75596bd640 1 -- 172.21.15.40:0/1796452218 wait complete. 2024-06-06T03:38:38.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.842+0000 7f75596bd640 1 Processor -- start 2024-06-06T03:38:38.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.842+0000 7f75596bd640 1 -- start start 2024-06-06T03:38:38.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.842+0000 7f75596bd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7554102530 0x7f7554197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:38.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.842+0000 7f75596bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7554104e60 0x7f75541981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:38.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.842+0000 7f75596bd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7554107790 0x7f755419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:38.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.842+0000 7f75596bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f755410d6e0 con 0x7f7554104e60 2024-06-06T03:38:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.842+0000 7f75596bd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f755410d560 con 0x7f7554107790 2024-06-06T03:38:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.842+0000 7f75596bd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f755410d860 con 0x7f7554102530 2024-06-06T03:38:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f75527fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7554104e60 0x7f75541981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f75537fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7554107790 0x7f755419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f7552ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7554102530 0x7f7554197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f75527fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7554104e60 0x7f75541981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44436/0 (socket says 172.21.15.40:44436) 2024-06-06T03:38:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f75527fc640 1 -- 172.21.15.40:0/3603936109 learned_addr learned my addr 172.21.15.40:0/3603936109 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f75527fc640 1 -- 172.21.15.40:0/3603936109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7554102530 msgr2=0x7f7554197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f75527fc640 1 --2- 172.21.15.40:0/3603936109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7554102530 0x7f7554197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f75527fc640 1 -- 172.21.15.40:0/3603936109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7554107790 msgr2=0x7f755419f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f75527fc640 1 --2- 172.21.15.40:0/3603936109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7554107790 0x7f755419f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f75527fc640 1 -- 172.21.15.40:0/3603936109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755419f890 con 0x7f7554104e60 2024-06-06T03:38:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f75537fe640 1 --2- 172.21.15.40:0/3603936109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7554107790 0x7f755419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f75527fc640 1 --2- 172.21.15.40:0/3603936109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7554104e60 0x7f75541981c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f754800e9a0 tx=0x7f754800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f753bfff640 1 -- 172.21.15.40:0/3603936109 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754800cd80 con 0x7f7554104e60 2024-06-06T03:38:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f75596bd640 1 -- 172.21.15.40:0/3603936109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755419fb80 con 0x7f7554104e60 2024-06-06T03:38:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f753bfff640 1 -- 172.21.15.40:0/3603936109 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75480035f0 con 0x7f7554104e60 2024-06-06T03:38:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f753bfff640 1 -- 172.21.15.40:0/3603936109 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548010670 con 0x7f7554104e60 2024-06-06T03:38:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.843+0000 7f75596bd640 1 -- 172.21.15.40:0/3603936109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75541a00d0 con 0x7f7554104e60 2024-06-06T03:38:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.844+0000 7f75596bd640 1 -- 172.21.15.40:0/3603936109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75540fda10 con 0x7f7554104e60 2024-06-06T03:38:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.845+0000 7f753bfff640 1 -- 172.21.15.40:0/3603936109 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f75480037a0 con 0x7f7554104e60 2024-06-06T03:38:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.846+0000 7f753bfff640 1 --2- 172.21.15.40:0/3603936109 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f753c0a2a40 0x7f753c0a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.846+0000 7f7552ffd640 1 --2- 172.21.15.40:0/3603936109 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f753c0a2a40 0x7f753c0a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.846+0000 7f753bfff640 1 -- 172.21.15.40:0/3603936109 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f75480cefa0 con 0x7f7554104e60 2024-06-06T03:38:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.847+0000 7f7552ffd640 1 --2- 172.21.15.40:0/3603936109 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f753c0a2a40 0x7f753c0a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7544006010 tx=0x7f7544000f40 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:38.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:38.851+0000 7f753bfff640 1 -- 172.21.15.40:0/3603936109 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f754809c160 con 0x7f7554104e60 2024-06-06T03:38:39.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.060+0000 7f75596bd640 1 -- 172.21.15.40:0/3603936109 --> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f7554061e80 con 0x7f753c0a2a40 2024-06-06T03:38:39.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.117+0000 7f753bfff640 1 -- 172.21.15.40:0/3603936109 <== mgr.24790 v2:172.21.15.40:6800/1568651190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f7554061e80 con 0x7f753c0a2a40 2024-06-06T03:38:39.117 INFO:teuthology.orchestra.run.smithi040.stdout:Retention added to path / 2024-06-06T03:38:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.119+0000 7f75596bd640 1 -- 172.21.15.40:0/3603936109 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f753c0a2a40 msgr2=0x7f753c0a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.119+0000 7f75596bd640 1 --2- 172.21.15.40:0/3603936109 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f753c0a2a40 0x7f753c0a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7544006010 tx=0x7f7544000f40 comp rx=0 tx=0).stop 2024-06-06T03:38:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.119+0000 7f75596bd640 1 -- 172.21.15.40:0/3603936109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7554104e60 msgr2=0x7f75541981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.119+0000 7f75596bd640 1 --2- 172.21.15.40:0/3603936109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7554104e60 0x7f75541981c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f754800e9a0 tx=0x7f754800ee60 comp rx=0 tx=0).stop 2024-06-06T03:38:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.119+0000 7f75596bd640 1 -- 172.21.15.40:0/3603936109 shutdown_connections 2024-06-06T03:38:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.119+0000 7f75596bd640 1 --2- 172.21.15.40:0/3603936109 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f753c0a2a40 0x7f753c0a4f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.119+0000 7f75596bd640 1 --2- 172.21.15.40:0/3603936109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7554107790 0x7f755419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.119+0000 7f75596bd640 1 --2- 172.21.15.40:0/3603936109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7554104e60 0x7f75541981c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.119+0000 7f75596bd640 1 --2- 172.21.15.40:0/3603936109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7554102530 0x7f7554197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.119+0000 7f75596bd640 1 -- 172.21.15.40:0/3603936109 >> 172.21.15.40:0/3603936109 conn(0x7f75540fbf20 msgr2=0x7f7554102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.120+0000 7f75596bd640 1 -- 172.21.15.40:0/3603936109 shutdown_connections 2024-06-06T03:38:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.120+0000 7f75596bd640 1 -- 172.21.15.40:0/3603936109 wait complete. 2024-06-06T03:38:39.130 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-06-06T03:38:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.239+0000 7fa768da9640 1 -- 172.21.15.40:0/3845224556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764104ec0 msgr2=0x7fa7641072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.239+0000 7fa768da9640 1 --2- 172.21.15.40:0/3845224556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764104ec0 0x7fa7641072b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa758009900 tx=0x7fa75802a410 comp rx=0 tx=0).stop 2024-06-06T03:38:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.239+0000 7fa768da9640 1 -- 172.21.15.40:0/3845224556 shutdown_connections 2024-06-06T03:38:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.239+0000 7fa768da9640 1 --2- 172.21.15.40:0/3845224556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7641077f0 0x7fa764109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.239+0000 7fa768da9640 1 --2- 172.21.15.40:0/3845224556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764104ec0 0x7fa7641072b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.239+0000 7fa768da9640 1 --2- 172.21.15.40:0/3845224556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa764102590 0x7fa764104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.239+0000 7fa768da9640 1 -- 172.21.15.40:0/3845224556 >> 172.21.15.40:0/3845224556 conn(0x7fa7640fbf20 msgr2=0x7fa7640fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.239+0000 7fa768da9640 1 -- 172.21.15.40:0/3845224556 shutdown_connections 2024-06-06T03:38:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.239+0000 7fa768da9640 1 -- 172.21.15.40:0/3845224556 wait complete. 2024-06-06T03:38:39.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa768da9640 1 Processor -- start 2024-06-06T03:38:39.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa768da9640 1 -- start start 2024-06-06T03:38:39.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa768da9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764102590 0x7fa764101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:39.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa768da9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa764104ec0 0x7fa764100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:39.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa768da9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7641077f0 0x7fa764100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:39.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa768da9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa76410d6b0 con 0x7fa764102590 2024-06-06T03:38:39.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa768da9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa76410d530 con 0x7fa764104ec0 2024-06-06T03:38:39.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa768da9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa76410d830 con 0x7fa7641077f0 2024-06-06T03:38:39.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa76259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764102590 0x7fa764101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:39.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa76259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764102590 0x7fa764101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44446/0 (socket says 172.21.15.40:44446) 2024-06-06T03:38:39.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa76259c640 1 -- 172.21.15.40:0/3580750929 learned_addr learned my addr 172.21.15.40:0/3580750929 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:39.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa761d9b640 1 --2- 172.21.15.40:0/3580750929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa764104ec0 0x7fa764100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:39.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa762d9d640 1 --2- 172.21.15.40:0/3580750929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7641077f0 0x7fa764100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:39.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa76259c640 1 -- 172.21.15.40:0/3580750929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7641077f0 msgr2=0x7fa764100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:39.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa76259c640 1 --2- 172.21.15.40:0/3580750929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7641077f0 0x7fa764100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa76259c640 1 -- 172.21.15.40:0/3580750929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa764104ec0 msgr2=0x7fa764100540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:39.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa76259c640 1 --2- 172.21.15.40:0/3580750929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa764104ec0 0x7fa764100540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa76259c640 1 -- 172.21.15.40:0/3580750929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa764101310 con 0x7fa764102590 2024-06-06T03:38:39.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.240+0000 7fa76259c640 1 --2- 172.21.15.40:0/3580750929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764102590 0x7fa764101f00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa75400d770 tx=0x7fa75400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:39.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.241+0000 7fa7537fe640 1 -- 172.21.15.40:0/3580750929 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754017070 con 0x7fa764102590 2024-06-06T03:38:39.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.241+0000 7fa7537fe640 1 -- 172.21.15.40:0/3580750929 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa754005500 con 0x7fa764102590 2024-06-06T03:38:39.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.241+0000 7fa768da9640 1 -- 172.21.15.40:0/3580750929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7641a1db0 con 0x7fa764102590 2024-06-06T03:38:39.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.241+0000 7fa7537fe640 1 -- 172.21.15.40:0/3580750929 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754004e00 con 0x7fa764102590 2024-06-06T03:38:39.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.241+0000 7fa768da9640 1 -- 172.21.15.40:0/3580750929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7641a21c0 con 0x7fa764102590 2024-06-06T03:38:39.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.242+0000 7fa768da9640 1 -- 172.21.15.40:0/3580750929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7640fda10 con 0x7fa764102590 2024-06-06T03:38:39.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.243+0000 7fa7537fe640 1 -- 172.21.15.40:0/3580750929 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fa754005000 con 0x7fa764102590 2024-06-06T03:38:39.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.244+0000 7fa7537fe640 1 --2- 172.21.15.40:0/3580750929 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa74c0a2ac0 0x7fa74c0a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:39.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.244+0000 7fa761d9b640 1 --2- 172.21.15.40:0/3580750929 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa74c0a2ac0 0x7fa74c0a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:39.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.244+0000 7fa7537fe640 1 -- 172.21.15.40:0/3580750929 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fa7540ce610 con 0x7fa764102590 2024-06-06T03:38:39.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.244+0000 7fa761d9b640 1 --2- 172.21.15.40:0/3580750929 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa74c0a2ac0 0x7fa74c0a4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa758005ab0 tx=0x7fa758000bc0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:39.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.248+0000 7fa7537fe640 1 -- 172.21.15.40:0/3580750929 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa75409b180 con 0x7fa764102590 2024-06-06T03:38:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.472+0000 7fa768da9640 1 -- 172.21.15.40:0/3580750929 --> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa764061de0 con 0x7fa74c0a2ac0 2024-06-06T03:38:39.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.480+0000 7fa7537fe640 1 -- 172.21.15.40:0/3580750929 <== mgr.24790 v2:172.21.15.40:6800/1568651190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fa764061de0 con 0x7fa74c0a2ac0 2024-06-06T03:38:39.479 INFO:teuthology.orchestra.run.smithi040.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-06-06T00:00:00", "created": "2024-06-06T03:38:38", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-06-06T03:38:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.482+0000 7fa768da9640 1 -- 172.21.15.40:0/3580750929 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa74c0a2ac0 msgr2=0x7fa74c0a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.482+0000 7fa768da9640 1 --2- 172.21.15.40:0/3580750929 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa74c0a2ac0 0x7fa74c0a4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa758005ab0 tx=0x7fa758000bc0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.482+0000 7fa768da9640 1 -- 172.21.15.40:0/3580750929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764102590 msgr2=0x7fa764101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.482+0000 7fa768da9640 1 --2- 172.21.15.40:0/3580750929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764102590 0x7fa764101f00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa75400d770 tx=0x7fa75400dc30 comp rx=0 tx=0).stop 2024-06-06T03:38:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.482+0000 7fa768da9640 1 -- 172.21.15.40:0/3580750929 shutdown_connections 2024-06-06T03:38:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.482+0000 7fa768da9640 1 --2- 172.21.15.40:0/3580750929 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa74c0a2ac0 0x7fa74c0a4f80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.482+0000 7fa768da9640 1 --2- 172.21.15.40:0/3580750929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7641077f0 0x7fa764100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.482+0000 7fa768da9640 1 --2- 172.21.15.40:0/3580750929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa764104ec0 0x7fa764100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.482+0000 7fa768da9640 1 --2- 172.21.15.40:0/3580750929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764102590 0x7fa764101f00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.482+0000 7fa768da9640 1 -- 172.21.15.40:0/3580750929 >> 172.21.15.40:0/3580750929 conn(0x7fa7640fbf20 msgr2=0x7fa764102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.482+0000 7fa768da9640 1 -- 172.21.15.40:0/3580750929 shutdown_connections 2024-06-06T03:38:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.482+0000 7fa768da9640 1 -- 172.21.15.40:0/3580750929 wait complete. 2024-06-06T03:38:39.493 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-06-06T03:38:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.611+0000 7fc0a5279640 1 -- 172.21.15.40:0/1112304600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a0103c50 msgr2=0x7fc0a0104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.611+0000 7fc0a5279640 1 --2- 172.21.15.40:0/1112304600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a0103c50 0x7fc0a0104070 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc090009900 tx=0x7fc09002a410 comp rx=0 tx=0).stop 2024-06-06T03:38:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.611+0000 7fc0a5279640 1 -- 172.21.15.40:0/1112304600 shutdown_connections 2024-06-06T03:38:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.611+0000 7fc0a5279640 1 --2- 172.21.15.40:0/1112304600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a0105800 0x7fc0a010c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.611+0000 7fc0a5279640 1 --2- 172.21.15.40:0/1112304600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0a0104e40 0x7fc0a01052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.611+0000 7fc0a5279640 1 --2- 172.21.15.40:0/1112304600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a0103c50 0x7fc0a0104070 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.611+0000 7fc0a5279640 1 -- 172.21.15.40:0/1112304600 >> 172.21.15.40:0/1112304600 conn(0x7fc0a00ff1c0 msgr2=0x7fc0a0101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.611+0000 7fc0a5279640 1 -- 172.21.15.40:0/1112304600 shutdown_connections 2024-06-06T03:38:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.611+0000 7fc0a5279640 1 -- 172.21.15.40:0/1112304600 wait complete. 2024-06-06T03:38:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.611+0000 7fc0a5279640 1 Processor -- start 2024-06-06T03:38:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.611+0000 7fc0a5279640 1 -- start start 2024-06-06T03:38:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc0a5279640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a0103c50 0x7fc0a0199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc0a5279640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a0104e40 0x7fc0a019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc0a5279640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0a0105800 0x7fc0a01a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc0a5279640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a010f8d0 con 0x7fc0a0103c50 2024-06-06T03:38:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc0a5279640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a010f750 con 0x7fc0a0105800 2024-06-06T03:38:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc0a5279640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a010fa50 con 0x7fc0a0104e40 2024-06-06T03:38:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc09effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a0103c50 0x7fc0a0199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc09effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a0103c50 0x7fc0a0199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44462/0 (socket says 172.21.15.40:44462) 2024-06-06T03:38:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc09effd640 1 -- 172.21.15.40:0/1957086384 learned_addr learned my addr 172.21.15.40:0/1957086384 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc097fff640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a0104e40 0x7fc0a019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc09f7fe640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0a0105800 0x7fc0a01a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:39.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc09f7fe640 1 -- 172.21.15.40:0/1957086384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a0104e40 msgr2=0x7fc0a019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:39.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc09f7fe640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a0104e40 0x7fc0a019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc09f7fe640 1 -- 172.21.15.40:0/1957086384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a0103c50 msgr2=0x7fc0a0199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:39.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc09f7fe640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a0103c50 0x7fc0a0199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc09f7fe640 1 -- 172.21.15.40:0/1957086384 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a01a1ad0 con 0x7fc0a0105800 2024-06-06T03:38:39.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.612+0000 7fc09effd640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a0103c50 0x7fc0a0199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:39.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.613+0000 7fc09f7fe640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0a0105800 0x7fc0a01a1460 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc098009fd0 tx=0x7fc09800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:39.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.613+0000 7fc09cff9640 1 -- 172.21.15.40:0/1957086384 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09800ce20 con 0x7fc0a0105800 2024-06-06T03:38:39.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.613+0000 7fc09cff9640 1 -- 172.21.15.40:0/1957086384 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc098005020 con 0x7fc0a0105800 2024-06-06T03:38:39.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.613+0000 7fc0a5279640 1 -- 172.21.15.40:0/1957086384 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a0075e40 con 0x7fc0a0105800 2024-06-06T03:38:39.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.613+0000 7fc09cff9640 1 -- 172.21.15.40:0/1957086384 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098010720 con 0x7fc0a0105800 2024-06-06T03:38:39.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.613+0000 7fc0a5279640 1 -- 172.21.15.40:0/1957086384 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a0076340 con 0x7fc0a0105800 2024-06-06T03:38:39.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.614+0000 7fc0a5279640 1 -- 172.21.15.40:0/1957086384 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a010af70 con 0x7fc0a0105800 2024-06-06T03:38:39.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.615+0000 7fc09cff9640 1 -- 172.21.15.40:0/1957086384 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fc098005540 con 0x7fc0a0105800 2024-06-06T03:38:39.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.616+0000 7fc09cff9640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fc08c0a2a40 0x7fc08c0a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:39.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.616+0000 7fc09cff9640 1 -- 172.21.15.40:0/1957086384 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fc0980cf3d0 con 0x7fc0a0105800 2024-06-06T03:38:39.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.616+0000 7fc09effd640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fc08c0a2a40 0x7fc08c0a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:39.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.616+0000 7fc09effd640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fc08c0a2a40 0x7fc08c0a4f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc090000c00 tx=0x7fc0900046c0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:39.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.622+0000 7fc09cff9640 1 -- 172.21.15.40:0/1957086384 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc09809c590 con 0x7fc0a0105800 2024-06-06T03:38:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.649+0000 7fc09cff9640 1 -- 172.21.15.40:0/1957086384 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 23) ==== 137953+0+0 (secure 0 0 0) 0x7fc098098500 con 0x7fc0a0105800 2024-06-06T03:38:39.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:39 smithi040 ceph-mon[32220]: [06/Jun/2024:03:38:38] ENGINE Bus STARTING 2024-06-06T03:38:39.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:39 smithi040 ceph-mon[32220]: [06/Jun/2024:03:38:38] ENGINE Serving on https://172.21.15.40:7150 2024-06-06T03:38:39.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:39 smithi040 ceph-mon[32220]: [06/Jun/2024:03:38:38] ENGINE Serving on http://172.21.15.40:8765 2024-06-06T03:38:39.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:39 smithi040 ceph-mon[32220]: [06/Jun/2024:03:38:38] ENGINE Bus STARTED 2024-06-06T03:38:39.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:39 smithi040 ceph-mon[32220]: mgrmap e22: x(active, since 1.01989s), standbys: z, y 2024-06-06T03:38:39.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:39 smithi040 ceph-mon[32220]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:39.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:39 smithi040 ceph-mon[32220]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:38:39.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:39 smithi040 ceph-mon[32220]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:39.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.873+0000 7fc0a5279640 1 -- 172.21.15.40:0/1957086384 --> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fc0a0061de0 con 0x7fc08c0a2a40 2024-06-06T03:38:39.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.880+0000 7fc09cff9640 1 -- 172.21.15.40:0/1957086384 <== mgr.24790 v2:172.21.15.40:6800/1568651190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fc0a0061de0 con 0x7fc08c0a2a40 2024-06-06T03:38:39.880 INFO:teuthology.orchestra.run.smithi040.stdout:/ 1m 6m3h 2024-06-06T03:38:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.882+0000 7fc0a5279640 1 -- 172.21.15.40:0/1957086384 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fc08c0a2a40 msgr2=0x7fc08c0a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.882+0000 7fc0a5279640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fc08c0a2a40 0x7fc08c0a4f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc090000c00 tx=0x7fc0900046c0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.882+0000 7fc0a5279640 1 -- 172.21.15.40:0/1957086384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0a0105800 msgr2=0x7fc0a01a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.882+0000 7fc0a5279640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0a0105800 0x7fc0a01a1460 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc098009fd0 tx=0x7fc09800ef10 comp rx=0 tx=0).stop 2024-06-06T03:38:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.882+0000 7fc0a5279640 1 -- 172.21.15.40:0/1957086384 shutdown_connections 2024-06-06T03:38:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.882+0000 7fc0a5279640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fc08c0a2a40 0x7fc08c0a4f00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.882+0000 7fc0a5279640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0a0105800 0x7fc0a01a1460 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.882+0000 7fc0a5279640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a0104e40 0x7fc0a019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.882+0000 7fc0a5279640 1 --2- 172.21.15.40:0/1957086384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a0103c50 0x7fc0a0199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.882+0000 7fc0a5279640 1 -- 172.21.15.40:0/1957086384 >> 172.21.15.40:0/1957086384 conn(0x7fc0a00ff1c0 msgr2=0x7fc0a0108140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.882+0000 7fc0a5279640 1 -- 172.21.15.40:0/1957086384 shutdown_connections 2024-06-06T03:38:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:39.882+0000 7fc0a5279640 1 -- 172.21.15.40:0/1957086384 wait complete. 2024-06-06T03:38:39.893 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-06-06T03:38:39.920 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-06-06T03:38:39.931 INFO:teuthology.run_tasks:Running task quiescer... 2024-06-06T03:38:40.095 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-06T03:38:40.096 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T03:38:40.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:39 smithi104 ceph-mon[36569]: [06/Jun/2024:03:38:38] ENGINE Bus STARTING 2024-06-06T03:38:40.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:39 smithi104 ceph-mon[36569]: [06/Jun/2024:03:38:38] ENGINE Serving on https://172.21.15.40:7150 2024-06-06T03:38:40.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:39 smithi104 ceph-mon[36569]: [06/Jun/2024:03:38:38] ENGINE Serving on http://172.21.15.40:8765 2024-06-06T03:38:40.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:39 smithi104 ceph-mon[36569]: [06/Jun/2024:03:38:38] ENGINE Bus STARTED 2024-06-06T03:38:40.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:39 smithi104 ceph-mon[36569]: mgrmap e22: x(active, since 1.01989s), standbys: z, y 2024-06-06T03:38:40.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:39 smithi104 ceph-mon[36569]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:40.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:39 smithi104 ceph-mon[36569]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:38:40.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:39 smithi104 ceph-mon[36569]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:40.098 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:39 smithi131 ceph-mon[36687]: [06/Jun/2024:03:38:38] ENGINE Bus STARTING 2024-06-06T03:38:40.098 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:39 smithi131 ceph-mon[36687]: [06/Jun/2024:03:38:38] ENGINE Serving on https://172.21.15.40:7150 2024-06-06T03:38:40.098 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:39 smithi131 ceph-mon[36687]: [06/Jun/2024:03:38:38] ENGINE Serving on http://172.21.15.40:8765 2024-06-06T03:38:40.098 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:39 smithi131 ceph-mon[36687]: [06/Jun/2024:03:38:38] ENGINE Bus STARTED 2024-06-06T03:38:40.098 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:39 smithi131 ceph-mon[36687]: mgrmap e22: x(active, since 1.01989s), standbys: z, y 2024-06-06T03:38:40.098 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:39 smithi131 ceph-mon[36687]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:40.098 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:39 smithi131 ceph-mon[36687]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:38:40.098 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:39 smithi131 ceph-mon[36687]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:40.348 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:40.748 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:40 smithi040 ceph-mon[32220]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:40.748 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:40 smithi040 ceph-mon[32220]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:38:40.748 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:40 smithi040 ceph-mon[32220]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-06-06T03:38:40.749 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:40 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:40.749 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:40 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:40.749 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:40 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:40.749 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:40 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:40.749 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:40 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:40.749 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:40 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:40.749 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:40 smithi040 ceph-mon[32220]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:41.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:40 smithi131 ceph-mon[36687]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:41.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:40 smithi131 ceph-mon[36687]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:38:41.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:40 smithi131 ceph-mon[36687]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-06-06T03:38:41.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:40 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:40 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:40 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:40 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:40 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:40 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.033 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:40 smithi131 ceph-mon[36687]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:41.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:40 smithi104 ceph-mon[36569]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:41.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:40 smithi104 ceph-mon[36569]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:38:41.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:40 smithi104 ceph-mon[36569]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-06-06T03:38:41.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:40 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:40 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:40 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:40 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:40 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:40 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:40 smithi104 ceph-mon[36569]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:38:41.959 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.959 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.959 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:41.959 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:41.959 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:41.959 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:41.959 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi131 to 176.2M 2024-06-06T03:38:41.960 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: Unable to set osd_memory_target on smithi131 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-06T03:38:41.960 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.960 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.960 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:41.960 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:41.960 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:41.960 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:41.960 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-06-06T03:38:41.960 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: Unable to set osd_memory_target on smithi104 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-06-06T03:38:41.960 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.960 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:41.960 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:41.961 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:41.961 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:41.961 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:41.961 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:38:41.961 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:41 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:38:41.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.977+0000 7fa8eb128700 1 -- 172.21.15.40:0/3808783757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e4101d50 msgr2=0x7fa8e410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:41.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.977+0000 7fa8eb128700 1 --2- 172.21.15.40:0/3808783757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e4101d50 0x7fa8e410e240 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00b0a0 tx=0x7fa8dc00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:38:41.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.978+0000 7fa8eb128700 1 -- 172.21.15.40:0/3808783757 shutdown_connections 2024-06-06T03:38:41.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.978+0000 7fa8eb128700 1 --2- 172.21.15.40:0/3808783757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e4107400 0x7fa8e41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:41.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.978+0000 7fa8eb128700 1 --2- 172.21.15.40:0/3808783757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e4101d50 0x7fa8e410e240 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:41.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.978+0000 7fa8eb128700 1 --2- 172.21.15.40:0/3808783757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e4101390 0x7fa8e4101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:41.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.978+0000 7fa8eb128700 1 -- 172.21.15.40:0/3808783757 >> 172.21.15.40:0/3808783757 conn(0x7fa8e4078340 msgr2=0x7fa8e4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:41.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.978+0000 7fa8eb128700 1 -- 172.21.15.40:0/3808783757 shutdown_connections 2024-06-06T03:38:41.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.978+0000 7fa8eb128700 1 -- 172.21.15.40:0/3808783757 wait complete. 2024-06-06T03:38:41.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.978+0000 7fa8eb128700 1 Processor -- start 2024-06-06T03:38:41.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.978+0000 7fa8eb128700 1 -- start start 2024-06-06T03:38:41.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8eb128700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e4101390 0x7fa8e4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:41.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8eb128700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e4101d50 0x7fa8e4197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8eb128700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e4107400 0x7fa8e419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8eb128700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e41116e0 con 0x7fa8e4101390 2024-06-06T03:38:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8eb128700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e4111560 con 0x7fa8e4101d50 2024-06-06T03:38:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8eb128700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e4111860 con 0x7fa8e4107400 2024-06-06T03:38:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8e8ec4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e4101390 0x7fa8e4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8e96c5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e4107400 0x7fa8e419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8e96c5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e4107400 0x7fa8e419c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.40:34648/0 (socket says 172.21.15.40:34648) 2024-06-06T03:38:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8e96c5700 1 -- 172.21.15.40:0/3462555775 learned_addr learned my addr 172.21.15.40:0/3462555775 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8e3fff700 1 --2- 172.21.15.40:0/3462555775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e4101d50 0x7fa8e4197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8e96c5700 1 -- 172.21.15.40:0/3462555775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e4101d50 msgr2=0x7fa8e4197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8e96c5700 1 --2- 172.21.15.40:0/3462555775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e4101d50 0x7fa8e4197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8e96c5700 1 -- 172.21.15.40:0/3462555775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e4101390 msgr2=0x7fa8e4197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8e96c5700 1 --2- 172.21.15.40:0/3462555775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e4101390 0x7fa8e4197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:41.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8e96c5700 1 -- 172.21.15.40:0/3462555775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e419ce00 con 0x7fa8e4107400 2024-06-06T03:38:41.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.979+0000 7fa8e8ec4700 1 --2- 172.21.15.40:0/3462555775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e4101390 0x7fa8e4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:38:41.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.980+0000 7fa8e96c5700 1 --2- 172.21.15.40:0/3462555775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e4107400 0x7fa8e419c8c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00b070 tx=0x7fa8dc004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:41.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.980+0000 7fa8e1ffb700 1 -- 172.21.15.40:0/3462555775 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc021070 con 0x7fa8e4107400 2024-06-06T03:38:41.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.980+0000 7fa8eb128700 1 -- 172.21.15.40:0/3462555775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e419d090 con 0x7fa8e4107400 2024-06-06T03:38:41.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.980+0000 7fa8e1ffb700 1 -- 172.21.15.40:0/3462555775 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8dc003c00 con 0x7fa8e4107400 2024-06-06T03:38:41.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.980+0000 7fa8e1ffb700 1 -- 172.21.15.40:0/3462555775 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc004740 con 0x7fa8e4107400 2024-06-06T03:38:41.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.980+0000 7fa8eb128700 1 -- 172.21.15.40:0/3462555775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e419d620 con 0x7fa8e4107400 2024-06-06T03:38:41.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.981+0000 7fa8eb128700 1 -- 172.21.15.40:0/3462555775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e4102ab0 con 0x7fa8e4107400 2024-06-06T03:38:41.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.983+0000 7fa8e1ffb700 1 -- 172.21.15.40:0/3462555775 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) ==== 137953+0+0 (secure 0 0 0) 0x7fa8dc007980 con 0x7fa8e4107400 2024-06-06T03:38:41.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.984+0000 7fa8e1ffb700 1 --2- 172.21.15.40:0/3462555775 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa8cc0a2e10 0x7fa8cc0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:41.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.984+0000 7fa8e8ec4700 1 --2- 172.21.15.40:0/3462555775 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa8cc0a2e10 0x7fa8cc0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:41.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.984+0000 7fa8e1ffb700 1 -- 172.21.15.40:0/3462555775 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fa8dc00f020 con 0x7fa8e4107400 2024-06-06T03:38:41.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.984+0000 7fa8e8ec4700 1 --2- 172.21.15.40:0/3462555775 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa8cc0a2e10 0x7fa8cc0a52d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009fd0 tx=0x7fa8d4009260 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:41.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:41.986+0000 7fa8e1ffb700 1 -- 172.21.15.40:0/3462555775 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa8dc02a020 con 0x7fa8e4107400 2024-06-06T03:38:42.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:42.195+0000 7fa8eb128700 1 -- 172.21.15.40:0/3462555775 --> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa8e406a730 con 0x7fa8cc0a2e10 2024-06-06T03:38:42.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:42.210+0000 7fa8e1ffb700 1 -- 172.21.15.40:0/3462555775 <== mgr.24790 v2:172.21.15.40:6800/1568651190 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258046 (secure 0 0 0) 0x7fa8e406a730 con 0x7fa8cc0a2e10 2024-06-06T03:38:42.213 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:38:42.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:42.213+0000 7fa8eb128700 1 -- 172.21.15.40:0/3462555775 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa8cc0a2e10 msgr2=0x7fa8cc0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:42.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:42.213+0000 7fa8eb128700 1 --2- 172.21.15.40:0/3462555775 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa8cc0a2e10 0x7fa8cc0a52d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009fd0 tx=0x7fa8d4009260 comp rx=0 tx=0).stop 2024-06-06T03:38:42.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:42.213+0000 7fa8eb128700 1 -- 172.21.15.40:0/3462555775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e4107400 msgr2=0x7fa8e419c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:42.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:42.213+0000 7fa8eb128700 1 --2- 172.21.15.40:0/3462555775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e4107400 0x7fa8e419c8c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00b070 tx=0x7fa8dc004c60 comp rx=0 tx=0).stop 2024-06-06T03:38:42.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:42.213+0000 7fa8eb128700 1 -- 172.21.15.40:0/3462555775 shutdown_connections 2024-06-06T03:38:42.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:42.213+0000 7fa8eb128700 1 --2- 172.21.15.40:0/3462555775 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa8cc0a2e10 0x7fa8cc0a52d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:42.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:42.213+0000 7fa8eb128700 1 --2- 172.21.15.40:0/3462555775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e4107400 0x7fa8e419c8c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:42.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:42.213+0000 7fa8eb128700 1 --2- 172.21.15.40:0/3462555775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8e4101390 0x7fa8e4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:42.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:42.213+0000 7fa8eb128700 1 --2- 172.21.15.40:0/3462555775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e4101d50 0x7fa8e4197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:42.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:42.213+0000 7fa8eb128700 1 -- 172.21.15.40:0/3462555775 >> 172.21.15.40:0/3462555775 conn(0x7fa8e4078340 msgr2=0x7fa8e40ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:42.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:42.213+0000 7fa8eb128700 1 -- 172.21.15.40:0/3462555775 shutdown_connections 2024-06-06T03:38:42.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:42.213+0000 7fa8eb128700 1 -- 172.21.15.40:0/3462555775 wait complete. 2024-06-06T03:38:42.216 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-06-06T03:38:42.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:42.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:42.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi131 to 176.2M 2024-06-06T03:38:42.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: Unable to set osd_memory_target on smithi131 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-06T03:38:42.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:42.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:42.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-06-06T03:38:42.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: Unable to set osd_memory_target on smithi104 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-06-06T03:38:42.285 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:42.285 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:42.285 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.286 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.286 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.286 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.286 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:38:42.286 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:41 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:38:42.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:42.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:42.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi131 to 176.2M 2024-06-06T03:38:42.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: Unable to set osd_memory_target on smithi131 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-06T03:38:42.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:42.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:42.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-06-06T03:38:42.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: Unable to set osd_memory_target on smithi104 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-06-06T03:38:42.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:42.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:42.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:38:42.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:38:42.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:41 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:38:42.855 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-06-06T03:38:41.646188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193145,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2117,"num_read_kb":1227,"num_write":565,"num_write_kb":4411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":946,"ondisk_log_size":946,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365768,"kb_used_data":12416,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576056,"statfs":{"total":1151940427776,"available":1151565881344,"internally_reserved":0,"allocated":12713984,"data_stored":9407591,"data_compressed":211606,"data_compressed_allocated":274432,"data_compressed_original":3399680,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.994488"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890341+0000","last_change":"2024-06-06T03:35:13.854938+0000","last_active":"2024-06-06T03:38:37.890341+0000","last_peered":"2024-06-06T03:38:37.890341+0000","last_clean":"2024-06-06T03:38:37.890341+0000","last_became_active":"2024-06-06T03:35:13.853574+0000","last_became_peered":"2024-06-06T03:35:13.853574+0000","last_unstale":"2024-06-06T03:38:37.890341+0000","last_undegraded":"2024-06-06T03:38:37.890341+0000","last_fullsized":"2024-06-06T03:38:37.890341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:21:09.256002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.892012+0000","last_change":"2024-06-06T03:35:13.858299+0000","last_active":"2024-06-06T03:38:37.892012+0000","last_peered":"2024-06-06T03:38:37.892012+0000","last_clean":"2024-06-06T03:38:37.892012+0000","last_became_active":"2024-06-06T03:35:13.857993+0000","last_became_peered":"2024-06-06T03:35:13.857993+0000","last_unstale":"2024-06-06T03:38:37.892012+0000","last_undegraded":"2024-06-06T03:38:37.892012+0000","last_fullsized":"2024-06-06T03:38:37.892012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.811056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891638+0000","last_change":"2024-06-06T03:35:13.852783+0000","last_active":"2024-06-06T03:38:37.891638+0000","last_peered":"2024-06-06T03:38:37.891638+0000","last_clean":"2024-06-06T03:38:37.891638+0000","last_became_active":"2024-06-06T03:35:13.852468+0000","last_became_peered":"2024-06-06T03:35:13.852468+0000","last_unstale":"2024-06-06T03:38:37.891638+0000","last_undegraded":"2024-06-06T03:38:37.891638+0000","last_fullsized":"2024-06-06T03:38:37.891638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:04:59.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.629947+0000","last_change":"2024-06-06T03:35:13.845223+0000","last_active":"2024-06-06T03:38:37.629947+0000","last_peered":"2024-06-06T03:38:37.629947+0000","last_clean":"2024-06-06T03:38:37.629947+0000","last_became_active":"2024-06-06T03:35:13.844854+0000","last_became_peered":"2024-06-06T03:35:13.844854+0000","last_unstale":"2024-06-06T03:38:37.629947+0000","last_undegraded":"2024-06-06T03:38:37.629947+0000","last_fullsized":"2024-06-06T03:38:37.629947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:35:20.058188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.727987+0000","last_change":"2024-06-06T03:35:13.853703+0000","last_active":"2024-06-06T03:38:37.727987+0000","last_peered":"2024-06-06T03:38:37.727987+0000","last_clean":"2024-06-06T03:38:37.727987+0000","last_became_active":"2024-06-06T03:35:13.853358+0000","last_became_peered":"2024-06-06T03:35:13.853358+0000","last_unstale":"2024-06-06T03:38:37.727987+0000","last_undegraded":"2024-06-06T03:38:37.727987+0000","last_fullsized":"2024-06-06T03:38:37.727987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:23:39.092306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736404+0000","last_change":"2024-06-06T03:35:13.859414+0000","last_active":"2024-06-06T03:38:37.736404+0000","last_peered":"2024-06-06T03:38:37.736404+0000","last_clean":"2024-06-06T03:38:37.736404+0000","last_became_active":"2024-06-06T03:35:13.859038+0000","last_became_peered":"2024-06-06T03:35:13.859038+0000","last_unstale":"2024-06-06T03:38:37.736404+0000","last_undegraded":"2024-06-06T03:38:37.736404+0000","last_fullsized":"2024-06-06T03:38:37.736404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:14:31.780064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890276+0000","last_change":"2024-06-06T03:35:13.859245+0000","last_active":"2024-06-06T03:38:37.890276+0000","last_peered":"2024-06-06T03:38:37.890276+0000","last_clean":"2024-06-06T03:38:37.890276+0000","last_became_active":"2024-06-06T03:35:13.858857+0000","last_became_peered":"2024-06-06T03:35:13.858857+0000","last_unstale":"2024-06-06T03:38:37.890276+0000","last_undegraded":"2024-06-06T03:38:37.890276+0000","last_fullsized":"2024-06-06T03:38:37.890276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:03:23.240007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717064+0000","last_change":"2024-06-06T03:35:13.851249+0000","last_active":"2024-06-06T03:38:37.717064+0000","last_peered":"2024-06-06T03:38:37.717064+0000","last_clean":"2024-06-06T03:38:37.717064+0000","last_became_active":"2024-06-06T03:35:13.850777+0000","last_became_peered":"2024-06-06T03:35:13.850777+0000","last_unstale":"2024-06-06T03:38:37.717064+0000","last_undegraded":"2024-06-06T03:38:37.717064+0000","last_fullsized":"2024-06-06T03:38:37.717064+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:53:48.463672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.645289+0000","last_change":"2024-06-06T03:35:13.852950+0000","last_active":"2024-06-06T03:38:37.645289+0000","last_peered":"2024-06-06T03:38:37.645289+0000","last_clean":"2024-06-06T03:38:37.645289+0000","last_became_active":"2024-06-06T03:35:13.852160+0000","last_became_peered":"2024-06-06T03:35:13.852160+0000","last_unstale":"2024-06-06T03:38:37.645289+0000","last_undegraded":"2024-06-06T03:38:37.645289+0000","last_fullsized":"2024-06-06T03:38:37.645289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:00.498440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736760+0000","last_change":"2024-06-06T03:35:13.856081+0000","last_active":"2024-06-06T03:38:37.736760+0000","last_peered":"2024-06-06T03:38:37.736760+0000","last_clean":"2024-06-06T03:38:37.736760+0000","last_became_active":"2024-06-06T03:35:13.855718+0000","last_became_peered":"2024-06-06T03:35:13.855718+0000","last_unstale":"2024-06-06T03:38:37.736760+0000","last_undegraded":"2024-06-06T03:38:37.736760+0000","last_fullsized":"2024-06-06T03:38:37.736760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:33:38.171996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890584+0000","last_change":"2024-06-06T03:35:13.861850+0000","last_active":"2024-06-06T03:38:37.890584+0000","last_peered":"2024-06-06T03:38:37.890584+0000","last_clean":"2024-06-06T03:38:37.890584+0000","last_became_active":"2024-06-06T03:35:13.861513+0000","last_became_peered":"2024-06-06T03:35:13.861513+0000","last_unstale":"2024-06-06T03:38:37.890584+0000","last_undegraded":"2024-06-06T03:38:37.890584+0000","last_fullsized":"2024-06-06T03:38:37.890584+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:32:45.938302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.629839+0000","last_change":"2024-06-06T03:35:13.847742+0000","last_active":"2024-06-06T03:38:37.629839+0000","last_peered":"2024-06-06T03:38:37.629839+0000","last_clean":"2024-06-06T03:38:37.629839+0000","last_became_active":"2024-06-06T03:35:13.847409+0000","last_became_peered":"2024-06-06T03:35:13.847409+0000","last_unstale":"2024-06-06T03:38:37.629839+0000","last_undegraded":"2024-06-06T03:38:37.629839+0000","last_fullsized":"2024-06-06T03:38:37.629839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:20:31.353919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890838+0000","last_change":"2024-06-06T03:35:13.848785+0000","last_active":"2024-06-06T03:38:37.890838+0000","last_peered":"2024-06-06T03:38:37.890838+0000","last_clean":"2024-06-06T03:38:37.890838+0000","last_became_active":"2024-06-06T03:35:13.848048+0000","last_became_peered":"2024-06-06T03:35:13.848048+0000","last_unstale":"2024-06-06T03:38:37.890838+0000","last_undegraded":"2024-06-06T03:38:37.890838+0000","last_fullsized":"2024-06-06T03:38:37.890838+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:43.657761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891499+0000","last_change":"2024-06-06T03:35:13.856083+0000","last_active":"2024-06-06T03:38:37.891499+0000","last_peered":"2024-06-06T03:38:37.891499+0000","last_clean":"2024-06-06T03:38:37.891499+0000","last_became_active":"2024-06-06T03:35:13.854390+0000","last_became_peered":"2024-06-06T03:35:13.854390+0000","last_unstale":"2024-06-06T03:38:37.891499+0000","last_undegraded":"2024-06-06T03:38:37.891499+0000","last_fullsized":"2024-06-06T03:38:37.891499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:51:37.163814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890857+0000","last_change":"2024-06-06T03:35:13.852611+0000","last_active":"2024-06-06T03:38:37.890857+0000","last_peered":"2024-06-06T03:38:37.890857+0000","last_clean":"2024-06-06T03:38:37.890857+0000","last_became_active":"2024-06-06T03:35:13.852244+0000","last_became_peered":"2024-06-06T03:35:13.852244+0000","last_unstale":"2024-06-06T03:38:37.890857+0000","last_undegraded":"2024-06-06T03:38:37.890857+0000","last_fullsized":"2024-06-06T03:38:37.890857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:59:53.513150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.644789+0000","last_change":"2024-06-06T03:35:13.857840+0000","last_active":"2024-06-06T03:38:37.644789+0000","last_peered":"2024-06-06T03:38:37.644789+0000","last_clean":"2024-06-06T03:38:37.644789+0000","last_became_active":"2024-06-06T03:35:13.857137+0000","last_became_peered":"2024-06-06T03:35:13.857137+0000","last_unstale":"2024-06-06T03:38:37.644789+0000","last_undegraded":"2024-06-06T03:38:37.644789+0000","last_fullsized":"2024-06-06T03:38:37.644789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:42:07.924735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890462+0000","last_change":"2024-06-06T03:35:13.857033+0000","last_active":"2024-06-06T03:38:37.890462+0000","last_peered":"2024-06-06T03:38:37.890462+0000","last_clean":"2024-06-06T03:38:37.890462+0000","last_became_active":"2024-06-06T03:35:13.856519+0000","last_became_peered":"2024-06-06T03:35:13.856519+0000","last_unstale":"2024-06-06T03:38:37.890462+0000","last_undegraded":"2024-06-06T03:38:37.890462+0000","last_fullsized":"2024-06-06T03:38:37.890462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:05:25.668344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630726+0000","last_change":"2024-06-06T03:35:13.851356+0000","last_active":"2024-06-06T03:38:37.630726+0000","last_peered":"2024-06-06T03:38:37.630726+0000","last_clean":"2024-06-06T03:38:37.630726+0000","last_became_active":"2024-06-06T03:35:13.851030+0000","last_became_peered":"2024-06-06T03:35:13.851030+0000","last_unstale":"2024-06-06T03:38:37.630726+0000","last_undegraded":"2024-06-06T03:38:37.630726+0000","last_fullsized":"2024-06-06T03:38:37.630726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:49:38.833022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632287+0000","last_change":"2024-06-06T03:35:13.858613+0000","last_active":"2024-06-06T03:38:37.632287+0000","last_peered":"2024-06-06T03:38:37.632287+0000","last_clean":"2024-06-06T03:38:37.632287+0000","last_became_active":"2024-06-06T03:35:13.858182+0000","last_became_peered":"2024-06-06T03:35:13.858182+0000","last_unstale":"2024-06-06T03:38:37.632287+0000","last_undegraded":"2024-06-06T03:38:37.632287+0000","last_fullsized":"2024-06-06T03:38:37.632287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:23:08.216431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'15","reported_seq":229,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630796+0000","last_change":"2024-06-06T03:35:09.821896+0000","last_active":"2024-06-06T03:38:37.630796+0000","last_peered":"2024-06-06T03:38:37.630796+0000","last_clean":"2024-06-06T03:38:37.630796+0000","last_became_active":"2024-06-06T03:35:09.821567+0000","last_became_peered":"2024-06-06T03:35:09.821567+0000","last_unstale":"2024-06-06T03:38:37.630796+0000","last_undegraded":"2024-06-06T03:38:37.630796+0000","last_fullsized":"2024-06-06T03:38:37.630796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:04:51.918393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717587+0000","last_change":"2024-06-06T03:35:13.857357+0000","last_active":"2024-06-06T03:38:37.717587+0000","last_peered":"2024-06-06T03:38:37.717587+0000","last_clean":"2024-06-06T03:38:37.717587+0000","last_became_active":"2024-06-06T03:35:13.857016+0000","last_became_peered":"2024-06-06T03:35:13.857016+0000","last_unstale":"2024-06-06T03:38:37.717587+0000","last_undegraded":"2024-06-06T03:38:37.717587+0000","last_fullsized":"2024-06-06T03:38:37.717587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:42.566425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717796+0000","last_change":"2024-06-06T03:35:10.277316+0000","last_active":"2024-06-06T03:38:37.717796+0000","last_peered":"2024-06-06T03:38:37.717796+0000","last_clean":"2024-06-06T03:38:37.717796+0000","last_became_active":"2024-06-06T03:35:10.276986+0000","last_became_peered":"2024-06-06T03:35:10.276986+0000","last_unstale":"2024-06-06T03:38:37.717796+0000","last_undegraded":"2024-06-06T03:38:37.717796+0000","last_fullsized":"2024-06-06T03:38:37.717796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:17:29.034224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.892351+0000","last_change":"2024-06-06T03:35:13.863069+0000","last_active":"2024-06-06T03:38:37.892351+0000","last_peered":"2024-06-06T03:38:37.892351+0000","last_clean":"2024-06-06T03:38:37.892351+0000","last_became_active":"2024-06-06T03:35:13.862647+0000","last_became_peered":"2024-06-06T03:35:13.862647+0000","last_unstale":"2024-06-06T03:38:37.892351+0000","last_undegraded":"2024-06-06T03:38:37.892351+0000","last_fullsized":"2024-06-06T03:38:37.892351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:48.252606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'13","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728456+0000","last_change":"2024-06-06T03:35:09.824977+0000","last_active":"2024-06-06T03:38:37.728456+0000","last_peered":"2024-06-06T03:38:37.728456+0000","last_clean":"2024-06-06T03:38:37.728456+0000","last_became_active":"2024-06-06T03:35:09.824643+0000","last_became_peered":"2024-06-06T03:35:09.824643+0000","last_unstale":"2024-06-06T03:38:37.728456+0000","last_undegraded":"2024-06-06T03:38:37.728456+0000","last_fullsized":"2024-06-06T03:38:37.728456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:35:37.208023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.644898+0000","last_change":"2024-06-06T03:35:13.860572+0000","last_active":"2024-06-06T03:38:37.644898+0000","last_peered":"2024-06-06T03:38:37.644898+0000","last_clean":"2024-06-06T03:38:37.644898+0000","last_became_active":"2024-06-06T03:35:13.858398+0000","last_became_peered":"2024-06-06T03:35:13.858398+0000","last_unstale":"2024-06-06T03:38:37.644898+0000","last_undegraded":"2024-06-06T03:38:37.644898+0000","last_fullsized":"2024-06-06T03:38:37.644898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:02:12.683096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728187+0000","last_change":"2024-06-06T03:35:09.827016+0000","last_active":"2024-06-06T03:38:37.728187+0000","last_peered":"2024-06-06T03:38:37.728187+0000","last_clean":"2024-06-06T03:38:37.728187+0000","last_became_active":"2024-06-06T03:35:09.826576+0000","last_became_peered":"2024-06-06T03:35:09.826576+0000","last_unstale":"2024-06-06T03:38:37.728187+0000","last_undegraded":"2024-06-06T03:38:37.728187+0000","last_fullsized":"2024-06-06T03:38:37.728187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:24.120549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733815+0000","last_change":"2024-06-06T03:35:13.853988+0000","last_active":"2024-06-06T03:38:37.733815+0000","last_peered":"2024-06-06T03:38:37.733815+0000","last_clean":"2024-06-06T03:38:37.733815+0000","last_became_active":"2024-06-06T03:35:13.852343+0000","last_became_peered":"2024-06-06T03:35:13.852343+0000","last_unstale":"2024-06-06T03:38:37.733815+0000","last_undegraded":"2024-06-06T03:38:37.733815+0000","last_fullsized":"2024-06-06T03:38:37.733815+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:39:03.054045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":226,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:38.205587+0000","last_change":"2024-06-06T03:35:09.829109+0000","last_active":"2024-06-06T03:38:38.205587+0000","last_peered":"2024-06-06T03:38:38.205587+0000","last_clean":"2024-06-06T03:38:38.205587+0000","last_became_active":"2024-06-06T03:35:09.828734+0000","last_became_peered":"2024-06-06T03:35:09.828734+0000","last_unstale":"2024-06-06T03:38:38.205587+0000","last_undegraded":"2024-06-06T03:38:38.205587+0000","last_fullsized":"2024-06-06T03:38:38.205587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:50.475600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.727932+0000","last_change":"2024-06-06T03:35:13.859198+0000","last_active":"2024-06-06T03:38:37.727932+0000","last_peered":"2024-06-06T03:38:37.727932+0000","last_clean":"2024-06-06T03:38:37.727932+0000","last_became_active":"2024-06-06T03:35:13.858863+0000","last_became_peered":"2024-06-06T03:35:13.858863+0000","last_unstale":"2024-06-06T03:38:37.727932+0000","last_undegraded":"2024-06-06T03:38:37.727932+0000","last_fullsized":"2024-06-06T03:38:37.727932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:57.153174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736340+0000","last_change":"2024-06-06T03:35:09.824352+0000","last_active":"2024-06-06T03:38:37.736340+0000","last_peered":"2024-06-06T03:38:37.736340+0000","last_clean":"2024-06-06T03:38:37.736340+0000","last_became_active":"2024-06-06T03:35:09.823998+0000","last_became_peered":"2024-06-06T03:35:09.823998+0000","last_unstale":"2024-06-06T03:38:37.736340+0000","last_undegraded":"2024-06-06T03:38:37.736340+0000","last_fullsized":"2024-06-06T03:38:37.736340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:47:24.646067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733489+0000","last_change":"2024-06-06T03:35:13.836724+0000","last_active":"2024-06-06T03:38:37.733489+0000","last_peered":"2024-06-06T03:38:37.733489+0000","last_clean":"2024-06-06T03:38:37.733489+0000","last_became_active":"2024-06-06T03:35:13.836180+0000","last_became_peered":"2024-06-06T03:35:13.836180+0000","last_unstale":"2024-06-06T03:38:37.733489+0000","last_undegraded":"2024-06-06T03:38:37.733489+0000","last_fullsized":"2024-06-06T03:38:37.733489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:08.868462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890183+0000","last_change":"2024-06-06T03:35:09.823047+0000","last_active":"2024-06-06T03:38:37.890183+0000","last_peered":"2024-06-06T03:38:37.890183+0000","last_clean":"2024-06-06T03:38:37.890183+0000","last_became_active":"2024-06-06T03:35:09.822744+0000","last_became_peered":"2024-06-06T03:35:09.822744+0000","last_unstale":"2024-06-06T03:38:37.890183+0000","last_undegraded":"2024-06-06T03:38:37.890183+0000","last_fullsized":"2024-06-06T03:38:37.890183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:20:20.165682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.716965+0000","last_change":"2024-06-06T03:35:13.857222+0000","last_active":"2024-06-06T03:38:37.716965+0000","last_peered":"2024-06-06T03:38:37.716965+0000","last_clean":"2024-06-06T03:38:37.716965+0000","last_became_active":"2024-06-06T03:35:13.856873+0000","last_became_peered":"2024-06-06T03:35:13.856873+0000","last_unstale":"2024-06-06T03:38:37.716965+0000","last_undegraded":"2024-06-06T03:38:37.716965+0000","last_fullsized":"2024-06-06T03:38:37.716965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:32:16.646275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733256+0000","last_change":"2024-06-06T03:35:09.826036+0000","last_active":"2024-06-06T03:38:37.733256+0000","last_peered":"2024-06-06T03:38:37.733256+0000","last_clean":"2024-06-06T03:38:37.733256+0000","last_became_active":"2024-06-06T03:35:09.825690+0000","last_became_peered":"2024-06-06T03:35:09.825690+0000","last_unstale":"2024-06-06T03:38:37.733256+0000","last_undegraded":"2024-06-06T03:38:37.733256+0000","last_fullsized":"2024-06-06T03:38:37.733256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:03:35.858828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630424+0000","last_change":"2024-06-06T03:35:13.849016+0000","last_active":"2024-06-06T03:38:37.630424+0000","last_peered":"2024-06-06T03:38:37.630424+0000","last_clean":"2024-06-06T03:38:37.630424+0000","last_became_active":"2024-06-06T03:35:13.848709+0000","last_became_peered":"2024-06-06T03:35:13.848709+0000","last_unstale":"2024-06-06T03:38:37.630424+0000","last_undegraded":"2024-06-06T03:38:37.630424+0000","last_fullsized":"2024-06-06T03:38:37.630424+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:42.778612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":404,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:38.205096+0000","last_change":"2024-06-06T03:35:09.827072+0000","last_active":"2024-06-06T03:38:38.205096+0000","last_peered":"2024-06-06T03:38:38.205096+0000","last_clean":"2024-06-06T03:38:38.205096+0000","last_became_active":"2024-06-06T03:35:09.826734+0000","last_became_peered":"2024-06-06T03:35:09.826734+0000","last_unstale":"2024-06-06T03:38:38.205096+0000","last_undegraded":"2024-06-06T03:38:38.205096+0000","last_fullsized":"2024-06-06T03:38:38.205096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:07:13.388398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":368,"num_read_kb":368,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.645230+0000","last_change":"2024-06-06T03:35:13.856883+0000","last_active":"2024-06-06T03:38:37.645230+0000","last_peered":"2024-06-06T03:38:37.645230+0000","last_clean":"2024-06-06T03:38:37.645230+0000","last_became_active":"2024-06-06T03:35:13.856521+0000","last_became_peered":"2024-06-06T03:35:13.856521+0000","last_unstale":"2024-06-06T03:38:37.645230+0000","last_undegraded":"2024-06-06T03:38:37.645230+0000","last_fullsized":"2024-06-06T03:38:37.645230+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:19.466218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890304+0000","last_change":"2024-06-06T03:35:09.829153+0000","last_active":"2024-06-06T03:38:37.890304+0000","last_peered":"2024-06-06T03:38:37.890304+0000","last_clean":"2024-06-06T03:38:37.890304+0000","last_became_active":"2024-06-06T03:35:09.828797+0000","last_became_peered":"2024-06-06T03:35:09.828797+0000","last_unstale":"2024-06-06T03:38:37.890304+0000","last_undegraded":"2024-06-06T03:38:37.890304+0000","last_fullsized":"2024-06-06T03:38:37.890304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:06:02.986396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.892165+0000","last_change":"2024-06-06T03:35:13.857094+0000","last_active":"2024-06-06T03:38:37.892165+0000","last_peered":"2024-06-06T03:38:37.892165+0000","last_clean":"2024-06-06T03:38:37.892165+0000","last_became_active":"2024-06-06T03:35:13.856778+0000","last_became_peered":"2024-06-06T03:35:13.856778+0000","last_unstale":"2024-06-06T03:38:37.892165+0000","last_undegraded":"2024-06-06T03:38:37.892165+0000","last_fullsized":"2024-06-06T03:38:37.892165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:50:30.097762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733828+0000","last_change":"2024-06-06T03:35:10.277802+0000","last_active":"2024-06-06T03:38:37.733828+0000","last_peered":"2024-06-06T03:38:37.733828+0000","last_clean":"2024-06-06T03:38:37.733828+0000","last_became_active":"2024-06-06T03:35:10.277417+0000","last_became_peered":"2024-06-06T03:35:10.277417+0000","last_unstale":"2024-06-06T03:38:37.733828+0000","last_undegraded":"2024-06-06T03:38:37.733828+0000","last_fullsized":"2024-06-06T03:38:37.733828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:44:35.178184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.645024+0000","last_change":"2024-06-06T03:35:13.858160+0000","last_active":"2024-06-06T03:38:37.645024+0000","last_peered":"2024-06-06T03:38:37.645024+0000","last_clean":"2024-06-06T03:38:37.645024+0000","last_became_active":"2024-06-06T03:35:13.853149+0000","last_became_peered":"2024-06-06T03:35:13.853149+0000","last_unstale":"2024-06-06T03:38:37.645024+0000","last_undegraded":"2024-06-06T03:38:37.645024+0000","last_fullsized":"2024-06-06T03:38:37.645024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:10:06.697274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":447,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:38.205556+0000","last_change":"2024-06-06T03:35:09.829443+0000","last_active":"2024-06-06T03:38:38.205556+0000","last_peered":"2024-06-06T03:38:38.205556+0000","last_clean":"2024-06-06T03:38:38.205556+0000","last_became_active":"2024-06-06T03:35:09.828093+0000","last_became_peered":"2024-06-06T03:35:09.828093+0000","last_unstale":"2024-06-06T03:38:38.205556+0000","last_undegraded":"2024-06-06T03:38:38.205556+0000","last_fullsized":"2024-06-06T03:38:38.205556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:05:19.395177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67985,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":380,"num_read_kb":256,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728167+0000","last_change":"2024-06-06T03:35:13.850869+0000","last_active":"2024-06-06T03:38:37.728167+0000","last_peered":"2024-06-06T03:38:37.728167+0000","last_clean":"2024-06-06T03:38:37.728167+0000","last_became_active":"2024-06-06T03:35:13.849489+0000","last_became_peered":"2024-06-06T03:35:13.849489+0000","last_unstale":"2024-06-06T03:38:37.728167+0000","last_undegraded":"2024-06-06T03:38:37.728167+0000","last_fullsized":"2024-06-06T03:38:37.728167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:27:45.052033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632136+0000","last_change":"2024-06-06T03:35:09.823117+0000","last_active":"2024-06-06T03:38:37.632136+0000","last_peered":"2024-06-06T03:38:37.632136+0000","last_clean":"2024-06-06T03:38:37.632136+0000","last_became_active":"2024-06-06T03:35:09.822624+0000","last_became_peered":"2024-06-06T03:35:09.822624+0000","last_unstale":"2024-06-06T03:38:37.632136+0000","last_undegraded":"2024-06-06T03:38:37.632136+0000","last_fullsized":"2024-06-06T03:38:37.632136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:59.171343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890529+0000","last_change":"2024-06-06T03:35:13.861043+0000","last_active":"2024-06-06T03:38:37.890529+0000","last_peered":"2024-06-06T03:38:37.890529+0000","last_clean":"2024-06-06T03:38:37.890529+0000","last_became_active":"2024-06-06T03:35:13.860718+0000","last_became_peered":"2024-06-06T03:35:13.860718+0000","last_unstale":"2024-06-06T03:38:37.890529+0000","last_undegraded":"2024-06-06T03:38:37.890529+0000","last_fullsized":"2024-06-06T03:38:37.890529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:34:04.595780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733532+0000","last_change":"2024-06-06T03:35:09.825505+0000","last_active":"2024-06-06T03:38:37.733532+0000","last_peered":"2024-06-06T03:38:37.733532+0000","last_clean":"2024-06-06T03:38:37.733532+0000","last_became_active":"2024-06-06T03:35:09.825127+0000","last_became_peered":"2024-06-06T03:35:09.825127+0000","last_unstale":"2024-06-06T03:38:37.733532+0000","last_undegraded":"2024-06-06T03:38:37.733532+0000","last_fullsized":"2024-06-06T03:38:37.733532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:07:27.664589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890667+0000","last_change":"2024-06-06T03:35:48.987834+0000","last_active":"2024-06-06T03:38:37.890667+0000","last_peered":"2024-06-06T03:38:37.890667+0000","last_clean":"2024-06-06T03:38:37.890667+0000","last_became_active":"2024-06-06T03:35:48.987515+0000","last_became_peered":"2024-06-06T03:35:48.987515+0000","last_unstale":"2024-06-06T03:38:37.890667+0000","last_undegraded":"2024-06-06T03:38:37.890667+0000","last_fullsized":"2024-06-06T03:38:37.890667+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:05:19.342289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":223,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:38.205152+0000","last_change":"2024-06-06T03:35:09.826192+0000","last_active":"2024-06-06T03:38:38.205152+0000","last_peered":"2024-06-06T03:38:38.205152+0000","last_clean":"2024-06-06T03:38:38.205152+0000","last_became_active":"2024-06-06T03:35:09.825481+0000","last_became_peered":"2024-06-06T03:35:09.825481+0000","last_unstale":"2024-06-06T03:38:38.205152+0000","last_undegraded":"2024-06-06T03:38:38.205152+0000","last_fullsized":"2024-06-06T03:38:38.205152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:18:30.772991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1368,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":188,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890452+0000","last_change":"2024-06-06T03:35:13.853113+0000","last_active":"2024-06-06T03:38:37.890452+0000","last_peered":"2024-06-06T03:38:37.890452+0000","last_clean":"2024-06-06T03:38:37.890452+0000","last_became_active":"2024-06-06T03:35:13.852751+0000","last_became_peered":"2024-06-06T03:35:13.852751+0000","last_unstale":"2024-06-06T03:38:37.890452+0000","last_undegraded":"2024-06-06T03:38:37.890452+0000","last_fullsized":"2024-06-06T03:38:37.890452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:53.536685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890032+0000","last_change":"2024-06-06T03:35:49.005958+0000","last_active":"2024-06-06T03:38:37.890032+0000","last_peered":"2024-06-06T03:38:37.890032+0000","last_clean":"2024-06-06T03:38:37.890032+0000","last_became_active":"2024-06-06T03:35:48.993266+0000","last_became_peered":"2024-06-06T03:35:48.993266+0000","last_unstale":"2024-06-06T03:38:37.890032+0000","last_undegraded":"2024-06-06T03:38:37.890032+0000","last_fullsized":"2024-06-06T03:38:37.890032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:47.782824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733337+0000","last_change":"2024-06-06T03:35:13.853614+0000","last_active":"2024-06-06T03:38:37.733337+0000","last_peered":"2024-06-06T03:38:37.733337+0000","last_clean":"2024-06-06T03:38:37.733337+0000","last_became_active":"2024-06-06T03:35:13.852327+0000","last_became_peered":"2024-06-06T03:35:13.852327+0000","last_unstale":"2024-06-06T03:38:37.733337+0000","last_undegraded":"2024-06-06T03:38:37.733337+0000","last_fullsized":"2024-06-06T03:38:37.733337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:02:32.463028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728378+0000","last_change":"2024-06-06T03:35:09.826220+0000","last_active":"2024-06-06T03:38:37.728378+0000","last_peered":"2024-06-06T03:38:37.728378+0000","last_clean":"2024-06-06T03:38:37.728378+0000","last_became_active":"2024-06-06T03:35:09.825887+0000","last_became_peered":"2024-06-06T03:35:09.825887+0000","last_unstale":"2024-06-06T03:38:37.728378+0000","last_undegraded":"2024-06-06T03:38:37.728378+0000","last_fullsized":"2024-06-06T03:38:37.728378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:23:33.704822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890232+0000","last_change":"2024-06-06T03:35:13.852455+0000","last_active":"2024-06-06T03:38:37.890232+0000","last_peered":"2024-06-06T03:38:37.890232+0000","last_clean":"2024-06-06T03:38:37.890232+0000","last_became_active":"2024-06-06T03:35:13.852134+0000","last_became_peered":"2024-06-06T03:35:13.852134+0000","last_unstale":"2024-06-06T03:38:37.890232+0000","last_undegraded":"2024-06-06T03:38:37.890232+0000","last_fullsized":"2024-06-06T03:38:37.890232+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:00:16.418479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630691+0000","last_change":"2024-06-06T03:35:10.277767+0000","last_active":"2024-06-06T03:38:37.630691+0000","last_peered":"2024-06-06T03:38:37.630691+0000","last_clean":"2024-06-06T03:38:37.630691+0000","last_became_active":"2024-06-06T03:35:10.277381+0000","last_became_peered":"2024-06-06T03:35:10.277381+0000","last_unstale":"2024-06-06T03:38:37.630691+0000","last_undegraded":"2024-06-06T03:38:37.630691+0000","last_fullsized":"2024-06-06T03:38:37.630691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:12:13.425284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736536+0000","last_change":"2024-06-06T03:35:13.859594+0000","last_active":"2024-06-06T03:38:37.736536+0000","last_peered":"2024-06-06T03:38:37.736536+0000","last_clean":"2024-06-06T03:38:37.736536+0000","last_became_active":"2024-06-06T03:35:13.859240+0000","last_became_peered":"2024-06-06T03:35:13.859240+0000","last_unstale":"2024-06-06T03:38:37.736536+0000","last_undegraded":"2024-06-06T03:38:37.736536+0000","last_fullsized":"2024-06-06T03:38:37.736536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:39:34.557581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'8","reported_seq":225,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890475+0000","last_change":"2024-06-06T03:35:09.826204+0000","last_active":"2024-06-06T03:38:37.890475+0000","last_peered":"2024-06-06T03:38:37.890475+0000","last_clean":"2024-06-06T03:38:37.890475+0000","last_became_active":"2024-06-06T03:35:09.825815+0000","last_became_peered":"2024-06-06T03:35:09.825815+0000","last_unstale":"2024-06-06T03:38:37.890475+0000","last_undegraded":"2024-06-06T03:38:37.890475+0000","last_fullsized":"2024-06-06T03:38:37.890475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:02:04.169227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1427,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":188,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736428+0000","last_change":"2024-06-06T03:35:48.992786+0000","last_active":"2024-06-06T03:38:37.736428+0000","last_peered":"2024-06-06T03:38:37.736428+0000","last_clean":"2024-06-06T03:38:37.736428+0000","last_became_active":"2024-06-06T03:35:48.992456+0000","last_became_peered":"2024-06-06T03:35:48.992456+0000","last_unstale":"2024-06-06T03:38:37.736428+0000","last_undegraded":"2024-06-06T03:38:37.736428+0000","last_fullsized":"2024-06-06T03:38:37.736428+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:06:58.563975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630421+0000","last_change":"2024-06-06T03:35:49.005119+0000","last_active":"2024-06-06T03:38:37.630421+0000","last_peered":"2024-06-06T03:38:37.630421+0000","last_clean":"2024-06-06T03:38:37.630421+0000","last_became_active":"2024-06-06T03:35:49.004788+0000","last_became_peered":"2024-06-06T03:35:49.004788+0000","last_unstale":"2024-06-06T03:38:37.630421+0000","last_undegraded":"2024-06-06T03:38:37.630421+0000","last_fullsized":"2024-06-06T03:38:37.630421+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:20:06.961454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":220,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.892812+0000","last_change":"2024-06-06T03:35:10.278559+0000","last_active":"2024-06-06T03:38:37.892812+0000","last_peered":"2024-06-06T03:38:37.892812+0000","last_clean":"2024-06-06T03:38:37.892812+0000","last_became_active":"2024-06-06T03:35:10.278104+0000","last_became_peered":"2024-06-06T03:35:10.278104+0000","last_unstale":"2024-06-06T03:38:37.892812+0000","last_undegraded":"2024-06-06T03:38:37.892812+0000","last_fullsized":"2024-06-06T03:38:37.892812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:23:47.131731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717376+0000","last_change":"2024-06-06T03:35:13.862303+0000","last_active":"2024-06-06T03:38:37.717376+0000","last_peered":"2024-06-06T03:38:37.717376+0000","last_clean":"2024-06-06T03:38:37.717376+0000","last_became_active":"2024-06-06T03:35:13.861978+0000","last_became_peered":"2024-06-06T03:35:13.861978+0000","last_unstale":"2024-06-06T03:38:37.717376+0000","last_undegraded":"2024-06-06T03:38:37.717376+0000","last_fullsized":"2024-06-06T03:38:37.717376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:24:00.824646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.629723+0000","last_change":"2024-06-06T03:35:09.825365+0000","last_active":"2024-06-06T03:38:37.629723+0000","last_peered":"2024-06-06T03:38:37.629723+0000","last_clean":"2024-06-06T03:38:37.629723+0000","last_became_active":"2024-06-06T03:35:09.824948+0000","last_became_peered":"2024-06-06T03:35:09.824948+0000","last_unstale":"2024-06-06T03:38:37.629723+0000","last_undegraded":"2024-06-06T03:38:37.629723+0000","last_fullsized":"2024-06-06T03:38:37.629723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:23:48.189292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890531+0000","last_change":"2024-06-06T03:33:44.124626+0000","last_active":"2024-06-06T03:38:37.890531+0000","last_peered":"2024-06-06T03:38:37.890531+0000","last_clean":"2024-06-06T03:38:37.890531+0000","last_became_active":"2024-06-06T03:33:44.124046+0000","last_became_peered":"2024-06-06T03:33:44.124046+0000","last_unstale":"2024-06-06T03:38:37.890531+0000","last_undegraded":"2024-06-06T03:38:37.890531+0000","last_fullsized":"2024-06-06T03:38:37.890531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:30:14.586719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:30:14.586719+0000","last_clean_scrub_stamp":"2024-06-06T03:30:14.586719+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:17.604180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890344+0000","last_change":"2024-06-06T03:35:13.852044+0000","last_active":"2024-06-06T03:38:37.890344+0000","last_peered":"2024-06-06T03:38:37.890344+0000","last_clean":"2024-06-06T03:38:37.890344+0000","last_became_active":"2024-06-06T03:35:13.851715+0000","last_became_peered":"2024-06-06T03:35:13.851715+0000","last_unstale":"2024-06-06T03:38:37.890344+0000","last_undegraded":"2024-06-06T03:38:37.890344+0000","last_fullsized":"2024-06-06T03:38:37.890344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:19:23.778367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890093+0000","last_change":"2024-06-06T03:35:09.828463+0000","last_active":"2024-06-06T03:38:37.890093+0000","last_peered":"2024-06-06T03:38:37.890093+0000","last_clean":"2024-06-06T03:38:37.890093+0000","last_became_active":"2024-06-06T03:35:09.828121+0000","last_became_peered":"2024-06-06T03:35:09.828121+0000","last_unstale":"2024-06-06T03:38:37.890093+0000","last_undegraded":"2024-06-06T03:38:37.890093+0000","last_fullsized":"2024-06-06T03:38:37.890093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:54:25.095741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728370+0000","last_change":"2024-06-06T03:35:10.277901+0000","last_active":"2024-06-06T03:38:37.728370+0000","last_peered":"2024-06-06T03:38:37.728370+0000","last_clean":"2024-06-06T03:38:37.728370+0000","last_became_active":"2024-06-06T03:35:10.277447+0000","last_became_peered":"2024-06-06T03:35:10.277447+0000","last_unstale":"2024-06-06T03:38:37.728370+0000","last_undegraded":"2024-06-06T03:38:37.728370+0000","last_fullsized":"2024-06-06T03:38:37.728370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:22:50.550287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891033+0000","last_change":"2024-06-06T03:35:13.864323+0000","last_active":"2024-06-06T03:38:37.891033+0000","last_peered":"2024-06-06T03:38:37.891033+0000","last_clean":"2024-06-06T03:38:37.891033+0000","last_became_active":"2024-06-06T03:35:13.863101+0000","last_became_peered":"2024-06-06T03:35:13.863101+0000","last_unstale":"2024-06-06T03:38:37.891033+0000","last_undegraded":"2024-06-06T03:38:37.891033+0000","last_fullsized":"2024-06-06T03:38:37.891033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:32.399325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'51","reported_seq":103,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.843821+0000","last_change":"2024-06-06T03:35:09.822280+0000","last_active":"2024-06-06T03:38:37.843821+0000","last_peered":"2024-06-06T03:38:37.843821+0000","last_clean":"2024-06-06T03:38:37.843821+0000","last_became_active":"2024-06-06T03:35:09.821922+0000","last_became_peered":"2024-06-06T03:35:09.821922+0000","last_unstale":"2024-06-06T03:38:37.843821+0000","last_undegraded":"2024-06-06T03:38:37.843821+0000","last_fullsized":"2024-06-06T03:38:37.843821+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:00:03.328747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717835+0000","last_change":"2024-06-06T03:35:13.853538+0000","last_active":"2024-06-06T03:38:37.717835+0000","last_peered":"2024-06-06T03:38:37.717835+0000","last_clean":"2024-06-06T03:38:37.717835+0000","last_became_active":"2024-06-06T03:35:13.853200+0000","last_became_peered":"2024-06-06T03:35:13.853200+0000","last_unstale":"2024-06-06T03:38:37.717835+0000","last_undegraded":"2024-06-06T03:38:37.717835+0000","last_fullsized":"2024-06-06T03:38:37.717835+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:13:06.574272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630204+0000","last_change":"2024-06-06T03:35:10.761675+0000","last_active":"2024-06-06T03:38:37.630204+0000","last_peered":"2024-06-06T03:38:37.630204+0000","last_clean":"2024-06-06T03:38:37.630204+0000","last_became_active":"2024-06-06T03:35:10.761287+0000","last_became_peered":"2024-06-06T03:35:10.761287+0000","last_unstale":"2024-06-06T03:38:37.630204+0000","last_undegraded":"2024-06-06T03:38:37.630204+0000","last_fullsized":"2024-06-06T03:38:37.630204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:58:18.524273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.892143+0000","last_change":"2024-06-06T03:35:13.847680+0000","last_active":"2024-06-06T03:38:37.892143+0000","last_peered":"2024-06-06T03:38:37.892143+0000","last_clean":"2024-06-06T03:38:37.892143+0000","last_became_active":"2024-06-06T03:35:13.847359+0000","last_became_peered":"2024-06-06T03:35:13.847359+0000","last_unstale":"2024-06-06T03:38:37.892143+0000","last_undegraded":"2024-06-06T03:38:37.892143+0000","last_fullsized":"2024-06-06T03:38:37.892143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:29:18.041078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.644794+0000","last_change":"2024-06-06T03:35:10.277491+0000","last_active":"2024-06-06T03:38:37.644794+0000","last_peered":"2024-06-06T03:38:37.644794+0000","last_clean":"2024-06-06T03:38:37.644794+0000","last_became_active":"2024-06-06T03:35:10.277135+0000","last_became_peered":"2024-06-06T03:35:10.277135+0000","last_unstale":"2024-06-06T03:38:37.644794+0000","last_undegraded":"2024-06-06T03:38:37.644794+0000","last_fullsized":"2024-06-06T03:38:37.644794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:55.198789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.727917+0000","last_change":"2024-06-06T03:35:13.846013+0000","last_active":"2024-06-06T03:38:37.727917+0000","last_peered":"2024-06-06T03:38:37.727917+0000","last_clean":"2024-06-06T03:38:37.727917+0000","last_became_active":"2024-06-06T03:35:13.845650+0000","last_became_peered":"2024-06-06T03:35:13.845650+0000","last_unstale":"2024-06-06T03:38:37.727917+0000","last_undegraded":"2024-06-06T03:38:37.727917+0000","last_fullsized":"2024-06-06T03:38:37.727917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:51.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.716995+0000","last_change":"2024-06-06T03:35:09.826380+0000","last_active":"2024-06-06T03:38:37.716995+0000","last_peered":"2024-06-06T03:38:37.716995+0000","last_clean":"2024-06-06T03:38:37.716995+0000","last_became_active":"2024-06-06T03:35:09.825813+0000","last_became_peered":"2024-06-06T03:35:09.825813+0000","last_unstale":"2024-06-06T03:38:37.716995+0000","last_undegraded":"2024-06-06T03:38:37.716995+0000","last_fullsized":"2024-06-06T03:38:37.716995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:24.792398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890127+0000","last_change":"2024-06-06T03:35:13.855051+0000","last_active":"2024-06-06T03:38:37.890127+0000","last_peered":"2024-06-06T03:38:37.890127+0000","last_clean":"2024-06-06T03:38:37.890127+0000","last_became_active":"2024-06-06T03:35:13.853781+0000","last_became_peered":"2024-06-06T03:35:13.853781+0000","last_unstale":"2024-06-06T03:38:37.890127+0000","last_undegraded":"2024-06-06T03:38:37.890127+0000","last_fullsized":"2024-06-06T03:38:37.890127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:17:56.174924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890090+0000","last_change":"2024-06-06T03:35:49.009583+0000","last_active":"2024-06-06T03:38:37.890090+0000","last_peered":"2024-06-06T03:38:37.890090+0000","last_clean":"2024-06-06T03:38:37.890090+0000","last_became_active":"2024-06-06T03:35:48.994275+0000","last_became_peered":"2024-06-06T03:35:48.994275+0000","last_unstale":"2024-06-06T03:38:37.890090+0000","last_undegraded":"2024-06-06T03:38:37.890090+0000","last_fullsized":"2024-06-06T03:38:37.890090+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:24:54.985854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717547+0000","last_change":"2024-06-06T03:35:13.855503+0000","last_active":"2024-06-06T03:38:37.717547+0000","last_peered":"2024-06-06T03:38:37.717547+0000","last_clean":"2024-06-06T03:38:37.717547+0000","last_became_active":"2024-06-06T03:35:13.855069+0000","last_became_peered":"2024-06-06T03:35:13.855069+0000","last_unstale":"2024-06-06T03:38:37.717547+0000","last_undegraded":"2024-06-06T03:38:37.717547+0000","last_fullsized":"2024-06-06T03:38:37.717547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:43.778258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.727997+0000","last_change":"2024-06-06T03:35:09.827116+0000","last_active":"2024-06-06T03:38:37.727997+0000","last_peered":"2024-06-06T03:38:37.727997+0000","last_clean":"2024-06-06T03:38:37.727997+0000","last_became_active":"2024-06-06T03:35:09.826756+0000","last_became_peered":"2024-06-06T03:35:09.826756+0000","last_unstale":"2024-06-06T03:38:37.727997+0000","last_undegraded":"2024-06-06T03:38:37.727997+0000","last_fullsized":"2024-06-06T03:38:37.727997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:31.501411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890818+0000","last_change":"2024-06-06T03:35:13.861189+0000","last_active":"2024-06-06T03:38:37.890818+0000","last_peered":"2024-06-06T03:38:37.890818+0000","last_clean":"2024-06-06T03:38:37.890818+0000","last_became_active":"2024-06-06T03:35:13.860751+0000","last_became_peered":"2024-06-06T03:35:13.860751+0000","last_unstale":"2024-06-06T03:38:37.890818+0000","last_undegraded":"2024-06-06T03:38:37.890818+0000","last_fullsized":"2024-06-06T03:38:37.890818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:16:34.036420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":223,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:38.239580+0000","last_change":"2024-06-06T03:35:10.761606+0000","last_active":"2024-06-06T03:38:38.239580+0000","last_peered":"2024-06-06T03:38:38.239580+0000","last_clean":"2024-06-06T03:38:38.239580+0000","last_became_active":"2024-06-06T03:35:10.761104+0000","last_became_peered":"2024-06-06T03:35:10.761104+0000","last_unstale":"2024-06-06T03:38:38.239580+0000","last_undegraded":"2024-06-06T03:38:38.239580+0000","last_fullsized":"2024-06-06T03:38:38.239580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:51:39.867088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1950,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":187,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890803+0000","last_change":"2024-06-06T03:35:13.856047+0000","last_active":"2024-06-06T03:38:37.890803+0000","last_peered":"2024-06-06T03:38:37.890803+0000","last_clean":"2024-06-06T03:38:37.890803+0000","last_became_active":"2024-06-06T03:35:13.855718+0000","last_became_peered":"2024-06-06T03:35:13.855718+0000","last_unstale":"2024-06-06T03:38:37.890803+0000","last_undegraded":"2024-06-06T03:38:37.890803+0000","last_fullsized":"2024-06-06T03:38:37.890803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:18:40.926872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":78,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.841606+0000","last_change":"2024-06-06T03:35:09.827283+0000","last_active":"2024-06-06T03:38:37.841606+0000","last_peered":"2024-06-06T03:38:37.841606+0000","last_clean":"2024-06-06T03:38:37.841606+0000","last_became_active":"2024-06-06T03:35:09.826595+0000","last_became_peered":"2024-06-06T03:35:09.826595+0000","last_unstale":"2024-06-06T03:38:37.841606+0000","last_undegraded":"2024-06-06T03:38:37.841606+0000","last_fullsized":"2024-06-06T03:38:37.841606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:34:16.398847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890527+0000","last_change":"2024-06-06T03:35:13.851900+0000","last_active":"2024-06-06T03:38:37.890527+0000","last_peered":"2024-06-06T03:38:37.890527+0000","last_clean":"2024-06-06T03:38:37.890527+0000","last_became_active":"2024-06-06T03:35:13.851477+0000","last_became_peered":"2024-06-06T03:35:13.851477+0000","last_unstale":"2024-06-06T03:38:37.890527+0000","last_undegraded":"2024-06-06T03:38:37.890527+0000","last_fullsized":"2024-06-06T03:38:37.890527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:31:54.001226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890429+0000","last_change":"2024-06-06T03:35:09.824675+0000","last_active":"2024-06-06T03:38:37.890429+0000","last_peered":"2024-06-06T03:38:37.890429+0000","last_clean":"2024-06-06T03:38:37.890429+0000","last_became_active":"2024-06-06T03:35:09.824347+0000","last_became_peered":"2024-06-06T03:35:09.824347+0000","last_unstale":"2024-06-06T03:38:37.890429+0000","last_undegraded":"2024-06-06T03:38:37.890429+0000","last_fullsized":"2024-06-06T03:38:37.890429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:01:43.080552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891036+0000","last_change":"2024-06-06T03:35:13.854706+0000","last_active":"2024-06-06T03:38:37.891036+0000","last_peered":"2024-06-06T03:38:37.891036+0000","last_clean":"2024-06-06T03:38:37.891036+0000","last_became_active":"2024-06-06T03:35:13.854392+0000","last_became_peered":"2024-06-06T03:35:13.854392+0000","last_unstale":"2024-06-06T03:38:37.891036+0000","last_undegraded":"2024-06-06T03:38:37.891036+0000","last_fullsized":"2024-06-06T03:38:37.891036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:58:10.102367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890213+0000","last_change":"2024-06-06T03:35:13.853594+0000","last_active":"2024-06-06T03:38:37.890213+0000","last_peered":"2024-06-06T03:38:37.890213+0000","last_clean":"2024-06-06T03:38:37.890213+0000","last_became_active":"2024-06-06T03:35:13.852962+0000","last_became_peered":"2024-06-06T03:35:13.852962+0000","last_unstale":"2024-06-06T03:38:37.890213+0000","last_undegraded":"2024-06-06T03:38:37.890213+0000","last_fullsized":"2024-06-06T03:38:37.890213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:02:44.233868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632115+0000","last_change":"2024-06-06T03:35:13.848790+0000","last_active":"2024-06-06T03:38:37.632115+0000","last_peered":"2024-06-06T03:38:37.632115+0000","last_clean":"2024-06-06T03:38:37.632115+0000","last_became_active":"2024-06-06T03:35:13.848442+0000","last_became_peered":"2024-06-06T03:35:13.848442+0000","last_unstale":"2024-06-06T03:38:37.632115+0000","last_undegraded":"2024-06-06T03:38:37.632115+0000","last_fullsized":"2024-06-06T03:38:37.632115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:51.382210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632024+0000","last_change":"2024-06-06T03:35:13.866999+0000","last_active":"2024-06-06T03:38:37.632024+0000","last_peered":"2024-06-06T03:38:37.632024+0000","last_clean":"2024-06-06T03:38:37.632024+0000","last_became_active":"2024-06-06T03:35:13.866664+0000","last_became_peered":"2024-06-06T03:35:13.866664+0000","last_unstale":"2024-06-06T03:38:37.632024+0000","last_undegraded":"2024-06-06T03:38:37.632024+0000","last_fullsized":"2024-06-06T03:38:37.632024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:31:14.166122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891400+0000","last_change":"2024-06-06T03:35:13.849062+0000","last_active":"2024-06-06T03:38:37.891400+0000","last_peered":"2024-06-06T03:38:37.891400+0000","last_clean":"2024-06-06T03:38:37.891400+0000","last_became_active":"2024-06-06T03:35:13.848750+0000","last_became_peered":"2024-06-06T03:35:13.848750+0000","last_unstale":"2024-06-06T03:38:37.891400+0000","last_undegraded":"2024-06-06T03:38:37.891400+0000","last_fullsized":"2024-06-06T03:38:37.891400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:55:58.084908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890931+0000","last_change":"2024-06-06T03:35:13.857601+0000","last_active":"2024-06-06T03:38:37.890931+0000","last_peered":"2024-06-06T03:38:37.890931+0000","last_clean":"2024-06-06T03:38:37.890931+0000","last_became_active":"2024-06-06T03:35:13.857270+0000","last_became_peered":"2024-06-06T03:35:13.857270+0000","last_unstale":"2024-06-06T03:38:37.890931+0000","last_undegraded":"2024-06-06T03:38:37.890931+0000","last_fullsized":"2024-06-06T03:38:37.890931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:56:29.970210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890367+0000","last_change":"2024-06-06T03:35:13.846699+0000","last_active":"2024-06-06T03:38:37.890367+0000","last_peered":"2024-06-06T03:38:37.890367+0000","last_clean":"2024-06-06T03:38:37.890367+0000","last_became_active":"2024-06-06T03:35:13.846355+0000","last_became_peered":"2024-06-06T03:35:13.846355+0000","last_unstale":"2024-06-06T03:38:37.890367+0000","last_undegraded":"2024-06-06T03:38:37.890367+0000","last_fullsized":"2024-06-06T03:38:37.890367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:50:22.537339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632406+0000","last_change":"2024-06-06T03:35:13.854013+0000","last_active":"2024-06-06T03:38:37.632406+0000","last_peered":"2024-06-06T03:38:37.632406+0000","last_clean":"2024-06-06T03:38:37.632406+0000","last_became_active":"2024-06-06T03:35:13.853739+0000","last_became_peered":"2024-06-06T03:35:13.853739+0000","last_unstale":"2024-06-06T03:38:37.632406+0000","last_undegraded":"2024-06-06T03:38:37.632406+0000","last_fullsized":"2024-06-06T03:38:37.632406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:27:12.330943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717814+0000","last_change":"2024-06-06T03:35:13.854251+0000","last_active":"2024-06-06T03:38:37.717814+0000","last_peered":"2024-06-06T03:38:37.717814+0000","last_clean":"2024-06-06T03:38:37.717814+0000","last_became_active":"2024-06-06T03:35:13.853915+0000","last_became_peered":"2024-06-06T03:35:13.853915+0000","last_unstale":"2024-06-06T03:38:37.717814+0000","last_undegraded":"2024-06-06T03:38:37.717814+0000","last_fullsized":"2024-06-06T03:38:37.717814+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:08.813316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733345+0000","last_change":"2024-06-06T03:35:13.849998+0000","last_active":"2024-06-06T03:38:37.733345+0000","last_peered":"2024-06-06T03:38:37.733345+0000","last_clean":"2024-06-06T03:38:37.733345+0000","last_became_active":"2024-06-06T03:35:13.848789+0000","last_became_peered":"2024-06-06T03:35:13.848789+0000","last_unstale":"2024-06-06T03:38:37.733345+0000","last_undegraded":"2024-06-06T03:38:37.733345+0000","last_fullsized":"2024-06-06T03:38:37.733345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:53:39.043356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.716965+0000","last_change":"2024-06-06T03:35:13.858020+0000","last_active":"2024-06-06T03:38:37.716965+0000","last_peered":"2024-06-06T03:38:37.716965+0000","last_clean":"2024-06-06T03:38:37.716965+0000","last_became_active":"2024-06-06T03:35:13.857723+0000","last_became_peered":"2024-06-06T03:35:13.857723+0000","last_unstale":"2024-06-06T03:38:37.716965+0000","last_undegraded":"2024-06-06T03:38:37.716965+0000","last_fullsized":"2024-06-06T03:38:37.716965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:08:08.651213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.644929+0000","last_change":"2024-06-06T03:35:13.867546+0000","last_active":"2024-06-06T03:38:37.644929+0000","last_peered":"2024-06-06T03:38:37.644929+0000","last_clean":"2024-06-06T03:38:37.644929+0000","last_became_active":"2024-06-06T03:35:13.867182+0000","last_became_peered":"2024-06-06T03:35:13.867182+0000","last_unstale":"2024-06-06T03:38:37.644929+0000","last_undegraded":"2024-06-06T03:38:37.644929+0000","last_fullsized":"2024-06-06T03:38:37.644929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:25:55.831943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890895+0000","last_change":"2024-06-06T03:35:13.849080+0000","last_active":"2024-06-06T03:38:37.890895+0000","last_peered":"2024-06-06T03:38:37.890895+0000","last_clean":"2024-06-06T03:38:37.890895+0000","last_became_active":"2024-06-06T03:35:13.848476+0000","last_became_peered":"2024-06-06T03:35:13.848476+0000","last_unstale":"2024-06-06T03:38:37.890895+0000","last_undegraded":"2024-06-06T03:38:37.890895+0000","last_fullsized":"2024-06-06T03:38:37.890895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:48:37.306528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890161+0000","last_change":"2024-06-06T03:35:13.848442+0000","last_active":"2024-06-06T03:38:37.890161+0000","last_peered":"2024-06-06T03:38:37.890161+0000","last_clean":"2024-06-06T03:38:37.890161+0000","last_became_active":"2024-06-06T03:35:13.848074+0000","last_became_peered":"2024-06-06T03:35:13.848074+0000","last_unstale":"2024-06-06T03:38:37.890161+0000","last_undegraded":"2024-06-06T03:38:37.890161+0000","last_fullsized":"2024-06-06T03:38:37.890161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:07:00.197671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107001,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602543,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1981,"num_read_kb":1108,"num_write":382,"num_write_kb":1471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1807629,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383737,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":480334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645374,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":1274582,"data_compressed":44602,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808899,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":882208,"data_compressed":22026,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070536,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":882474,"data_compressed":22026,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332173,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":1084037,"data_compressed":38547,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495698,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":489548,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":558446,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":481208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1097728,"data_stored":559528,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182498,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1013350,"data_compressed":28182,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920861,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":488570,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757336,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1105920,"data_stored":1213306,"data_compressed":44703,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135341,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202101,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T03:38:42.860 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T03:38:43.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi040 to 177.2M 2024-06-06T03:38:43.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: Unable to set osd_memory_target on smithi040 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-06T03:38:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: Updating smithi040:/etc/ceph/ceph.conf 2024-06-06T03:38:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: Updating smithi104:/etc/ceph/ceph.conf 2024-06-06T03:38:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: Updating smithi131:/etc/ceph/ceph.conf 2024-06-06T03:38:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:38:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: Updating smithi040:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:38:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: Updating smithi104:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:38:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: Updating smithi131:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:38:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:38:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:38:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:38:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: from='client.24880 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:38:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-06-06T03:38:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:38:43.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:38:43.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:38:43.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:42 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.113 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi040 to 177.2M 2024-06-06T03:38:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: Unable to set osd_memory_target on smithi040 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: Updating smithi040:/etc/ceph/ceph.conf 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: Updating smithi104:/etc/ceph/ceph.conf 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: Updating smithi131:/etc/ceph/ceph.conf 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: Updating smithi040:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: Updating smithi104:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: Updating smithi131:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: from='client.24880 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:38:43.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:38:43.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:38:43.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:42 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi040 to 177.2M 2024-06-06T03:38:43.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: Unable to set osd_memory_target on smithi040 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-06T03:38:43.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: Updating smithi040:/etc/ceph/ceph.conf 2024-06-06T03:38:43.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: Updating smithi104:/etc/ceph/ceph.conf 2024-06-06T03:38:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: Updating smithi131:/etc/ceph/ceph.conf 2024-06-06T03:38:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:38:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: Updating smithi040:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:38:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: Updating smithi104:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:38:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: Updating smithi131:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T03:38:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:38:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:38:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-06-06T03:38:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: from='client.24880 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:38:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-06-06T03:38:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:43.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:38:43.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:38:43.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:38:43.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:42 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:38:44.109 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:43 smithi040 ceph-mon[32220]: Updating smithi040:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T03:38:44.110 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:43 smithi040 ceph-mon[32220]: Updating smithi104:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T03:38:44.110 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:43 smithi040 ceph-mon[32220]: Updating smithi131:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T03:38:44.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:43 smithi131 ceph-mon[36687]: Updating smithi040:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T03:38:44.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:43 smithi131 ceph-mon[36687]: Updating smithi104:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T03:38:44.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:43 smithi131 ceph-mon[36687]: Updating smithi131:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T03:38:44.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:43 smithi104 ceph-mon[36569]: Updating smithi040:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T03:38:44.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:43 smithi104 ceph-mon[36569]: Updating smithi104:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T03:38:44.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:43 smithi104 ceph-mon[36569]: Updating smithi131:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T03:38:44.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.636+0000 7f0ae0168700 1 -- 172.21.15.40:0/3210015358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ad8107840 msgr2=0x7f0ad810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:44.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.636+0000 7f0ae0168700 1 --2- 172.21.15.40:0/3210015358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ad8107840 0x7f0ad810f3c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000b0d0 tx=0x7f0ad000b3e0 comp rx=0 tx=0).stop 2024-06-06T03:38:44.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.636+0000 7f0ae0168700 1 -- 172.21.15.40:0/3210015358 shutdown_connections 2024-06-06T03:38:44.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.636+0000 7f0ae0168700 1 --2- 172.21.15.40:0/3210015358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ad8107840 0x7f0ad810f3c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:44.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.636+0000 7f0ae0168700 1 --2- 172.21.15.40:0/3210015358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ad8102740 0x7f0ad8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:44.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.636+0000 7f0ae0168700 1 --2- 172.21.15.40:0/3210015358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad81030f0 0x7f0ad8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:44.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.636+0000 7f0ae0168700 1 -- 172.21.15.40:0/3210015358 >> 172.21.15.40:0/3210015358 conn(0x7f0ad80795f0 msgr2=0x7f0ad8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:44.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.636+0000 7f0ae0168700 1 -- 172.21.15.40:0/3210015358 shutdown_connections 2024-06-06T03:38:44.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.636+0000 7f0ae0168700 1 -- 172.21.15.40:0/3210015358 wait complete. 2024-06-06T03:38:44.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.637+0000 7f0ae0168700 1 Processor -- start 2024-06-06T03:38:44.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.637+0000 7f0ae0168700 1 -- start start 2024-06-06T03:38:44.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.637+0000 7f0ae0168700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ad8102740 0x7f0ad819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:44.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.637+0000 7f0ae0168700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad81030f0 0x7f0ad819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:44.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.637+0000 7f0ae0168700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ad8107840 0x7f0ad819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:44.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.637+0000 7f0ae0168700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad81127d0 con 0x7f0ad8107840 2024-06-06T03:38:44.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.637+0000 7f0ae0168700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad8112650 con 0x7f0ad8102740 2024-06-06T03:38:44.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.637+0000 7f0ae0168700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad8112950 con 0x7f0ad81030f0 2024-06-06T03:38:44.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.637+0000 7f0ade705700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ad8107840 0x7f0ad819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:44.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.637+0000 7f0ade705700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ad8107840 0x7f0ad819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44502/0 (socket says 172.21.15.40:44502) 2024-06-06T03:38:44.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.637+0000 7f0ade705700 1 -- 172.21.15.40:0/3442254618 learned_addr learned my addr 172.21.15.40:0/3442254618 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:44.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.638+0000 7f0addf04700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ad8102740 0x7f0ad819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:44.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.638+0000 7f0add703700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad81030f0 0x7f0ad819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:44.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.638+0000 7f0ade705700 1 -- 172.21.15.40:0/3442254618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad81030f0 msgr2=0x7f0ad819b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:44.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.638+0000 7f0ade705700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad81030f0 0x7f0ad819b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:44.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.638+0000 7f0ade705700 1 -- 172.21.15.40:0/3442254618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ad8102740 msgr2=0x7f0ad819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:44.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.638+0000 7f0ade705700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ad8102740 0x7f0ad819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:44.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.638+0000 7f0ade705700 1 -- 172.21.15.40:0/3442254618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad819fd50 con 0x7f0ad8107840 2024-06-06T03:38:44.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.638+0000 7f0addf04700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ad8102740 0x7f0ad819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:38:44.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.638+0000 7f0ade705700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ad8107840 0x7f0ad819f610 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0006070 tx=0x7f0ad0004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:44.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.638+0000 7f0acaffd700 1 -- 172.21.15.40:0/3442254618 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0021070 con 0x7f0ad8107840 2024-06-06T03:38:44.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.638+0000 7f0acaffd700 1 -- 172.21.15.40:0/3442254618 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad0007c80 con 0x7f0ad8107840 2024-06-06T03:38:44.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.638+0000 7f0ae0168700 1 -- 172.21.15.40:0/3442254618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad819ffe0 con 0x7f0ad8107840 2024-06-06T03:38:44.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.638+0000 7f0acaffd700 1 -- 172.21.15.40:0/3442254618 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad00053c0 con 0x7f0ad8107840 2024-06-06T03:38:44.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.638+0000 7f0ae0168700 1 -- 172.21.15.40:0/3442254618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad81a78c0 con 0x7f0ad8107840 2024-06-06T03:38:44.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.639+0000 7f0ae0168700 1 -- 172.21.15.40:0/3442254618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad8069180 con 0x7f0ad8107840 2024-06-06T03:38:44.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.641+0000 7f0acaffd700 1 -- 172.21.15.40:0/3442254618 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f0ad0024070 con 0x7f0ad8107840 2024-06-06T03:38:44.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.642+0000 7f0acaffd700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f0ac40a2e20 0x7f0ac40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:44.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.642+0000 7f0addf04700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f0ac40a2e20 0x7f0ac40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:44.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.642+0000 7f0acaffd700 1 -- 172.21.15.40:0/3442254618 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f0ad000f020 con 0x7f0ad8107840 2024-06-06T03:38:44.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.642+0000 7f0addf04700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f0ac40a2e20 0x7f0ac40a52e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4005950 tx=0x7f0ad40058e0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:44.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.646+0000 7f0acaffd700 1 -- 172.21.15.40:0/3442254618 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0ad002b020 con 0x7f0ad8107840 2024-06-06T03:38:44.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.848+0000 7f0ae0168700 1 -- 172.21.15.40:0/3442254618 --> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ad8100a60 con 0x7f0ac40a2e20 2024-06-06T03:38:44.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.856+0000 7f0acaffd700 1 -- 172.21.15.40:0/3442254618 <== mgr.24790 v2:172.21.15.40:6800/1568651190 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258082 (secure 0 0 0) 0x7f0ad8100a60 con 0x7f0ac40a2e20 2024-06-06T03:38:44.860 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:38:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.859+0000 7f0ae0168700 1 -- 172.21.15.40:0/3442254618 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f0ac40a2e20 msgr2=0x7f0ac40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.859+0000 7f0ae0168700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f0ac40a2e20 0x7f0ac40a52e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4005950 tx=0x7f0ad40058e0 comp rx=0 tx=0).stop 2024-06-06T03:38:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.859+0000 7f0ae0168700 1 -- 172.21.15.40:0/3442254618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ad8107840 msgr2=0x7f0ad819f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.859+0000 7f0ae0168700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ad8107840 0x7f0ad819f610 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0006070 tx=0x7f0ad0004db0 comp rx=0 tx=0).stop 2024-06-06T03:38:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.859+0000 7f0ae0168700 1 -- 172.21.15.40:0/3442254618 shutdown_connections 2024-06-06T03:38:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.859+0000 7f0ae0168700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f0ac40a2e20 0x7f0ac40a52e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.859+0000 7f0ae0168700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ad8107840 0x7f0ad819f610 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.859+0000 7f0ae0168700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ad8102740 0x7f0ad819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.859+0000 7f0ae0168700 1 --2- 172.21.15.40:0/3442254618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad81030f0 0x7f0ad819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.859+0000 7f0ae0168700 1 -- 172.21.15.40:0/3442254618 >> 172.21.15.40:0/3442254618 conn(0x7f0ad80795f0 msgr2=0x7f0ad8100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.859+0000 7f0ae0168700 1 -- 172.21.15.40:0/3442254618 shutdown_connections 2024-06-06T03:38:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:44.859+0000 7f0ae0168700 1 -- 172.21.15.40:0/3442254618 wait complete. 2024-06-06T03:38:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-06-06T03:38:45.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:44 smithi131 ceph-mon[36687]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 183 KiB/s wr, 106 op/s 2024-06-06T03:38:45.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:44 smithi040 ceph-mon[32220]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 183 KiB/s wr, 106 op/s 2024-06-06T03:38:45.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:44 smithi104 ceph-mon[36569]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 183 KiB/s wr, 106 op/s 2024-06-06T03:38:45.502 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-06-06T03:38:43.646561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389777,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2501,"num_read_kb":1678,"num_write":713,"num_write_kb":5326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1037,"ondisk_log_size":1037,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365796,"kb_used_data":12452,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576028,"statfs":{"total":1151940427776,"available":1151565852672,"internally_reserved":0,"allocated":12750848,"data_stored":9615013,"data_compressed":223818,"data_compressed_allocated":290816,"data_compressed_original":3592192,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":384,"num_read_kb":451,"num_write":148,"num_write_kb":915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.994861"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890341+0000","last_change":"2024-06-06T03:35:13.854938+0000","last_active":"2024-06-06T03:38:37.890341+0000","last_peered":"2024-06-06T03:38:37.890341+0000","last_clean":"2024-06-06T03:38:37.890341+0000","last_became_active":"2024-06-06T03:35:13.853574+0000","last_became_peered":"2024-06-06T03:35:13.853574+0000","last_unstale":"2024-06-06T03:38:37.890341+0000","last_undegraded":"2024-06-06T03:38:37.890341+0000","last_fullsized":"2024-06-06T03:38:37.890341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:21:09.256002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.892012+0000","last_change":"2024-06-06T03:35:13.858299+0000","last_active":"2024-06-06T03:38:37.892012+0000","last_peered":"2024-06-06T03:38:37.892012+0000","last_clean":"2024-06-06T03:38:37.892012+0000","last_became_active":"2024-06-06T03:35:13.857993+0000","last_became_peered":"2024-06-06T03:35:13.857993+0000","last_unstale":"2024-06-06T03:38:37.892012+0000","last_undegraded":"2024-06-06T03:38:37.892012+0000","last_fullsized":"2024-06-06T03:38:37.892012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.811056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891638+0000","last_change":"2024-06-06T03:35:13.852783+0000","last_active":"2024-06-06T03:38:37.891638+0000","last_peered":"2024-06-06T03:38:37.891638+0000","last_clean":"2024-06-06T03:38:37.891638+0000","last_became_active":"2024-06-06T03:35:13.852468+0000","last_became_peered":"2024-06-06T03:35:13.852468+0000","last_unstale":"2024-06-06T03:38:37.891638+0000","last_undegraded":"2024-06-06T03:38:37.891638+0000","last_fullsized":"2024-06-06T03:38:37.891638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:04:59.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.629947+0000","last_change":"2024-06-06T03:35:13.845223+0000","last_active":"2024-06-06T03:38:37.629947+0000","last_peered":"2024-06-06T03:38:37.629947+0000","last_clean":"2024-06-06T03:38:37.629947+0000","last_became_active":"2024-06-06T03:35:13.844854+0000","last_became_peered":"2024-06-06T03:35:13.844854+0000","last_unstale":"2024-06-06T03:38:37.629947+0000","last_undegraded":"2024-06-06T03:38:37.629947+0000","last_fullsized":"2024-06-06T03:38:37.629947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:35:20.058188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.727987+0000","last_change":"2024-06-06T03:35:13.853703+0000","last_active":"2024-06-06T03:38:37.727987+0000","last_peered":"2024-06-06T03:38:37.727987+0000","last_clean":"2024-06-06T03:38:37.727987+0000","last_became_active":"2024-06-06T03:35:13.853358+0000","last_became_peered":"2024-06-06T03:35:13.853358+0000","last_unstale":"2024-06-06T03:38:37.727987+0000","last_undegraded":"2024-06-06T03:38:37.727987+0000","last_fullsized":"2024-06-06T03:38:37.727987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:23:39.092306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736404+0000","last_change":"2024-06-06T03:35:13.859414+0000","last_active":"2024-06-06T03:38:37.736404+0000","last_peered":"2024-06-06T03:38:37.736404+0000","last_clean":"2024-06-06T03:38:37.736404+0000","last_became_active":"2024-06-06T03:35:13.859038+0000","last_became_peered":"2024-06-06T03:35:13.859038+0000","last_unstale":"2024-06-06T03:38:37.736404+0000","last_undegraded":"2024-06-06T03:38:37.736404+0000","last_fullsized":"2024-06-06T03:38:37.736404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:14:31.780064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890276+0000","last_change":"2024-06-06T03:35:13.859245+0000","last_active":"2024-06-06T03:38:37.890276+0000","last_peered":"2024-06-06T03:38:37.890276+0000","last_clean":"2024-06-06T03:38:37.890276+0000","last_became_active":"2024-06-06T03:35:13.858857+0000","last_became_peered":"2024-06-06T03:35:13.858857+0000","last_unstale":"2024-06-06T03:38:37.890276+0000","last_undegraded":"2024-06-06T03:38:37.890276+0000","last_fullsized":"2024-06-06T03:38:37.890276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:03:23.240007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717064+0000","last_change":"2024-06-06T03:35:13.851249+0000","last_active":"2024-06-06T03:38:37.717064+0000","last_peered":"2024-06-06T03:38:37.717064+0000","last_clean":"2024-06-06T03:38:37.717064+0000","last_became_active":"2024-06-06T03:35:13.850777+0000","last_became_peered":"2024-06-06T03:35:13.850777+0000","last_unstale":"2024-06-06T03:38:37.717064+0000","last_undegraded":"2024-06-06T03:38:37.717064+0000","last_fullsized":"2024-06-06T03:38:37.717064+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:53:48.463672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.645289+0000","last_change":"2024-06-06T03:35:13.852950+0000","last_active":"2024-06-06T03:38:37.645289+0000","last_peered":"2024-06-06T03:38:37.645289+0000","last_clean":"2024-06-06T03:38:37.645289+0000","last_became_active":"2024-06-06T03:35:13.852160+0000","last_became_peered":"2024-06-06T03:35:13.852160+0000","last_unstale":"2024-06-06T03:38:37.645289+0000","last_undegraded":"2024-06-06T03:38:37.645289+0000","last_fullsized":"2024-06-06T03:38:37.645289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:00.498440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736760+0000","last_change":"2024-06-06T03:35:13.856081+0000","last_active":"2024-06-06T03:38:37.736760+0000","last_peered":"2024-06-06T03:38:37.736760+0000","last_clean":"2024-06-06T03:38:37.736760+0000","last_became_active":"2024-06-06T03:35:13.855718+0000","last_became_peered":"2024-06-06T03:35:13.855718+0000","last_unstale":"2024-06-06T03:38:37.736760+0000","last_undegraded":"2024-06-06T03:38:37.736760+0000","last_fullsized":"2024-06-06T03:38:37.736760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:33:38.171996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890584+0000","last_change":"2024-06-06T03:35:13.861850+0000","last_active":"2024-06-06T03:38:37.890584+0000","last_peered":"2024-06-06T03:38:37.890584+0000","last_clean":"2024-06-06T03:38:37.890584+0000","last_became_active":"2024-06-06T03:35:13.861513+0000","last_became_peered":"2024-06-06T03:35:13.861513+0000","last_unstale":"2024-06-06T03:38:37.890584+0000","last_undegraded":"2024-06-06T03:38:37.890584+0000","last_fullsized":"2024-06-06T03:38:37.890584+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:32:45.938302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.629839+0000","last_change":"2024-06-06T03:35:13.847742+0000","last_active":"2024-06-06T03:38:37.629839+0000","last_peered":"2024-06-06T03:38:37.629839+0000","last_clean":"2024-06-06T03:38:37.629839+0000","last_became_active":"2024-06-06T03:35:13.847409+0000","last_became_peered":"2024-06-06T03:35:13.847409+0000","last_unstale":"2024-06-06T03:38:37.629839+0000","last_undegraded":"2024-06-06T03:38:37.629839+0000","last_fullsized":"2024-06-06T03:38:37.629839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:20:31.353919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890838+0000","last_change":"2024-06-06T03:35:13.848785+0000","last_active":"2024-06-06T03:38:37.890838+0000","last_peered":"2024-06-06T03:38:37.890838+0000","last_clean":"2024-06-06T03:38:37.890838+0000","last_became_active":"2024-06-06T03:35:13.848048+0000","last_became_peered":"2024-06-06T03:35:13.848048+0000","last_unstale":"2024-06-06T03:38:37.890838+0000","last_undegraded":"2024-06-06T03:38:37.890838+0000","last_fullsized":"2024-06-06T03:38:37.890838+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:43.657761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891499+0000","last_change":"2024-06-06T03:35:13.856083+0000","last_active":"2024-06-06T03:38:37.891499+0000","last_peered":"2024-06-06T03:38:37.891499+0000","last_clean":"2024-06-06T03:38:37.891499+0000","last_became_active":"2024-06-06T03:35:13.854390+0000","last_became_peered":"2024-06-06T03:35:13.854390+0000","last_unstale":"2024-06-06T03:38:37.891499+0000","last_undegraded":"2024-06-06T03:38:37.891499+0000","last_fullsized":"2024-06-06T03:38:37.891499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:51:37.163814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890857+0000","last_change":"2024-06-06T03:35:13.852611+0000","last_active":"2024-06-06T03:38:37.890857+0000","last_peered":"2024-06-06T03:38:37.890857+0000","last_clean":"2024-06-06T03:38:37.890857+0000","last_became_active":"2024-06-06T03:35:13.852244+0000","last_became_peered":"2024-06-06T03:35:13.852244+0000","last_unstale":"2024-06-06T03:38:37.890857+0000","last_undegraded":"2024-06-06T03:38:37.890857+0000","last_fullsized":"2024-06-06T03:38:37.890857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:59:53.513150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.644789+0000","last_change":"2024-06-06T03:35:13.857840+0000","last_active":"2024-06-06T03:38:37.644789+0000","last_peered":"2024-06-06T03:38:37.644789+0000","last_clean":"2024-06-06T03:38:37.644789+0000","last_became_active":"2024-06-06T03:35:13.857137+0000","last_became_peered":"2024-06-06T03:35:13.857137+0000","last_unstale":"2024-06-06T03:38:37.644789+0000","last_undegraded":"2024-06-06T03:38:37.644789+0000","last_fullsized":"2024-06-06T03:38:37.644789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:42:07.924735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890462+0000","last_change":"2024-06-06T03:35:13.857033+0000","last_active":"2024-06-06T03:38:37.890462+0000","last_peered":"2024-06-06T03:38:37.890462+0000","last_clean":"2024-06-06T03:38:37.890462+0000","last_became_active":"2024-06-06T03:35:13.856519+0000","last_became_peered":"2024-06-06T03:35:13.856519+0000","last_unstale":"2024-06-06T03:38:37.890462+0000","last_undegraded":"2024-06-06T03:38:37.890462+0000","last_fullsized":"2024-06-06T03:38:37.890462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:05:25.668344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630726+0000","last_change":"2024-06-06T03:35:13.851356+0000","last_active":"2024-06-06T03:38:37.630726+0000","last_peered":"2024-06-06T03:38:37.630726+0000","last_clean":"2024-06-06T03:38:37.630726+0000","last_became_active":"2024-06-06T03:35:13.851030+0000","last_became_peered":"2024-06-06T03:35:13.851030+0000","last_unstale":"2024-06-06T03:38:37.630726+0000","last_undegraded":"2024-06-06T03:38:37.630726+0000","last_fullsized":"2024-06-06T03:38:37.630726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:49:38.833022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632287+0000","last_change":"2024-06-06T03:35:13.858613+0000","last_active":"2024-06-06T03:38:37.632287+0000","last_peered":"2024-06-06T03:38:37.632287+0000","last_clean":"2024-06-06T03:38:37.632287+0000","last_became_active":"2024-06-06T03:35:13.858182+0000","last_became_peered":"2024-06-06T03:35:13.858182+0000","last_unstale":"2024-06-06T03:38:37.632287+0000","last_undegraded":"2024-06-06T03:38:37.632287+0000","last_fullsized":"2024-06-06T03:38:37.632287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:23:08.216431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'15","reported_seq":234,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:42.244022+0000","last_change":"2024-06-06T03:35:09.821896+0000","last_active":"2024-06-06T03:38:42.244022+0000","last_peered":"2024-06-06T03:38:42.244022+0000","last_clean":"2024-06-06T03:38:42.244022+0000","last_became_active":"2024-06-06T03:35:09.821567+0000","last_became_peered":"2024-06-06T03:35:09.821567+0000","last_unstale":"2024-06-06T03:38:42.244022+0000","last_undegraded":"2024-06-06T03:38:42.244022+0000","last_fullsized":"2024-06-06T03:38:42.244022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:04:51.918393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717587+0000","last_change":"2024-06-06T03:35:13.857357+0000","last_active":"2024-06-06T03:38:37.717587+0000","last_peered":"2024-06-06T03:38:37.717587+0000","last_clean":"2024-06-06T03:38:37.717587+0000","last_became_active":"2024-06-06T03:35:13.857016+0000","last_became_peered":"2024-06-06T03:35:13.857016+0000","last_unstale":"2024-06-06T03:38:37.717587+0000","last_undegraded":"2024-06-06T03:38:37.717587+0000","last_fullsized":"2024-06-06T03:38:37.717587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:42.566425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717796+0000","last_change":"2024-06-06T03:35:10.277316+0000","last_active":"2024-06-06T03:38:37.717796+0000","last_peered":"2024-06-06T03:38:37.717796+0000","last_clean":"2024-06-06T03:38:37.717796+0000","last_became_active":"2024-06-06T03:35:10.276986+0000","last_became_peered":"2024-06-06T03:35:10.276986+0000","last_unstale":"2024-06-06T03:38:37.717796+0000","last_undegraded":"2024-06-06T03:38:37.717796+0000","last_fullsized":"2024-06-06T03:38:37.717796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:17:29.034224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.892351+0000","last_change":"2024-06-06T03:35:13.863069+0000","last_active":"2024-06-06T03:38:37.892351+0000","last_peered":"2024-06-06T03:38:37.892351+0000","last_clean":"2024-06-06T03:38:37.892351+0000","last_became_active":"2024-06-06T03:35:13.862647+0000","last_became_peered":"2024-06-06T03:35:13.862647+0000","last_unstale":"2024-06-06T03:38:37.892351+0000","last_undegraded":"2024-06-06T03:38:37.892351+0000","last_fullsized":"2024-06-06T03:38:37.892351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:48.252606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'13","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728456+0000","last_change":"2024-06-06T03:35:09.824977+0000","last_active":"2024-06-06T03:38:37.728456+0000","last_peered":"2024-06-06T03:38:37.728456+0000","last_clean":"2024-06-06T03:38:37.728456+0000","last_became_active":"2024-06-06T03:35:09.824643+0000","last_became_peered":"2024-06-06T03:35:09.824643+0000","last_unstale":"2024-06-06T03:38:37.728456+0000","last_undegraded":"2024-06-06T03:38:37.728456+0000","last_fullsized":"2024-06-06T03:38:37.728456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:35:37.208023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.644898+0000","last_change":"2024-06-06T03:35:13.860572+0000","last_active":"2024-06-06T03:38:37.644898+0000","last_peered":"2024-06-06T03:38:37.644898+0000","last_clean":"2024-06-06T03:38:37.644898+0000","last_became_active":"2024-06-06T03:35:13.858398+0000","last_became_peered":"2024-06-06T03:35:13.858398+0000","last_unstale":"2024-06-06T03:38:37.644898+0000","last_undegraded":"2024-06-06T03:38:37.644898+0000","last_fullsized":"2024-06-06T03:38:37.644898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:02:12.683096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728187+0000","last_change":"2024-06-06T03:35:09.827016+0000","last_active":"2024-06-06T03:38:37.728187+0000","last_peered":"2024-06-06T03:38:37.728187+0000","last_clean":"2024-06-06T03:38:37.728187+0000","last_became_active":"2024-06-06T03:35:09.826576+0000","last_became_peered":"2024-06-06T03:35:09.826576+0000","last_unstale":"2024-06-06T03:38:37.728187+0000","last_undegraded":"2024-06-06T03:38:37.728187+0000","last_fullsized":"2024-06-06T03:38:37.728187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:24.120549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733815+0000","last_change":"2024-06-06T03:35:13.853988+0000","last_active":"2024-06-06T03:38:37.733815+0000","last_peered":"2024-06-06T03:38:37.733815+0000","last_clean":"2024-06-06T03:38:37.733815+0000","last_became_active":"2024-06-06T03:35:13.852343+0000","last_became_peered":"2024-06-06T03:35:13.852343+0000","last_unstale":"2024-06-06T03:38:37.733815+0000","last_undegraded":"2024-06-06T03:38:37.733815+0000","last_fullsized":"2024-06-06T03:38:37.733815+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:39:03.054045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":231,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:42.211538+0000","last_change":"2024-06-06T03:35:09.829109+0000","last_active":"2024-06-06T03:38:42.211538+0000","last_peered":"2024-06-06T03:38:42.211538+0000","last_clean":"2024-06-06T03:38:42.211538+0000","last_became_active":"2024-06-06T03:35:09.828734+0000","last_became_peered":"2024-06-06T03:35:09.828734+0000","last_unstale":"2024-06-06T03:38:42.211538+0000","last_undegraded":"2024-06-06T03:38:42.211538+0000","last_fullsized":"2024-06-06T03:38:42.211538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:50.475600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":194,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.727932+0000","last_change":"2024-06-06T03:35:13.859198+0000","last_active":"2024-06-06T03:38:37.727932+0000","last_peered":"2024-06-06T03:38:37.727932+0000","last_clean":"2024-06-06T03:38:37.727932+0000","last_became_active":"2024-06-06T03:35:13.858863+0000","last_became_peered":"2024-06-06T03:35:13.858863+0000","last_unstale":"2024-06-06T03:38:37.727932+0000","last_undegraded":"2024-06-06T03:38:37.727932+0000","last_fullsized":"2024-06-06T03:38:37.727932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:57.153174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736340+0000","last_change":"2024-06-06T03:35:09.824352+0000","last_active":"2024-06-06T03:38:37.736340+0000","last_peered":"2024-06-06T03:38:37.736340+0000","last_clean":"2024-06-06T03:38:37.736340+0000","last_became_active":"2024-06-06T03:35:09.823998+0000","last_became_peered":"2024-06-06T03:35:09.823998+0000","last_unstale":"2024-06-06T03:38:37.736340+0000","last_undegraded":"2024-06-06T03:38:37.736340+0000","last_fullsized":"2024-06-06T03:38:37.736340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:47:24.646067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733489+0000","last_change":"2024-06-06T03:35:13.836724+0000","last_active":"2024-06-06T03:38:37.733489+0000","last_peered":"2024-06-06T03:38:37.733489+0000","last_clean":"2024-06-06T03:38:37.733489+0000","last_became_active":"2024-06-06T03:35:13.836180+0000","last_became_peered":"2024-06-06T03:35:13.836180+0000","last_unstale":"2024-06-06T03:38:37.733489+0000","last_undegraded":"2024-06-06T03:38:37.733489+0000","last_fullsized":"2024-06-06T03:38:37.733489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:08.868462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890183+0000","last_change":"2024-06-06T03:35:09.823047+0000","last_active":"2024-06-06T03:38:37.890183+0000","last_peered":"2024-06-06T03:38:37.890183+0000","last_clean":"2024-06-06T03:38:37.890183+0000","last_became_active":"2024-06-06T03:35:09.822744+0000","last_became_peered":"2024-06-06T03:35:09.822744+0000","last_unstale":"2024-06-06T03:38:37.890183+0000","last_undegraded":"2024-06-06T03:38:37.890183+0000","last_fullsized":"2024-06-06T03:38:37.890183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:20:20.165682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.716965+0000","last_change":"2024-06-06T03:35:13.857222+0000","last_active":"2024-06-06T03:38:37.716965+0000","last_peered":"2024-06-06T03:38:37.716965+0000","last_clean":"2024-06-06T03:38:37.716965+0000","last_became_active":"2024-06-06T03:35:13.856873+0000","last_became_peered":"2024-06-06T03:35:13.856873+0000","last_unstale":"2024-06-06T03:38:37.716965+0000","last_undegraded":"2024-06-06T03:38:37.716965+0000","last_fullsized":"2024-06-06T03:38:37.716965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:32:16.646275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733256+0000","last_change":"2024-06-06T03:35:09.826036+0000","last_active":"2024-06-06T03:38:37.733256+0000","last_peered":"2024-06-06T03:38:37.733256+0000","last_clean":"2024-06-06T03:38:37.733256+0000","last_became_active":"2024-06-06T03:35:09.825690+0000","last_became_peered":"2024-06-06T03:35:09.825690+0000","last_unstale":"2024-06-06T03:38:37.733256+0000","last_undegraded":"2024-06-06T03:38:37.733256+0000","last_fullsized":"2024-06-06T03:38:37.733256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:03:35.858828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630424+0000","last_change":"2024-06-06T03:35:13.849016+0000","last_active":"2024-06-06T03:38:37.630424+0000","last_peered":"2024-06-06T03:38:37.630424+0000","last_clean":"2024-06-06T03:38:37.630424+0000","last_became_active":"2024-06-06T03:35:13.848709+0000","last_became_peered":"2024-06-06T03:35:13.848709+0000","last_unstale":"2024-06-06T03:38:37.630424+0000","last_undegraded":"2024-06-06T03:38:37.630424+0000","last_fullsized":"2024-06-06T03:38:37.630424+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:42.778612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":404,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:38.205096+0000","last_change":"2024-06-06T03:35:09.827072+0000","last_active":"2024-06-06T03:38:38.205096+0000","last_peered":"2024-06-06T03:38:38.205096+0000","last_clean":"2024-06-06T03:38:38.205096+0000","last_became_active":"2024-06-06T03:35:09.826734+0000","last_became_peered":"2024-06-06T03:35:09.826734+0000","last_unstale":"2024-06-06T03:38:38.205096+0000","last_undegraded":"2024-06-06T03:38:38.205096+0000","last_fullsized":"2024-06-06T03:38:38.205096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:07:13.388398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":368,"num_read_kb":368,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.645230+0000","last_change":"2024-06-06T03:35:13.856883+0000","last_active":"2024-06-06T03:38:37.645230+0000","last_peered":"2024-06-06T03:38:37.645230+0000","last_clean":"2024-06-06T03:38:37.645230+0000","last_became_active":"2024-06-06T03:35:13.856521+0000","last_became_peered":"2024-06-06T03:35:13.856521+0000","last_unstale":"2024-06-06T03:38:37.645230+0000","last_undegraded":"2024-06-06T03:38:37.645230+0000","last_fullsized":"2024-06-06T03:38:37.645230+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:19.466218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890304+0000","last_change":"2024-06-06T03:35:09.829153+0000","last_active":"2024-06-06T03:38:37.890304+0000","last_peered":"2024-06-06T03:38:37.890304+0000","last_clean":"2024-06-06T03:38:37.890304+0000","last_became_active":"2024-06-06T03:35:09.828797+0000","last_became_peered":"2024-06-06T03:35:09.828797+0000","last_unstale":"2024-06-06T03:38:37.890304+0000","last_undegraded":"2024-06-06T03:38:37.890304+0000","last_fullsized":"2024-06-06T03:38:37.890304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:06:02.986396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.892165+0000","last_change":"2024-06-06T03:35:13.857094+0000","last_active":"2024-06-06T03:38:37.892165+0000","last_peered":"2024-06-06T03:38:37.892165+0000","last_clean":"2024-06-06T03:38:37.892165+0000","last_became_active":"2024-06-06T03:35:13.856778+0000","last_became_peered":"2024-06-06T03:35:13.856778+0000","last_unstale":"2024-06-06T03:38:37.892165+0000","last_undegraded":"2024-06-06T03:38:37.892165+0000","last_fullsized":"2024-06-06T03:38:37.892165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:50:30.097762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733828+0000","last_change":"2024-06-06T03:35:10.277802+0000","last_active":"2024-06-06T03:38:37.733828+0000","last_peered":"2024-06-06T03:38:37.733828+0000","last_clean":"2024-06-06T03:38:37.733828+0000","last_became_active":"2024-06-06T03:35:10.277417+0000","last_became_peered":"2024-06-06T03:35:10.277417+0000","last_unstale":"2024-06-06T03:38:37.733828+0000","last_undegraded":"2024-06-06T03:38:37.733828+0000","last_fullsized":"2024-06-06T03:38:37.733828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:44:35.178184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.645024+0000","last_change":"2024-06-06T03:35:13.858160+0000","last_active":"2024-06-06T03:38:37.645024+0000","last_peered":"2024-06-06T03:38:37.645024+0000","last_clean":"2024-06-06T03:38:37.645024+0000","last_became_active":"2024-06-06T03:35:13.853149+0000","last_became_peered":"2024-06-06T03:35:13.853149+0000","last_unstale":"2024-06-06T03:38:37.645024+0000","last_undegraded":"2024-06-06T03:38:37.645024+0000","last_fullsized":"2024-06-06T03:38:37.645024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:10:06.697274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":447,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:38.205556+0000","last_change":"2024-06-06T03:35:09.829443+0000","last_active":"2024-06-06T03:38:38.205556+0000","last_peered":"2024-06-06T03:38:38.205556+0000","last_clean":"2024-06-06T03:38:38.205556+0000","last_became_active":"2024-06-06T03:35:09.828093+0000","last_became_peered":"2024-06-06T03:35:09.828093+0000","last_unstale":"2024-06-06T03:38:38.205556+0000","last_undegraded":"2024-06-06T03:38:38.205556+0000","last_fullsized":"2024-06-06T03:38:38.205556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:05:19.395177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67985,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":380,"num_read_kb":256,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728167+0000","last_change":"2024-06-06T03:35:13.850869+0000","last_active":"2024-06-06T03:38:37.728167+0000","last_peered":"2024-06-06T03:38:37.728167+0000","last_clean":"2024-06-06T03:38:37.728167+0000","last_became_active":"2024-06-06T03:35:13.849489+0000","last_became_peered":"2024-06-06T03:35:13.849489+0000","last_unstale":"2024-06-06T03:38:37.728167+0000","last_undegraded":"2024-06-06T03:38:37.728167+0000","last_fullsized":"2024-06-06T03:38:37.728167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:27:45.052033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632136+0000","last_change":"2024-06-06T03:35:09.823117+0000","last_active":"2024-06-06T03:38:37.632136+0000","last_peered":"2024-06-06T03:38:37.632136+0000","last_clean":"2024-06-06T03:38:37.632136+0000","last_became_active":"2024-06-06T03:35:09.822624+0000","last_became_peered":"2024-06-06T03:35:09.822624+0000","last_unstale":"2024-06-06T03:38:37.632136+0000","last_undegraded":"2024-06-06T03:38:37.632136+0000","last_fullsized":"2024-06-06T03:38:37.632136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:59.171343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890529+0000","last_change":"2024-06-06T03:35:13.861043+0000","last_active":"2024-06-06T03:38:37.890529+0000","last_peered":"2024-06-06T03:38:37.890529+0000","last_clean":"2024-06-06T03:38:37.890529+0000","last_became_active":"2024-06-06T03:35:13.860718+0000","last_became_peered":"2024-06-06T03:35:13.860718+0000","last_unstale":"2024-06-06T03:38:37.890529+0000","last_undegraded":"2024-06-06T03:38:37.890529+0000","last_fullsized":"2024-06-06T03:38:37.890529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:34:04.595780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733532+0000","last_change":"2024-06-06T03:35:09.825505+0000","last_active":"2024-06-06T03:38:37.733532+0000","last_peered":"2024-06-06T03:38:37.733532+0000","last_clean":"2024-06-06T03:38:37.733532+0000","last_became_active":"2024-06-06T03:35:09.825127+0000","last_became_peered":"2024-06-06T03:35:09.825127+0000","last_unstale":"2024-06-06T03:38:37.733532+0000","last_undegraded":"2024-06-06T03:38:37.733532+0000","last_fullsized":"2024-06-06T03:38:37.733532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:07:27.664589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890667+0000","last_change":"2024-06-06T03:35:48.987834+0000","last_active":"2024-06-06T03:38:37.890667+0000","last_peered":"2024-06-06T03:38:37.890667+0000","last_clean":"2024-06-06T03:38:37.890667+0000","last_became_active":"2024-06-06T03:35:48.987515+0000","last_became_peered":"2024-06-06T03:35:48.987515+0000","last_unstale":"2024-06-06T03:38:37.890667+0000","last_undegraded":"2024-06-06T03:38:37.890667+0000","last_fullsized":"2024-06-06T03:38:37.890667+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:05:19.342289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":228,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:42.211332+0000","last_change":"2024-06-06T03:35:09.826192+0000","last_active":"2024-06-06T03:38:42.211332+0000","last_peered":"2024-06-06T03:38:42.211332+0000","last_clean":"2024-06-06T03:38:42.211332+0000","last_became_active":"2024-06-06T03:35:09.825481+0000","last_became_peered":"2024-06-06T03:35:09.825481+0000","last_unstale":"2024-06-06T03:38:42.211332+0000","last_undegraded":"2024-06-06T03:38:42.211332+0000","last_fullsized":"2024-06-06T03:38:42.211332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:18:30.772991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1368,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890452+0000","last_change":"2024-06-06T03:35:13.853113+0000","last_active":"2024-06-06T03:38:37.890452+0000","last_peered":"2024-06-06T03:38:37.890452+0000","last_clean":"2024-06-06T03:38:37.890452+0000","last_became_active":"2024-06-06T03:35:13.852751+0000","last_became_peered":"2024-06-06T03:35:13.852751+0000","last_unstale":"2024-06-06T03:38:37.890452+0000","last_undegraded":"2024-06-06T03:38:37.890452+0000","last_fullsized":"2024-06-06T03:38:37.890452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:53.536685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890032+0000","last_change":"2024-06-06T03:35:49.005958+0000","last_active":"2024-06-06T03:38:37.890032+0000","last_peered":"2024-06-06T03:38:37.890032+0000","last_clean":"2024-06-06T03:38:37.890032+0000","last_became_active":"2024-06-06T03:35:48.993266+0000","last_became_peered":"2024-06-06T03:35:48.993266+0000","last_unstale":"2024-06-06T03:38:37.890032+0000","last_undegraded":"2024-06-06T03:38:37.890032+0000","last_fullsized":"2024-06-06T03:38:37.890032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:47.782824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733337+0000","last_change":"2024-06-06T03:35:13.853614+0000","last_active":"2024-06-06T03:38:37.733337+0000","last_peered":"2024-06-06T03:38:37.733337+0000","last_clean":"2024-06-06T03:38:37.733337+0000","last_became_active":"2024-06-06T03:35:13.852327+0000","last_became_peered":"2024-06-06T03:35:13.852327+0000","last_unstale":"2024-06-06T03:38:37.733337+0000","last_undegraded":"2024-06-06T03:38:37.733337+0000","last_fullsized":"2024-06-06T03:38:37.733337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:02:32.463028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728378+0000","last_change":"2024-06-06T03:35:09.826220+0000","last_active":"2024-06-06T03:38:37.728378+0000","last_peered":"2024-06-06T03:38:37.728378+0000","last_clean":"2024-06-06T03:38:37.728378+0000","last_became_active":"2024-06-06T03:35:09.825887+0000","last_became_peered":"2024-06-06T03:35:09.825887+0000","last_unstale":"2024-06-06T03:38:37.728378+0000","last_undegraded":"2024-06-06T03:38:37.728378+0000","last_fullsized":"2024-06-06T03:38:37.728378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:23:33.704822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890232+0000","last_change":"2024-06-06T03:35:13.852455+0000","last_active":"2024-06-06T03:38:37.890232+0000","last_peered":"2024-06-06T03:38:37.890232+0000","last_clean":"2024-06-06T03:38:37.890232+0000","last_became_active":"2024-06-06T03:35:13.852134+0000","last_became_peered":"2024-06-06T03:35:13.852134+0000","last_unstale":"2024-06-06T03:38:37.890232+0000","last_undegraded":"2024-06-06T03:38:37.890232+0000","last_fullsized":"2024-06-06T03:38:37.890232+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:00:16.418479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630691+0000","last_change":"2024-06-06T03:35:10.277767+0000","last_active":"2024-06-06T03:38:37.630691+0000","last_peered":"2024-06-06T03:38:37.630691+0000","last_clean":"2024-06-06T03:38:37.630691+0000","last_became_active":"2024-06-06T03:35:10.277381+0000","last_became_peered":"2024-06-06T03:35:10.277381+0000","last_unstale":"2024-06-06T03:38:37.630691+0000","last_undegraded":"2024-06-06T03:38:37.630691+0000","last_fullsized":"2024-06-06T03:38:37.630691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:12:13.425284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736536+0000","last_change":"2024-06-06T03:35:13.859594+0000","last_active":"2024-06-06T03:38:37.736536+0000","last_peered":"2024-06-06T03:38:37.736536+0000","last_clean":"2024-06-06T03:38:37.736536+0000","last_became_active":"2024-06-06T03:35:13.859240+0000","last_became_peered":"2024-06-06T03:35:13.859240+0000","last_unstale":"2024-06-06T03:38:37.736536+0000","last_undegraded":"2024-06-06T03:38:37.736536+0000","last_fullsized":"2024-06-06T03:38:37.736536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:39:34.557581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'8","reported_seq":230,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:42.244501+0000","last_change":"2024-06-06T03:35:09.826204+0000","last_active":"2024-06-06T03:38:42.244501+0000","last_peered":"2024-06-06T03:38:42.244501+0000","last_clean":"2024-06-06T03:38:42.244501+0000","last_became_active":"2024-06-06T03:35:09.825815+0000","last_became_peered":"2024-06-06T03:35:09.825815+0000","last_unstale":"2024-06-06T03:38:42.244501+0000","last_undegraded":"2024-06-06T03:38:42.244501+0000","last_fullsized":"2024-06-06T03:38:42.244501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:02:04.169227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1427,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736428+0000","last_change":"2024-06-06T03:35:48.992786+0000","last_active":"2024-06-06T03:38:37.736428+0000","last_peered":"2024-06-06T03:38:37.736428+0000","last_clean":"2024-06-06T03:38:37.736428+0000","last_became_active":"2024-06-06T03:35:48.992456+0000","last_became_peered":"2024-06-06T03:35:48.992456+0000","last_unstale":"2024-06-06T03:38:37.736428+0000","last_undegraded":"2024-06-06T03:38:37.736428+0000","last_fullsized":"2024-06-06T03:38:37.736428+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:06:58.563975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630421+0000","last_change":"2024-06-06T03:35:49.005119+0000","last_active":"2024-06-06T03:38:37.630421+0000","last_peered":"2024-06-06T03:38:37.630421+0000","last_clean":"2024-06-06T03:38:37.630421+0000","last_became_active":"2024-06-06T03:35:49.004788+0000","last_became_peered":"2024-06-06T03:35:49.004788+0000","last_unstale":"2024-06-06T03:38:37.630421+0000","last_undegraded":"2024-06-06T03:38:37.630421+0000","last_fullsized":"2024-06-06T03:38:37.630421+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:20:06.961454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":225,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:42.210803+0000","last_change":"2024-06-06T03:35:10.278559+0000","last_active":"2024-06-06T03:38:42.210803+0000","last_peered":"2024-06-06T03:38:42.210803+0000","last_clean":"2024-06-06T03:38:42.210803+0000","last_became_active":"2024-06-06T03:35:10.278104+0000","last_became_peered":"2024-06-06T03:35:10.278104+0000","last_unstale":"2024-06-06T03:38:42.210803+0000","last_undegraded":"2024-06-06T03:38:42.210803+0000","last_fullsized":"2024-06-06T03:38:42.210803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:23:47.131731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717376+0000","last_change":"2024-06-06T03:35:13.862303+0000","last_active":"2024-06-06T03:38:37.717376+0000","last_peered":"2024-06-06T03:38:37.717376+0000","last_clean":"2024-06-06T03:38:37.717376+0000","last_became_active":"2024-06-06T03:35:13.861978+0000","last_became_peered":"2024-06-06T03:35:13.861978+0000","last_unstale":"2024-06-06T03:38:37.717376+0000","last_undegraded":"2024-06-06T03:38:37.717376+0000","last_fullsized":"2024-06-06T03:38:37.717376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:24:00.824646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.629723+0000","last_change":"2024-06-06T03:35:09.825365+0000","last_active":"2024-06-06T03:38:37.629723+0000","last_peered":"2024-06-06T03:38:37.629723+0000","last_clean":"2024-06-06T03:38:37.629723+0000","last_became_active":"2024-06-06T03:35:09.824948+0000","last_became_peered":"2024-06-06T03:35:09.824948+0000","last_unstale":"2024-06-06T03:38:37.629723+0000","last_undegraded":"2024-06-06T03:38:37.629723+0000","last_fullsized":"2024-06-06T03:38:37.629723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:23:48.189292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'148","reported_seq":406,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:39.739025+0000","last_change":"2024-06-06T03:33:44.124626+0000","last_active":"2024-06-06T03:38:39.739025+0000","last_peered":"2024-06-06T03:38:39.739025+0000","last_clean":"2024-06-06T03:38:39.739025+0000","last_became_active":"2024-06-06T03:33:44.124046+0000","last_became_peered":"2024-06-06T03:33:44.124046+0000","last_unstale":"2024-06-06T03:38:39.739025+0000","last_undegraded":"2024-06-06T03:38:39.739025+0000","last_fullsized":"2024-06-06T03:38:39.739025+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:30:14.586719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:30:14.586719+0000","last_clean_scrub_stamp":"2024-06-06T03:30:14.586719+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:17.604180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890344+0000","last_change":"2024-06-06T03:35:13.852044+0000","last_active":"2024-06-06T03:38:37.890344+0000","last_peered":"2024-06-06T03:38:37.890344+0000","last_clean":"2024-06-06T03:38:37.890344+0000","last_became_active":"2024-06-06T03:35:13.851715+0000","last_became_peered":"2024-06-06T03:35:13.851715+0000","last_unstale":"2024-06-06T03:38:37.890344+0000","last_undegraded":"2024-06-06T03:38:37.890344+0000","last_fullsized":"2024-06-06T03:38:37.890344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:19:23.778367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890093+0000","last_change":"2024-06-06T03:35:09.828463+0000","last_active":"2024-06-06T03:38:37.890093+0000","last_peered":"2024-06-06T03:38:37.890093+0000","last_clean":"2024-06-06T03:38:37.890093+0000","last_became_active":"2024-06-06T03:35:09.828121+0000","last_became_peered":"2024-06-06T03:35:09.828121+0000","last_unstale":"2024-06-06T03:38:37.890093+0000","last_undegraded":"2024-06-06T03:38:37.890093+0000","last_fullsized":"2024-06-06T03:38:37.890093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:54:25.095741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728370+0000","last_change":"2024-06-06T03:35:10.277901+0000","last_active":"2024-06-06T03:38:37.728370+0000","last_peered":"2024-06-06T03:38:37.728370+0000","last_clean":"2024-06-06T03:38:37.728370+0000","last_became_active":"2024-06-06T03:35:10.277447+0000","last_became_peered":"2024-06-06T03:35:10.277447+0000","last_unstale":"2024-06-06T03:38:37.728370+0000","last_undegraded":"2024-06-06T03:38:37.728370+0000","last_fullsized":"2024-06-06T03:38:37.728370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:22:50.550287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891033+0000","last_change":"2024-06-06T03:35:13.864323+0000","last_active":"2024-06-06T03:38:37.891033+0000","last_peered":"2024-06-06T03:38:37.891033+0000","last_clean":"2024-06-06T03:38:37.891033+0000","last_became_active":"2024-06-06T03:35:13.863101+0000","last_became_peered":"2024-06-06T03:35:13.863101+0000","last_unstale":"2024-06-06T03:38:37.891033+0000","last_undegraded":"2024-06-06T03:38:37.891033+0000","last_fullsized":"2024-06-06T03:38:37.891033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:32.399325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'105","reported_seq":226,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:39.879130+0000","last_change":"2024-06-06T03:35:09.822280+0000","last_active":"2024-06-06T03:38:39.879130+0000","last_peered":"2024-06-06T03:38:39.879130+0000","last_clean":"2024-06-06T03:38:39.879130+0000","last_became_active":"2024-06-06T03:35:09.821922+0000","last_became_peered":"2024-06-06T03:35:09.821922+0000","last_unstale":"2024-06-06T03:38:39.879130+0000","last_undegraded":"2024-06-06T03:38:39.879130+0000","last_fullsized":"2024-06-06T03:38:39.879130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:00:03.328747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":192,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717835+0000","last_change":"2024-06-06T03:35:13.853538+0000","last_active":"2024-06-06T03:38:37.717835+0000","last_peered":"2024-06-06T03:38:37.717835+0000","last_clean":"2024-06-06T03:38:37.717835+0000","last_became_active":"2024-06-06T03:35:13.853200+0000","last_became_peered":"2024-06-06T03:35:13.853200+0000","last_unstale":"2024-06-06T03:38:37.717835+0000","last_undegraded":"2024-06-06T03:38:37.717835+0000","last_fullsized":"2024-06-06T03:38:37.717835+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:13:06.574272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630204+0000","last_change":"2024-06-06T03:35:10.761675+0000","last_active":"2024-06-06T03:38:37.630204+0000","last_peered":"2024-06-06T03:38:37.630204+0000","last_clean":"2024-06-06T03:38:37.630204+0000","last_became_active":"2024-06-06T03:35:10.761287+0000","last_became_peered":"2024-06-06T03:35:10.761287+0000","last_unstale":"2024-06-06T03:38:37.630204+0000","last_undegraded":"2024-06-06T03:38:37.630204+0000","last_fullsized":"2024-06-06T03:38:37.630204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:58:18.524273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.892143+0000","last_change":"2024-06-06T03:35:13.847680+0000","last_active":"2024-06-06T03:38:37.892143+0000","last_peered":"2024-06-06T03:38:37.892143+0000","last_clean":"2024-06-06T03:38:37.892143+0000","last_became_active":"2024-06-06T03:35:13.847359+0000","last_became_peered":"2024-06-06T03:35:13.847359+0000","last_unstale":"2024-06-06T03:38:37.892143+0000","last_undegraded":"2024-06-06T03:38:37.892143+0000","last_fullsized":"2024-06-06T03:38:37.892143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:29:18.041078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.644794+0000","last_change":"2024-06-06T03:35:10.277491+0000","last_active":"2024-06-06T03:38:37.644794+0000","last_peered":"2024-06-06T03:38:37.644794+0000","last_clean":"2024-06-06T03:38:37.644794+0000","last_became_active":"2024-06-06T03:35:10.277135+0000","last_became_peered":"2024-06-06T03:35:10.277135+0000","last_unstale":"2024-06-06T03:38:37.644794+0000","last_undegraded":"2024-06-06T03:38:37.644794+0000","last_fullsized":"2024-06-06T03:38:37.644794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:55.198789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.727917+0000","last_change":"2024-06-06T03:35:13.846013+0000","last_active":"2024-06-06T03:38:37.727917+0000","last_peered":"2024-06-06T03:38:37.727917+0000","last_clean":"2024-06-06T03:38:37.727917+0000","last_became_active":"2024-06-06T03:35:13.845650+0000","last_became_peered":"2024-06-06T03:35:13.845650+0000","last_unstale":"2024-06-06T03:38:37.727917+0000","last_undegraded":"2024-06-06T03:38:37.727917+0000","last_fullsized":"2024-06-06T03:38:37.727917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:51.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.716995+0000","last_change":"2024-06-06T03:35:09.826380+0000","last_active":"2024-06-06T03:38:37.716995+0000","last_peered":"2024-06-06T03:38:37.716995+0000","last_clean":"2024-06-06T03:38:37.716995+0000","last_became_active":"2024-06-06T03:35:09.825813+0000","last_became_peered":"2024-06-06T03:35:09.825813+0000","last_unstale":"2024-06-06T03:38:37.716995+0000","last_undegraded":"2024-06-06T03:38:37.716995+0000","last_fullsized":"2024-06-06T03:38:37.716995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:24.792398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890127+0000","last_change":"2024-06-06T03:35:13.855051+0000","last_active":"2024-06-06T03:38:37.890127+0000","last_peered":"2024-06-06T03:38:37.890127+0000","last_clean":"2024-06-06T03:38:37.890127+0000","last_became_active":"2024-06-06T03:35:13.853781+0000","last_became_peered":"2024-06-06T03:35:13.853781+0000","last_unstale":"2024-06-06T03:38:37.890127+0000","last_undegraded":"2024-06-06T03:38:37.890127+0000","last_fullsized":"2024-06-06T03:38:37.890127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:17:56.174924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890090+0000","last_change":"2024-06-06T03:35:49.009583+0000","last_active":"2024-06-06T03:38:37.890090+0000","last_peered":"2024-06-06T03:38:37.890090+0000","last_clean":"2024-06-06T03:38:37.890090+0000","last_became_active":"2024-06-06T03:35:48.994275+0000","last_became_peered":"2024-06-06T03:35:48.994275+0000","last_unstale":"2024-06-06T03:38:37.890090+0000","last_undegraded":"2024-06-06T03:38:37.890090+0000","last_fullsized":"2024-06-06T03:38:37.890090+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:24:54.985854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717547+0000","last_change":"2024-06-06T03:35:13.855503+0000","last_active":"2024-06-06T03:38:37.717547+0000","last_peered":"2024-06-06T03:38:37.717547+0000","last_clean":"2024-06-06T03:38:37.717547+0000","last_became_active":"2024-06-06T03:35:13.855069+0000","last_became_peered":"2024-06-06T03:35:13.855069+0000","last_unstale":"2024-06-06T03:38:37.717547+0000","last_undegraded":"2024-06-06T03:38:37.717547+0000","last_fullsized":"2024-06-06T03:38:37.717547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:43.778258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.727997+0000","last_change":"2024-06-06T03:35:09.827116+0000","last_active":"2024-06-06T03:38:37.727997+0000","last_peered":"2024-06-06T03:38:37.727997+0000","last_clean":"2024-06-06T03:38:37.727997+0000","last_became_active":"2024-06-06T03:35:09.826756+0000","last_became_peered":"2024-06-06T03:35:09.826756+0000","last_unstale":"2024-06-06T03:38:37.727997+0000","last_undegraded":"2024-06-06T03:38:37.727997+0000","last_fullsized":"2024-06-06T03:38:37.727997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:31.501411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890818+0000","last_change":"2024-06-06T03:35:13.861189+0000","last_active":"2024-06-06T03:38:37.890818+0000","last_peered":"2024-06-06T03:38:37.890818+0000","last_clean":"2024-06-06T03:38:37.890818+0000","last_became_active":"2024-06-06T03:35:13.860751+0000","last_became_peered":"2024-06-06T03:35:13.860751+0000","last_unstale":"2024-06-06T03:38:37.890818+0000","last_undegraded":"2024-06-06T03:38:37.890818+0000","last_fullsized":"2024-06-06T03:38:37.890818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:16:34.036420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":228,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:42.244899+0000","last_change":"2024-06-06T03:35:10.761606+0000","last_active":"2024-06-06T03:38:42.244899+0000","last_peered":"2024-06-06T03:38:42.244899+0000","last_clean":"2024-06-06T03:38:42.244899+0000","last_became_active":"2024-06-06T03:35:10.761104+0000","last_became_peered":"2024-06-06T03:35:10.761104+0000","last_unstale":"2024-06-06T03:38:42.244899+0000","last_undegraded":"2024-06-06T03:38:42.244899+0000","last_fullsized":"2024-06-06T03:38:42.244899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:51:39.867088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1950,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890803+0000","last_change":"2024-06-06T03:35:13.856047+0000","last_active":"2024-06-06T03:38:37.890803+0000","last_peered":"2024-06-06T03:38:37.890803+0000","last_clean":"2024-06-06T03:38:37.890803+0000","last_became_active":"2024-06-06T03:35:13.855718+0000","last_became_peered":"2024-06-06T03:35:13.855718+0000","last_unstale":"2024-06-06T03:38:37.890803+0000","last_undegraded":"2024-06-06T03:38:37.890803+0000","last_fullsized":"2024-06-06T03:38:37.890803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:18:40.926872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'67","reported_seq":127,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:39.477847+0000","last_change":"2024-06-06T03:35:09.827283+0000","last_active":"2024-06-06T03:38:39.477847+0000","last_peered":"2024-06-06T03:38:39.477847+0000","last_clean":"2024-06-06T03:38:39.477847+0000","last_became_active":"2024-06-06T03:35:09.826595+0000","last_became_peered":"2024-06-06T03:35:09.826595+0000","last_unstale":"2024-06-06T03:38:39.477847+0000","last_undegraded":"2024-06-06T03:38:39.477847+0000","last_fullsized":"2024-06-06T03:38:39.477847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:34:16.398847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890527+0000","last_change":"2024-06-06T03:35:13.851900+0000","last_active":"2024-06-06T03:38:37.890527+0000","last_peered":"2024-06-06T03:38:37.890527+0000","last_clean":"2024-06-06T03:38:37.890527+0000","last_became_active":"2024-06-06T03:35:13.851477+0000","last_became_peered":"2024-06-06T03:35:13.851477+0000","last_unstale":"2024-06-06T03:38:37.890527+0000","last_undegraded":"2024-06-06T03:38:37.890527+0000","last_fullsized":"2024-06-06T03:38:37.890527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:31:54.001226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890429+0000","last_change":"2024-06-06T03:35:09.824675+0000","last_active":"2024-06-06T03:38:37.890429+0000","last_peered":"2024-06-06T03:38:37.890429+0000","last_clean":"2024-06-06T03:38:37.890429+0000","last_became_active":"2024-06-06T03:35:09.824347+0000","last_became_peered":"2024-06-06T03:35:09.824347+0000","last_unstale":"2024-06-06T03:38:37.890429+0000","last_undegraded":"2024-06-06T03:38:37.890429+0000","last_fullsized":"2024-06-06T03:38:37.890429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:01:43.080552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891036+0000","last_change":"2024-06-06T03:35:13.854706+0000","last_active":"2024-06-06T03:38:37.891036+0000","last_peered":"2024-06-06T03:38:37.891036+0000","last_clean":"2024-06-06T03:38:37.891036+0000","last_became_active":"2024-06-06T03:35:13.854392+0000","last_became_peered":"2024-06-06T03:35:13.854392+0000","last_unstale":"2024-06-06T03:38:37.891036+0000","last_undegraded":"2024-06-06T03:38:37.891036+0000","last_fullsized":"2024-06-06T03:38:37.891036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:58:10.102367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890213+0000","last_change":"2024-06-06T03:35:13.853594+0000","last_active":"2024-06-06T03:38:37.890213+0000","last_peered":"2024-06-06T03:38:37.890213+0000","last_clean":"2024-06-06T03:38:37.890213+0000","last_became_active":"2024-06-06T03:35:13.852962+0000","last_became_peered":"2024-06-06T03:35:13.852962+0000","last_unstale":"2024-06-06T03:38:37.890213+0000","last_undegraded":"2024-06-06T03:38:37.890213+0000","last_fullsized":"2024-06-06T03:38:37.890213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:02:44.233868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632115+0000","last_change":"2024-06-06T03:35:13.848790+0000","last_active":"2024-06-06T03:38:37.632115+0000","last_peered":"2024-06-06T03:38:37.632115+0000","last_clean":"2024-06-06T03:38:37.632115+0000","last_became_active":"2024-06-06T03:35:13.848442+0000","last_became_peered":"2024-06-06T03:35:13.848442+0000","last_unstale":"2024-06-06T03:38:37.632115+0000","last_undegraded":"2024-06-06T03:38:37.632115+0000","last_fullsized":"2024-06-06T03:38:37.632115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:51.382210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632024+0000","last_change":"2024-06-06T03:35:13.866999+0000","last_active":"2024-06-06T03:38:37.632024+0000","last_peered":"2024-06-06T03:38:37.632024+0000","last_clean":"2024-06-06T03:38:37.632024+0000","last_became_active":"2024-06-06T03:35:13.866664+0000","last_became_peered":"2024-06-06T03:35:13.866664+0000","last_unstale":"2024-06-06T03:38:37.632024+0000","last_undegraded":"2024-06-06T03:38:37.632024+0000","last_fullsized":"2024-06-06T03:38:37.632024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:31:14.166122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891400+0000","last_change":"2024-06-06T03:35:13.849062+0000","last_active":"2024-06-06T03:38:37.891400+0000","last_peered":"2024-06-06T03:38:37.891400+0000","last_clean":"2024-06-06T03:38:37.891400+0000","last_became_active":"2024-06-06T03:35:13.848750+0000","last_became_peered":"2024-06-06T03:35:13.848750+0000","last_unstale":"2024-06-06T03:38:37.891400+0000","last_undegraded":"2024-06-06T03:38:37.891400+0000","last_fullsized":"2024-06-06T03:38:37.891400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:55:58.084908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890931+0000","last_change":"2024-06-06T03:35:13.857601+0000","last_active":"2024-06-06T03:38:37.890931+0000","last_peered":"2024-06-06T03:38:37.890931+0000","last_clean":"2024-06-06T03:38:37.890931+0000","last_became_active":"2024-06-06T03:35:13.857270+0000","last_became_peered":"2024-06-06T03:35:13.857270+0000","last_unstale":"2024-06-06T03:38:37.890931+0000","last_undegraded":"2024-06-06T03:38:37.890931+0000","last_fullsized":"2024-06-06T03:38:37.890931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:56:29.970210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890367+0000","last_change":"2024-06-06T03:35:13.846699+0000","last_active":"2024-06-06T03:38:37.890367+0000","last_peered":"2024-06-06T03:38:37.890367+0000","last_clean":"2024-06-06T03:38:37.890367+0000","last_became_active":"2024-06-06T03:35:13.846355+0000","last_became_peered":"2024-06-06T03:35:13.846355+0000","last_unstale":"2024-06-06T03:38:37.890367+0000","last_undegraded":"2024-06-06T03:38:37.890367+0000","last_fullsized":"2024-06-06T03:38:37.890367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:50:22.537339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632406+0000","last_change":"2024-06-06T03:35:13.854013+0000","last_active":"2024-06-06T03:38:37.632406+0000","last_peered":"2024-06-06T03:38:37.632406+0000","last_clean":"2024-06-06T03:38:37.632406+0000","last_became_active":"2024-06-06T03:35:13.853739+0000","last_became_peered":"2024-06-06T03:35:13.853739+0000","last_unstale":"2024-06-06T03:38:37.632406+0000","last_undegraded":"2024-06-06T03:38:37.632406+0000","last_fullsized":"2024-06-06T03:38:37.632406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:27:12.330943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717814+0000","last_change":"2024-06-06T03:35:13.854251+0000","last_active":"2024-06-06T03:38:37.717814+0000","last_peered":"2024-06-06T03:38:37.717814+0000","last_clean":"2024-06-06T03:38:37.717814+0000","last_became_active":"2024-06-06T03:35:13.853915+0000","last_became_peered":"2024-06-06T03:35:13.853915+0000","last_unstale":"2024-06-06T03:38:37.717814+0000","last_undegraded":"2024-06-06T03:38:37.717814+0000","last_fullsized":"2024-06-06T03:38:37.717814+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:08.813316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733345+0000","last_change":"2024-06-06T03:35:13.849998+0000","last_active":"2024-06-06T03:38:37.733345+0000","last_peered":"2024-06-06T03:38:37.733345+0000","last_clean":"2024-06-06T03:38:37.733345+0000","last_became_active":"2024-06-06T03:35:13.848789+0000","last_became_peered":"2024-06-06T03:35:13.848789+0000","last_unstale":"2024-06-06T03:38:37.733345+0000","last_undegraded":"2024-06-06T03:38:37.733345+0000","last_fullsized":"2024-06-06T03:38:37.733345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:53:39.043356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.716965+0000","last_change":"2024-06-06T03:35:13.858020+0000","last_active":"2024-06-06T03:38:37.716965+0000","last_peered":"2024-06-06T03:38:37.716965+0000","last_clean":"2024-06-06T03:38:37.716965+0000","last_became_active":"2024-06-06T03:35:13.857723+0000","last_became_peered":"2024-06-06T03:35:13.857723+0000","last_unstale":"2024-06-06T03:38:37.716965+0000","last_undegraded":"2024-06-06T03:38:37.716965+0000","last_fullsized":"2024-06-06T03:38:37.716965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:08:08.651213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.644929+0000","last_change":"2024-06-06T03:35:13.867546+0000","last_active":"2024-06-06T03:38:37.644929+0000","last_peered":"2024-06-06T03:38:37.644929+0000","last_clean":"2024-06-06T03:38:37.644929+0000","last_became_active":"2024-06-06T03:35:13.867182+0000","last_became_peered":"2024-06-06T03:35:13.867182+0000","last_unstale":"2024-06-06T03:38:37.644929+0000","last_undegraded":"2024-06-06T03:38:37.644929+0000","last_fullsized":"2024-06-06T03:38:37.644929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:25:55.831943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890895+0000","last_change":"2024-06-06T03:35:13.849080+0000","last_active":"2024-06-06T03:38:37.890895+0000","last_peered":"2024-06-06T03:38:37.890895+0000","last_clean":"2024-06-06T03:38:37.890895+0000","last_became_active":"2024-06-06T03:35:13.848476+0000","last_became_peered":"2024-06-06T03:35:13.848476+0000","last_unstale":"2024-06-06T03:38:37.890895+0000","last_undegraded":"2024-06-06T03:38:37.890895+0000","last_fullsized":"2024-06-06T03:38:37.890895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:48:37.306528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890161+0000","last_change":"2024-06-06T03:35:13.848442+0000","last_active":"2024-06-06T03:38:37.890161+0000","last_peered":"2024-06-06T03:38:37.890161+0000","last_clean":"2024-06-06T03:38:37.890161+0000","last_became_active":"2024-06-06T03:35:13.848074+0000","last_became_peered":"2024-06-06T03:35:13.848074+0000","last_unstale":"2024-06-06T03:38:37.890161+0000","last_undegraded":"2024-06-06T03:38:37.890161+0000","last_fullsized":"2024-06-06T03:38:37.890161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:07:00.197671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107001,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799175,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2264,"num_read_kb":1348,"num_write":510,"num_write_kb":2386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":749568,"data_stored":2004261,"data_compressed":85238,"data_compressed_allocated":110592,"data_compressed_original":1732608,"omap_allocated":0,"internal_metadata":0},"log_size":387,"ondisk_log_size":387,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383737,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":480334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645374,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":1274582,"data_compressed":44602,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808900,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":882208,"data_compressed":22169,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070537,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":882474,"data_compressed":22169,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332174,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":1084037,"data_compressed":38547,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495699,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":489548,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":558446,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":490828,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1097728,"data_stored":559528,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182498,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1013350,"data_compressed":28182,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920862,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":489350,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757337,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1130496,"data_stored":1410328,"data_compressed":53749,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331973,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202101,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T03:38:45.507 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-06T03:38:45.507 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T03:38:45.765 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:46.181 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:45 smithi040 ceph-mon[32220]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:38:46.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:45 smithi131 ceph-mon[36687]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:38:46.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:45 smithi104 ceph-mon[36569]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:38:47.181 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:46 smithi040 ceph-mon[32220]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 131 KiB/s wr, 78 op/s 2024-06-06T03:38:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:46 smithi131 ceph-mon[36687]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 131 KiB/s wr, 78 op/s 2024-06-06T03:38:47.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.289+0000 7fdf86cee700 1 -- 172.21.15.40:0/2600549649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf801024e0 msgr2=0x7fdf80102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:47.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.289+0000 7fdf86cee700 1 --2- 172.21.15.40:0/2600549649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf801024e0 0x7fdf80102960 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009990 tx=0x7fdf70009ca0 comp rx=0 tx=0).stop 2024-06-06T03:38:47.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.289+0000 7fdf86cee700 1 -- 172.21.15.40:0/2600549649 shutdown_connections 2024-06-06T03:38:47.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.289+0000 7fdf86cee700 1 --2- 172.21.15.40:0/2600549649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf80102ea0 0x7fdf8010f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:47.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.289+0000 7fdf86cee700 1 --2- 172.21.15.40:0/2600549649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf801085f0 0x7fdf801089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:47.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.289+0000 7fdf86cee700 1 --2- 172.21.15.40:0/2600549649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf801024e0 0x7fdf80102960 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:47.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.289+0000 7fdf86cee700 1 -- 172.21.15.40:0/2600549649 >> 172.21.15.40:0/2600549649 conn(0x7fdf80079640 msgr2=0x7fdf80079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:47.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.289+0000 7fdf86cee700 1 -- 172.21.15.40:0/2600549649 shutdown_connections 2024-06-06T03:38:47.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.289+0000 7fdf86cee700 1 -- 172.21.15.40:0/2600549649 wait complete. 2024-06-06T03:38:47.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.290+0000 7fdf86cee700 1 Processor -- start 2024-06-06T03:38:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.290+0000 7fdf86cee700 1 -- start start 2024-06-06T03:38:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.290+0000 7fdf86cee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf801024e0 0x7fdf8019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.290+0000 7fdf86cee700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf80102ea0 0x7fdf8019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.290+0000 7fdf86cee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf801085f0 0x7fdf80198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.290+0000 7fdf86cee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdf801127b0 con 0x7fdf801024e0 2024-06-06T03:38:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.290+0000 7fdf86cee700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdf80112630 con 0x7fdf80102ea0 2024-06-06T03:38:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.290+0000 7fdf86cee700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdf80112930 con 0x7fdf801085f0 2024-06-06T03:38:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.290+0000 7fdf84a8a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf801024e0 0x7fdf8019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.290+0000 7fdf84a8a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf801024e0 0x7fdf8019e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44530/0 (socket says 172.21.15.40:44530) 2024-06-06T03:38:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.290+0000 7fdf84a8a700 1 -- 172.21.15.40:0/3947828849 learned_addr learned my addr 172.21.15.40:0/3947828849 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.291+0000 7fdf77fff700 1 --2- 172.21.15.40:0/3947828849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf80102ea0 0x7fdf8019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.291+0000 7fdf8528b700 1 --2- 172.21.15.40:0/3947828849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf801085f0 0x7fdf80198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.291+0000 7fdf84a8a700 1 -- 172.21.15.40:0/3947828849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf801085f0 msgr2=0x7fdf80198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.291+0000 7fdf84a8a700 1 --2- 172.21.15.40:0/3947828849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf801085f0 0x7fdf80198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.291+0000 7fdf84a8a700 1 -- 172.21.15.40:0/3947828849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf80102ea0 msgr2=0x7fdf8019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.291+0000 7fdf84a8a700 1 --2- 172.21.15.40:0/3947828849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf80102ea0 0x7fdf8019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.291+0000 7fdf84a8a700 1 -- 172.21.15.40:0/3947828849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf80199190 con 0x7fdf801024e0 2024-06-06T03:38:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.291+0000 7fdf84a8a700 1 --2- 172.21.15.40:0/3947828849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf801024e0 0x7fdf8019e870 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00c9b0 tx=0x7fdf7c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.291+0000 7fdf75ffb700 1 -- 172.21.15.40:0/3947828849 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c007b00 con 0x7fdf801024e0 2024-06-06T03:38:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.291+0000 7fdf75ffb700 1 -- 172.21.15.40:0/3947828849 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf7c00ce50 con 0x7fdf801024e0 2024-06-06T03:38:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.291+0000 7fdf86cee700 1 -- 172.21.15.40:0/3947828849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf80199480 con 0x7fdf801024e0 2024-06-06T03:38:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.291+0000 7fdf75ffb700 1 -- 172.21.15.40:0/3947828849 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c018660 con 0x7fdf801024e0 2024-06-06T03:38:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.291+0000 7fdf86cee700 1 -- 172.21.15.40:0/3947828849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf800746c0 con 0x7fdf801024e0 2024-06-06T03:38:47.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.292+0000 7fdf86cee700 1 -- 172.21.15.40:0/3947828849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf80069180 con 0x7fdf801024e0 2024-06-06T03:38:47.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.293+0000 7fdf75ffb700 1 -- 172.21.15.40:0/3947828849 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fdf7c00f450 con 0x7fdf801024e0 2024-06-06T03:38:47.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.294+0000 7fdf75ffb700 1 --2- 172.21.15.40:0/3947828849 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fdf680a2e20 0x7fdf680a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:47.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.294+0000 7fdf77fff700 1 --2- 172.21.15.40:0/3947828849 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fdf680a2e20 0x7fdf680a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:47.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.294+0000 7fdf75ffb700 1 -- 172.21.15.40:0/3947828849 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fdf7c0cf400 con 0x7fdf801024e0 2024-06-06T03:38:47.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.295+0000 7fdf77fff700 1 --2- 172.21.15.40:0/3947828849 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fdf680a2e20 0x7fdf680a52e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000b5c0 tx=0x7fdf70005960 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:47.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.299+0000 7fdf75ffb700 1 -- 172.21.15.40:0/3947828849 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdf7c014030 con 0x7fdf801024e0 2024-06-06T03:38:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:46 smithi104 ceph-mon[36569]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 131 KiB/s wr, 78 op/s 2024-06-06T03:38:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.499+0000 7fdf86cee700 1 -- 172.21.15.40:0/3947828849 --> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf80199f60 con 0x7fdf680a2e20 2024-06-06T03:38:47.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.507+0000 7fdf75ffb700 1 -- 172.21.15.40:0/3947828849 <== mgr.24790 v2:172.21.15.40:6800/1568651190 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258096 (secure 0 0 0) 0x7fdf80199f60 con 0x7fdf680a2e20 2024-06-06T03:38:47.510 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:38:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.509+0000 7fdf86cee700 1 -- 172.21.15.40:0/3947828849 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fdf680a2e20 msgr2=0x7fdf680a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.509+0000 7fdf86cee700 1 --2- 172.21.15.40:0/3947828849 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fdf680a2e20 0x7fdf680a52e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000b5c0 tx=0x7fdf70005960 comp rx=0 tx=0).stop 2024-06-06T03:38:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.509+0000 7fdf86cee700 1 -- 172.21.15.40:0/3947828849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf801024e0 msgr2=0x7fdf8019e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.509+0000 7fdf86cee700 1 --2- 172.21.15.40:0/3947828849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf801024e0 0x7fdf8019e870 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00c9b0 tx=0x7fdf7c00ccc0 comp rx=0 tx=0).stop 2024-06-06T03:38:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.510+0000 7fdf86cee700 1 -- 172.21.15.40:0/3947828849 shutdown_connections 2024-06-06T03:38:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.510+0000 7fdf86cee700 1 --2- 172.21.15.40:0/3947828849 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fdf680a2e20 0x7fdf680a52e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.510+0000 7fdf86cee700 1 --2- 172.21.15.40:0/3947828849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf801085f0 0x7fdf80198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.510+0000 7fdf86cee700 1 --2- 172.21.15.40:0/3947828849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf801024e0 0x7fdf8019e870 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.510+0000 7fdf86cee700 1 --2- 172.21.15.40:0/3947828849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf80102ea0 0x7fdf8019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.510+0000 7fdf86cee700 1 -- 172.21.15.40:0/3947828849 >> 172.21.15.40:0/3947828849 conn(0x7fdf80079640 msgr2=0x7fdf80100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.510+0000 7fdf86cee700 1 -- 172.21.15.40:0/3947828849 shutdown_connections 2024-06-06T03:38:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:47.510+0000 7fdf86cee700 1 -- 172.21.15.40:0/3947828849 wait complete. 2024-06-06T03:38:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-06-06T03:38:48.089 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-06-06T03:38:45.647012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389777,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2521,"num_read_kb":1693,"num_write":713,"num_write_kb":5326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1037,"ondisk_log_size":1037,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365872,"kb_used_data":12516,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575952,"statfs":{"total":1151940427776,"available":1151565774848,"internally_reserved":0,"allocated":12816384,"data_stored":10023771,"data_compressed":245034,"data_compressed_allocated":319488,"data_compressed_original":3969024,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":404,"num_read_kb":466,"num_write":148,"num_write_kb":915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.995312"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890341+0000","last_change":"2024-06-06T03:35:13.854938+0000","last_active":"2024-06-06T03:38:37.890341+0000","last_peered":"2024-06-06T03:38:37.890341+0000","last_clean":"2024-06-06T03:38:37.890341+0000","last_became_active":"2024-06-06T03:35:13.853574+0000","last_became_peered":"2024-06-06T03:35:13.853574+0000","last_unstale":"2024-06-06T03:38:37.890341+0000","last_undegraded":"2024-06-06T03:38:37.890341+0000","last_fullsized":"2024-06-06T03:38:37.890341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:21:09.256002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.892012+0000","last_change":"2024-06-06T03:35:13.858299+0000","last_active":"2024-06-06T03:38:37.892012+0000","last_peered":"2024-06-06T03:38:37.892012+0000","last_clean":"2024-06-06T03:38:37.892012+0000","last_became_active":"2024-06-06T03:35:13.857993+0000","last_became_peered":"2024-06-06T03:35:13.857993+0000","last_unstale":"2024-06-06T03:38:37.892012+0000","last_undegraded":"2024-06-06T03:38:37.892012+0000","last_fullsized":"2024-06-06T03:38:37.892012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:31.811056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891638+0000","last_change":"2024-06-06T03:35:13.852783+0000","last_active":"2024-06-06T03:38:37.891638+0000","last_peered":"2024-06-06T03:38:37.891638+0000","last_clean":"2024-06-06T03:38:37.891638+0000","last_became_active":"2024-06-06T03:35:13.852468+0000","last_became_peered":"2024-06-06T03:35:13.852468+0000","last_unstale":"2024-06-06T03:38:37.891638+0000","last_undegraded":"2024-06-06T03:38:37.891638+0000","last_fullsized":"2024-06-06T03:38:37.891638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:04:59.051277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.629947+0000","last_change":"2024-06-06T03:35:13.845223+0000","last_active":"2024-06-06T03:38:37.629947+0000","last_peered":"2024-06-06T03:38:37.629947+0000","last_clean":"2024-06-06T03:38:37.629947+0000","last_became_active":"2024-06-06T03:35:13.844854+0000","last_became_peered":"2024-06-06T03:35:13.844854+0000","last_unstale":"2024-06-06T03:38:37.629947+0000","last_undegraded":"2024-06-06T03:38:37.629947+0000","last_fullsized":"2024-06-06T03:38:37.629947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:35:20.058188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.727987+0000","last_change":"2024-06-06T03:35:13.853703+0000","last_active":"2024-06-06T03:38:37.727987+0000","last_peered":"2024-06-06T03:38:37.727987+0000","last_clean":"2024-06-06T03:38:37.727987+0000","last_became_active":"2024-06-06T03:35:13.853358+0000","last_became_peered":"2024-06-06T03:35:13.853358+0000","last_unstale":"2024-06-06T03:38:37.727987+0000","last_undegraded":"2024-06-06T03:38:37.727987+0000","last_fullsized":"2024-06-06T03:38:37.727987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:23:39.092306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736404+0000","last_change":"2024-06-06T03:35:13.859414+0000","last_active":"2024-06-06T03:38:37.736404+0000","last_peered":"2024-06-06T03:38:37.736404+0000","last_clean":"2024-06-06T03:38:37.736404+0000","last_became_active":"2024-06-06T03:35:13.859038+0000","last_became_peered":"2024-06-06T03:35:13.859038+0000","last_unstale":"2024-06-06T03:38:37.736404+0000","last_undegraded":"2024-06-06T03:38:37.736404+0000","last_fullsized":"2024-06-06T03:38:37.736404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:14:31.780064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890276+0000","last_change":"2024-06-06T03:35:13.859245+0000","last_active":"2024-06-06T03:38:37.890276+0000","last_peered":"2024-06-06T03:38:37.890276+0000","last_clean":"2024-06-06T03:38:37.890276+0000","last_became_active":"2024-06-06T03:35:13.858857+0000","last_became_peered":"2024-06-06T03:35:13.858857+0000","last_unstale":"2024-06-06T03:38:37.890276+0000","last_undegraded":"2024-06-06T03:38:37.890276+0000","last_fullsized":"2024-06-06T03:38:37.890276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:03:23.240007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717064+0000","last_change":"2024-06-06T03:35:13.851249+0000","last_active":"2024-06-06T03:38:37.717064+0000","last_peered":"2024-06-06T03:38:37.717064+0000","last_clean":"2024-06-06T03:38:37.717064+0000","last_became_active":"2024-06-06T03:35:13.850777+0000","last_became_peered":"2024-06-06T03:35:13.850777+0000","last_unstale":"2024-06-06T03:38:37.717064+0000","last_undegraded":"2024-06-06T03:38:37.717064+0000","last_fullsized":"2024-06-06T03:38:37.717064+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:53:48.463672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.645289+0000","last_change":"2024-06-06T03:35:13.852950+0000","last_active":"2024-06-06T03:38:37.645289+0000","last_peered":"2024-06-06T03:38:37.645289+0000","last_clean":"2024-06-06T03:38:37.645289+0000","last_became_active":"2024-06-06T03:35:13.852160+0000","last_became_peered":"2024-06-06T03:35:13.852160+0000","last_unstale":"2024-06-06T03:38:37.645289+0000","last_undegraded":"2024-06-06T03:38:37.645289+0000","last_fullsized":"2024-06-06T03:38:37.645289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:00.498440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736760+0000","last_change":"2024-06-06T03:35:13.856081+0000","last_active":"2024-06-06T03:38:37.736760+0000","last_peered":"2024-06-06T03:38:37.736760+0000","last_clean":"2024-06-06T03:38:37.736760+0000","last_became_active":"2024-06-06T03:35:13.855718+0000","last_became_peered":"2024-06-06T03:35:13.855718+0000","last_unstale":"2024-06-06T03:38:37.736760+0000","last_undegraded":"2024-06-06T03:38:37.736760+0000","last_fullsized":"2024-06-06T03:38:37.736760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:33:38.171996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890584+0000","last_change":"2024-06-06T03:35:13.861850+0000","last_active":"2024-06-06T03:38:37.890584+0000","last_peered":"2024-06-06T03:38:37.890584+0000","last_clean":"2024-06-06T03:38:37.890584+0000","last_became_active":"2024-06-06T03:35:13.861513+0000","last_became_peered":"2024-06-06T03:35:13.861513+0000","last_unstale":"2024-06-06T03:38:37.890584+0000","last_undegraded":"2024-06-06T03:38:37.890584+0000","last_fullsized":"2024-06-06T03:38:37.890584+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:32:45.938302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.629839+0000","last_change":"2024-06-06T03:35:13.847742+0000","last_active":"2024-06-06T03:38:37.629839+0000","last_peered":"2024-06-06T03:38:37.629839+0000","last_clean":"2024-06-06T03:38:37.629839+0000","last_became_active":"2024-06-06T03:35:13.847409+0000","last_became_peered":"2024-06-06T03:35:13.847409+0000","last_unstale":"2024-06-06T03:38:37.629839+0000","last_undegraded":"2024-06-06T03:38:37.629839+0000","last_fullsized":"2024-06-06T03:38:37.629839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:20:31.353919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890838+0000","last_change":"2024-06-06T03:35:13.848785+0000","last_active":"2024-06-06T03:38:37.890838+0000","last_peered":"2024-06-06T03:38:37.890838+0000","last_clean":"2024-06-06T03:38:37.890838+0000","last_became_active":"2024-06-06T03:35:13.848048+0000","last_became_peered":"2024-06-06T03:35:13.848048+0000","last_unstale":"2024-06-06T03:38:37.890838+0000","last_undegraded":"2024-06-06T03:38:37.890838+0000","last_fullsized":"2024-06-06T03:38:37.890838+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:43.657761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891499+0000","last_change":"2024-06-06T03:35:13.856083+0000","last_active":"2024-06-06T03:38:37.891499+0000","last_peered":"2024-06-06T03:38:37.891499+0000","last_clean":"2024-06-06T03:38:37.891499+0000","last_became_active":"2024-06-06T03:35:13.854390+0000","last_became_peered":"2024-06-06T03:35:13.854390+0000","last_unstale":"2024-06-06T03:38:37.891499+0000","last_undegraded":"2024-06-06T03:38:37.891499+0000","last_fullsized":"2024-06-06T03:38:37.891499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:51:37.163814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890857+0000","last_change":"2024-06-06T03:35:13.852611+0000","last_active":"2024-06-06T03:38:37.890857+0000","last_peered":"2024-06-06T03:38:37.890857+0000","last_clean":"2024-06-06T03:38:37.890857+0000","last_became_active":"2024-06-06T03:35:13.852244+0000","last_became_peered":"2024-06-06T03:35:13.852244+0000","last_unstale":"2024-06-06T03:38:37.890857+0000","last_undegraded":"2024-06-06T03:38:37.890857+0000","last_fullsized":"2024-06-06T03:38:37.890857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:59:53.513150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.644789+0000","last_change":"2024-06-06T03:35:13.857840+0000","last_active":"2024-06-06T03:38:37.644789+0000","last_peered":"2024-06-06T03:38:37.644789+0000","last_clean":"2024-06-06T03:38:37.644789+0000","last_became_active":"2024-06-06T03:35:13.857137+0000","last_became_peered":"2024-06-06T03:35:13.857137+0000","last_unstale":"2024-06-06T03:38:37.644789+0000","last_undegraded":"2024-06-06T03:38:37.644789+0000","last_fullsized":"2024-06-06T03:38:37.644789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:42:07.924735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890462+0000","last_change":"2024-06-06T03:35:13.857033+0000","last_active":"2024-06-06T03:38:37.890462+0000","last_peered":"2024-06-06T03:38:37.890462+0000","last_clean":"2024-06-06T03:38:37.890462+0000","last_became_active":"2024-06-06T03:35:13.856519+0000","last_became_peered":"2024-06-06T03:35:13.856519+0000","last_unstale":"2024-06-06T03:38:37.890462+0000","last_undegraded":"2024-06-06T03:38:37.890462+0000","last_fullsized":"2024-06-06T03:38:37.890462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:05:25.668344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630726+0000","last_change":"2024-06-06T03:35:13.851356+0000","last_active":"2024-06-06T03:38:37.630726+0000","last_peered":"2024-06-06T03:38:37.630726+0000","last_clean":"2024-06-06T03:38:37.630726+0000","last_became_active":"2024-06-06T03:35:13.851030+0000","last_became_peered":"2024-06-06T03:35:13.851030+0000","last_unstale":"2024-06-06T03:38:37.630726+0000","last_undegraded":"2024-06-06T03:38:37.630726+0000","last_fullsized":"2024-06-06T03:38:37.630726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:49:38.833022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632287+0000","last_change":"2024-06-06T03:35:13.858613+0000","last_active":"2024-06-06T03:38:37.632287+0000","last_peered":"2024-06-06T03:38:37.632287+0000","last_clean":"2024-06-06T03:38:37.632287+0000","last_became_active":"2024-06-06T03:35:13.858182+0000","last_became_peered":"2024-06-06T03:35:13.858182+0000","last_unstale":"2024-06-06T03:38:37.632287+0000","last_undegraded":"2024-06-06T03:38:37.632287+0000","last_fullsized":"2024-06-06T03:38:37.632287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:23:08.216431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'15","reported_seq":234,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:42.244022+0000","last_change":"2024-06-06T03:35:09.821896+0000","last_active":"2024-06-06T03:38:42.244022+0000","last_peered":"2024-06-06T03:38:42.244022+0000","last_clean":"2024-06-06T03:38:42.244022+0000","last_became_active":"2024-06-06T03:35:09.821567+0000","last_became_peered":"2024-06-06T03:35:09.821567+0000","last_unstale":"2024-06-06T03:38:42.244022+0000","last_undegraded":"2024-06-06T03:38:42.244022+0000","last_fullsized":"2024-06-06T03:38:42.244022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:04:51.918393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717587+0000","last_change":"2024-06-06T03:35:13.857357+0000","last_active":"2024-06-06T03:38:37.717587+0000","last_peered":"2024-06-06T03:38:37.717587+0000","last_clean":"2024-06-06T03:38:37.717587+0000","last_became_active":"2024-06-06T03:35:13.857016+0000","last_became_peered":"2024-06-06T03:35:13.857016+0000","last_unstale":"2024-06-06T03:38:37.717587+0000","last_undegraded":"2024-06-06T03:38:37.717587+0000","last_fullsized":"2024-06-06T03:38:37.717587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:42.566425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717796+0000","last_change":"2024-06-06T03:35:10.277316+0000","last_active":"2024-06-06T03:38:37.717796+0000","last_peered":"2024-06-06T03:38:37.717796+0000","last_clean":"2024-06-06T03:38:37.717796+0000","last_became_active":"2024-06-06T03:35:10.276986+0000","last_became_peered":"2024-06-06T03:35:10.276986+0000","last_unstale":"2024-06-06T03:38:37.717796+0000","last_undegraded":"2024-06-06T03:38:37.717796+0000","last_fullsized":"2024-06-06T03:38:37.717796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:17:29.034224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.892351+0000","last_change":"2024-06-06T03:35:13.863069+0000","last_active":"2024-06-06T03:38:37.892351+0000","last_peered":"2024-06-06T03:38:37.892351+0000","last_clean":"2024-06-06T03:38:37.892351+0000","last_became_active":"2024-06-06T03:35:13.862647+0000","last_became_peered":"2024-06-06T03:35:13.862647+0000","last_unstale":"2024-06-06T03:38:37.892351+0000","last_undegraded":"2024-06-06T03:38:37.892351+0000","last_fullsized":"2024-06-06T03:38:37.892351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:48.252606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'13","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728456+0000","last_change":"2024-06-06T03:35:09.824977+0000","last_active":"2024-06-06T03:38:37.728456+0000","last_peered":"2024-06-06T03:38:37.728456+0000","last_clean":"2024-06-06T03:38:37.728456+0000","last_became_active":"2024-06-06T03:35:09.824643+0000","last_became_peered":"2024-06-06T03:35:09.824643+0000","last_unstale":"2024-06-06T03:38:37.728456+0000","last_undegraded":"2024-06-06T03:38:37.728456+0000","last_fullsized":"2024-06-06T03:38:37.728456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:35:37.208023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.644898+0000","last_change":"2024-06-06T03:35:13.860572+0000","last_active":"2024-06-06T03:38:37.644898+0000","last_peered":"2024-06-06T03:38:37.644898+0000","last_clean":"2024-06-06T03:38:37.644898+0000","last_became_active":"2024-06-06T03:35:13.858398+0000","last_became_peered":"2024-06-06T03:35:13.858398+0000","last_unstale":"2024-06-06T03:38:37.644898+0000","last_undegraded":"2024-06-06T03:38:37.644898+0000","last_fullsized":"2024-06-06T03:38:37.644898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:02:12.683096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728187+0000","last_change":"2024-06-06T03:35:09.827016+0000","last_active":"2024-06-06T03:38:37.728187+0000","last_peered":"2024-06-06T03:38:37.728187+0000","last_clean":"2024-06-06T03:38:37.728187+0000","last_became_active":"2024-06-06T03:35:09.826576+0000","last_became_peered":"2024-06-06T03:35:09.826576+0000","last_unstale":"2024-06-06T03:38:37.728187+0000","last_undegraded":"2024-06-06T03:38:37.728187+0000","last_fullsized":"2024-06-06T03:38:37.728187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:24.120549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733815+0000","last_change":"2024-06-06T03:35:13.853988+0000","last_active":"2024-06-06T03:38:37.733815+0000","last_peered":"2024-06-06T03:38:37.733815+0000","last_clean":"2024-06-06T03:38:37.733815+0000","last_became_active":"2024-06-06T03:35:13.852343+0000","last_became_peered":"2024-06-06T03:35:13.852343+0000","last_unstale":"2024-06-06T03:38:37.733815+0000","last_undegraded":"2024-06-06T03:38:37.733815+0000","last_fullsized":"2024-06-06T03:38:37.733815+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:39:03.054045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":231,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:42.211538+0000","last_change":"2024-06-06T03:35:09.829109+0000","last_active":"2024-06-06T03:38:42.211538+0000","last_peered":"2024-06-06T03:38:42.211538+0000","last_clean":"2024-06-06T03:38:42.211538+0000","last_became_active":"2024-06-06T03:35:09.828734+0000","last_became_peered":"2024-06-06T03:35:09.828734+0000","last_unstale":"2024-06-06T03:38:42.211538+0000","last_undegraded":"2024-06-06T03:38:42.211538+0000","last_fullsized":"2024-06-06T03:38:42.211538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:50.475600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":194,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.727932+0000","last_change":"2024-06-06T03:35:13.859198+0000","last_active":"2024-06-06T03:38:37.727932+0000","last_peered":"2024-06-06T03:38:37.727932+0000","last_clean":"2024-06-06T03:38:37.727932+0000","last_became_active":"2024-06-06T03:35:13.858863+0000","last_became_peered":"2024-06-06T03:35:13.858863+0000","last_unstale":"2024-06-06T03:38:37.727932+0000","last_undegraded":"2024-06-06T03:38:37.727932+0000","last_fullsized":"2024-06-06T03:38:37.727932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:57.153174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736340+0000","last_change":"2024-06-06T03:35:09.824352+0000","last_active":"2024-06-06T03:38:37.736340+0000","last_peered":"2024-06-06T03:38:37.736340+0000","last_clean":"2024-06-06T03:38:37.736340+0000","last_became_active":"2024-06-06T03:35:09.823998+0000","last_became_peered":"2024-06-06T03:35:09.823998+0000","last_unstale":"2024-06-06T03:38:37.736340+0000","last_undegraded":"2024-06-06T03:38:37.736340+0000","last_fullsized":"2024-06-06T03:38:37.736340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:47:24.646067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733489+0000","last_change":"2024-06-06T03:35:13.836724+0000","last_active":"2024-06-06T03:38:37.733489+0000","last_peered":"2024-06-06T03:38:37.733489+0000","last_clean":"2024-06-06T03:38:37.733489+0000","last_became_active":"2024-06-06T03:35:13.836180+0000","last_became_peered":"2024-06-06T03:35:13.836180+0000","last_unstale":"2024-06-06T03:38:37.733489+0000","last_undegraded":"2024-06-06T03:38:37.733489+0000","last_fullsized":"2024-06-06T03:38:37.733489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:08.868462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890183+0000","last_change":"2024-06-06T03:35:09.823047+0000","last_active":"2024-06-06T03:38:37.890183+0000","last_peered":"2024-06-06T03:38:37.890183+0000","last_clean":"2024-06-06T03:38:37.890183+0000","last_became_active":"2024-06-06T03:35:09.822744+0000","last_became_peered":"2024-06-06T03:35:09.822744+0000","last_unstale":"2024-06-06T03:38:37.890183+0000","last_undegraded":"2024-06-06T03:38:37.890183+0000","last_fullsized":"2024-06-06T03:38:37.890183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:20:20.165682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.716965+0000","last_change":"2024-06-06T03:35:13.857222+0000","last_active":"2024-06-06T03:38:37.716965+0000","last_peered":"2024-06-06T03:38:37.716965+0000","last_clean":"2024-06-06T03:38:37.716965+0000","last_became_active":"2024-06-06T03:35:13.856873+0000","last_became_peered":"2024-06-06T03:35:13.856873+0000","last_unstale":"2024-06-06T03:38:37.716965+0000","last_undegraded":"2024-06-06T03:38:37.716965+0000","last_fullsized":"2024-06-06T03:38:37.716965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:32:16.646275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733256+0000","last_change":"2024-06-06T03:35:09.826036+0000","last_active":"2024-06-06T03:38:37.733256+0000","last_peered":"2024-06-06T03:38:37.733256+0000","last_clean":"2024-06-06T03:38:37.733256+0000","last_became_active":"2024-06-06T03:35:09.825690+0000","last_became_peered":"2024-06-06T03:35:09.825690+0000","last_unstale":"2024-06-06T03:38:37.733256+0000","last_undegraded":"2024-06-06T03:38:37.733256+0000","last_fullsized":"2024-06-06T03:38:37.733256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:03:35.858828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630424+0000","last_change":"2024-06-06T03:35:13.849016+0000","last_active":"2024-06-06T03:38:37.630424+0000","last_peered":"2024-06-06T03:38:37.630424+0000","last_clean":"2024-06-06T03:38:37.630424+0000","last_became_active":"2024-06-06T03:35:13.848709+0000","last_became_peered":"2024-06-06T03:35:13.848709+0000","last_unstale":"2024-06-06T03:38:37.630424+0000","last_undegraded":"2024-06-06T03:38:37.630424+0000","last_fullsized":"2024-06-06T03:38:37.630424+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:42.778612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":414,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:43.212059+0000","last_change":"2024-06-06T03:35:09.827072+0000","last_active":"2024-06-06T03:38:43.212059+0000","last_peered":"2024-06-06T03:38:43.212059+0000","last_clean":"2024-06-06T03:38:43.212059+0000","last_became_active":"2024-06-06T03:35:09.826734+0000","last_became_peered":"2024-06-06T03:35:09.826734+0000","last_unstale":"2024-06-06T03:38:43.212059+0000","last_undegraded":"2024-06-06T03:38:43.212059+0000","last_fullsized":"2024-06-06T03:38:43.212059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:07:13.388398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":378,"num_read_kb":378,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.645230+0000","last_change":"2024-06-06T03:35:13.856883+0000","last_active":"2024-06-06T03:38:37.645230+0000","last_peered":"2024-06-06T03:38:37.645230+0000","last_clean":"2024-06-06T03:38:37.645230+0000","last_became_active":"2024-06-06T03:35:13.856521+0000","last_became_peered":"2024-06-06T03:35:13.856521+0000","last_unstale":"2024-06-06T03:38:37.645230+0000","last_undegraded":"2024-06-06T03:38:37.645230+0000","last_fullsized":"2024-06-06T03:38:37.645230+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:19.466218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890304+0000","last_change":"2024-06-06T03:35:09.829153+0000","last_active":"2024-06-06T03:38:37.890304+0000","last_peered":"2024-06-06T03:38:37.890304+0000","last_clean":"2024-06-06T03:38:37.890304+0000","last_became_active":"2024-06-06T03:35:09.828797+0000","last_became_peered":"2024-06-06T03:35:09.828797+0000","last_unstale":"2024-06-06T03:38:37.890304+0000","last_undegraded":"2024-06-06T03:38:37.890304+0000","last_fullsized":"2024-06-06T03:38:37.890304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:06:02.986396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.892165+0000","last_change":"2024-06-06T03:35:13.857094+0000","last_active":"2024-06-06T03:38:37.892165+0000","last_peered":"2024-06-06T03:38:37.892165+0000","last_clean":"2024-06-06T03:38:37.892165+0000","last_became_active":"2024-06-06T03:35:13.856778+0000","last_became_peered":"2024-06-06T03:35:13.856778+0000","last_unstale":"2024-06-06T03:38:37.892165+0000","last_undegraded":"2024-06-06T03:38:37.892165+0000","last_fullsized":"2024-06-06T03:38:37.892165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:50:30.097762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733828+0000","last_change":"2024-06-06T03:35:10.277802+0000","last_active":"2024-06-06T03:38:37.733828+0000","last_peered":"2024-06-06T03:38:37.733828+0000","last_clean":"2024-06-06T03:38:37.733828+0000","last_became_active":"2024-06-06T03:35:10.277417+0000","last_became_peered":"2024-06-06T03:35:10.277417+0000","last_unstale":"2024-06-06T03:38:37.733828+0000","last_undegraded":"2024-06-06T03:38:37.733828+0000","last_fullsized":"2024-06-06T03:38:37.733828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:44:35.178184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.645024+0000","last_change":"2024-06-06T03:35:13.858160+0000","last_active":"2024-06-06T03:38:37.645024+0000","last_peered":"2024-06-06T03:38:37.645024+0000","last_clean":"2024-06-06T03:38:37.645024+0000","last_became_active":"2024-06-06T03:35:13.853149+0000","last_became_peered":"2024-06-06T03:35:13.853149+0000","last_unstale":"2024-06-06T03:38:37.645024+0000","last_undegraded":"2024-06-06T03:38:37.645024+0000","last_fullsized":"2024-06-06T03:38:37.645024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:10:06.697274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":457,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:43.211963+0000","last_change":"2024-06-06T03:35:09.829443+0000","last_active":"2024-06-06T03:38:43.211963+0000","last_peered":"2024-06-06T03:38:43.211963+0000","last_clean":"2024-06-06T03:38:43.211963+0000","last_became_active":"2024-06-06T03:35:09.828093+0000","last_became_peered":"2024-06-06T03:35:09.828093+0000","last_unstale":"2024-06-06T03:38:43.211963+0000","last_undegraded":"2024-06-06T03:38:43.211963+0000","last_fullsized":"2024-06-06T03:38:43.211963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:05:19.395177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67985,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":390,"num_read_kb":261,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728167+0000","last_change":"2024-06-06T03:35:13.850869+0000","last_active":"2024-06-06T03:38:37.728167+0000","last_peered":"2024-06-06T03:38:37.728167+0000","last_clean":"2024-06-06T03:38:37.728167+0000","last_became_active":"2024-06-06T03:35:13.849489+0000","last_became_peered":"2024-06-06T03:35:13.849489+0000","last_unstale":"2024-06-06T03:38:37.728167+0000","last_undegraded":"2024-06-06T03:38:37.728167+0000","last_fullsized":"2024-06-06T03:38:37.728167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:27:45.052033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632136+0000","last_change":"2024-06-06T03:35:09.823117+0000","last_active":"2024-06-06T03:38:37.632136+0000","last_peered":"2024-06-06T03:38:37.632136+0000","last_clean":"2024-06-06T03:38:37.632136+0000","last_became_active":"2024-06-06T03:35:09.822624+0000","last_became_peered":"2024-06-06T03:35:09.822624+0000","last_unstale":"2024-06-06T03:38:37.632136+0000","last_undegraded":"2024-06-06T03:38:37.632136+0000","last_fullsized":"2024-06-06T03:38:37.632136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:59.171343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890529+0000","last_change":"2024-06-06T03:35:13.861043+0000","last_active":"2024-06-06T03:38:37.890529+0000","last_peered":"2024-06-06T03:38:37.890529+0000","last_clean":"2024-06-06T03:38:37.890529+0000","last_became_active":"2024-06-06T03:35:13.860718+0000","last_became_peered":"2024-06-06T03:35:13.860718+0000","last_unstale":"2024-06-06T03:38:37.890529+0000","last_undegraded":"2024-06-06T03:38:37.890529+0000","last_fullsized":"2024-06-06T03:38:37.890529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:34:04.595780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733532+0000","last_change":"2024-06-06T03:35:09.825505+0000","last_active":"2024-06-06T03:38:37.733532+0000","last_peered":"2024-06-06T03:38:37.733532+0000","last_clean":"2024-06-06T03:38:37.733532+0000","last_became_active":"2024-06-06T03:35:09.825127+0000","last_became_peered":"2024-06-06T03:35:09.825127+0000","last_unstale":"2024-06-06T03:38:37.733532+0000","last_undegraded":"2024-06-06T03:38:37.733532+0000","last_fullsized":"2024-06-06T03:38:37.733532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:07:27.664589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890667+0000","last_change":"2024-06-06T03:35:48.987834+0000","last_active":"2024-06-06T03:38:37.890667+0000","last_peered":"2024-06-06T03:38:37.890667+0000","last_clean":"2024-06-06T03:38:37.890667+0000","last_became_active":"2024-06-06T03:35:48.987515+0000","last_became_peered":"2024-06-06T03:35:48.987515+0000","last_unstale":"2024-06-06T03:38:37.890667+0000","last_undegraded":"2024-06-06T03:38:37.890667+0000","last_fullsized":"2024-06-06T03:38:37.890667+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:05:19.342289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":228,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:42.211332+0000","last_change":"2024-06-06T03:35:09.826192+0000","last_active":"2024-06-06T03:38:42.211332+0000","last_peered":"2024-06-06T03:38:42.211332+0000","last_clean":"2024-06-06T03:38:42.211332+0000","last_became_active":"2024-06-06T03:35:09.825481+0000","last_became_peered":"2024-06-06T03:35:09.825481+0000","last_unstale":"2024-06-06T03:38:42.211332+0000","last_undegraded":"2024-06-06T03:38:42.211332+0000","last_fullsized":"2024-06-06T03:38:42.211332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:18:30.772991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1368,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890452+0000","last_change":"2024-06-06T03:35:13.853113+0000","last_active":"2024-06-06T03:38:37.890452+0000","last_peered":"2024-06-06T03:38:37.890452+0000","last_clean":"2024-06-06T03:38:37.890452+0000","last_became_active":"2024-06-06T03:35:13.852751+0000","last_became_peered":"2024-06-06T03:35:13.852751+0000","last_unstale":"2024-06-06T03:38:37.890452+0000","last_undegraded":"2024-06-06T03:38:37.890452+0000","last_fullsized":"2024-06-06T03:38:37.890452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:53.536685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890032+0000","last_change":"2024-06-06T03:35:49.005958+0000","last_active":"2024-06-06T03:38:37.890032+0000","last_peered":"2024-06-06T03:38:37.890032+0000","last_clean":"2024-06-06T03:38:37.890032+0000","last_became_active":"2024-06-06T03:35:48.993266+0000","last_became_peered":"2024-06-06T03:35:48.993266+0000","last_unstale":"2024-06-06T03:38:37.890032+0000","last_undegraded":"2024-06-06T03:38:37.890032+0000","last_fullsized":"2024-06-06T03:38:37.890032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:47.782824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733337+0000","last_change":"2024-06-06T03:35:13.853614+0000","last_active":"2024-06-06T03:38:37.733337+0000","last_peered":"2024-06-06T03:38:37.733337+0000","last_clean":"2024-06-06T03:38:37.733337+0000","last_became_active":"2024-06-06T03:35:13.852327+0000","last_became_peered":"2024-06-06T03:35:13.852327+0000","last_unstale":"2024-06-06T03:38:37.733337+0000","last_undegraded":"2024-06-06T03:38:37.733337+0000","last_fullsized":"2024-06-06T03:38:37.733337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:02:32.463028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728378+0000","last_change":"2024-06-06T03:35:09.826220+0000","last_active":"2024-06-06T03:38:37.728378+0000","last_peered":"2024-06-06T03:38:37.728378+0000","last_clean":"2024-06-06T03:38:37.728378+0000","last_became_active":"2024-06-06T03:35:09.825887+0000","last_became_peered":"2024-06-06T03:35:09.825887+0000","last_unstale":"2024-06-06T03:38:37.728378+0000","last_undegraded":"2024-06-06T03:38:37.728378+0000","last_fullsized":"2024-06-06T03:38:37.728378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:23:33.704822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890232+0000","last_change":"2024-06-06T03:35:13.852455+0000","last_active":"2024-06-06T03:38:37.890232+0000","last_peered":"2024-06-06T03:38:37.890232+0000","last_clean":"2024-06-06T03:38:37.890232+0000","last_became_active":"2024-06-06T03:35:13.852134+0000","last_became_peered":"2024-06-06T03:35:13.852134+0000","last_unstale":"2024-06-06T03:38:37.890232+0000","last_undegraded":"2024-06-06T03:38:37.890232+0000","last_fullsized":"2024-06-06T03:38:37.890232+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:00:16.418479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630691+0000","last_change":"2024-06-06T03:35:10.277767+0000","last_active":"2024-06-06T03:38:37.630691+0000","last_peered":"2024-06-06T03:38:37.630691+0000","last_clean":"2024-06-06T03:38:37.630691+0000","last_became_active":"2024-06-06T03:35:10.277381+0000","last_became_peered":"2024-06-06T03:35:10.277381+0000","last_unstale":"2024-06-06T03:38:37.630691+0000","last_undegraded":"2024-06-06T03:38:37.630691+0000","last_fullsized":"2024-06-06T03:38:37.630691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:12:13.425284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736536+0000","last_change":"2024-06-06T03:35:13.859594+0000","last_active":"2024-06-06T03:38:37.736536+0000","last_peered":"2024-06-06T03:38:37.736536+0000","last_clean":"2024-06-06T03:38:37.736536+0000","last_became_active":"2024-06-06T03:35:13.859240+0000","last_became_peered":"2024-06-06T03:35:13.859240+0000","last_unstale":"2024-06-06T03:38:37.736536+0000","last_undegraded":"2024-06-06T03:38:37.736536+0000","last_fullsized":"2024-06-06T03:38:37.736536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:39:34.557581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'8","reported_seq":230,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:42.244501+0000","last_change":"2024-06-06T03:35:09.826204+0000","last_active":"2024-06-06T03:38:42.244501+0000","last_peered":"2024-06-06T03:38:42.244501+0000","last_clean":"2024-06-06T03:38:42.244501+0000","last_became_active":"2024-06-06T03:35:09.825815+0000","last_became_peered":"2024-06-06T03:35:09.825815+0000","last_unstale":"2024-06-06T03:38:42.244501+0000","last_undegraded":"2024-06-06T03:38:42.244501+0000","last_fullsized":"2024-06-06T03:38:42.244501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:02:04.169227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1427,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.736428+0000","last_change":"2024-06-06T03:35:48.992786+0000","last_active":"2024-06-06T03:38:37.736428+0000","last_peered":"2024-06-06T03:38:37.736428+0000","last_clean":"2024-06-06T03:38:37.736428+0000","last_became_active":"2024-06-06T03:35:48.992456+0000","last_became_peered":"2024-06-06T03:35:48.992456+0000","last_unstale":"2024-06-06T03:38:37.736428+0000","last_undegraded":"2024-06-06T03:38:37.736428+0000","last_fullsized":"2024-06-06T03:38:37.736428+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:06:58.563975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630421+0000","last_change":"2024-06-06T03:35:49.005119+0000","last_active":"2024-06-06T03:38:37.630421+0000","last_peered":"2024-06-06T03:38:37.630421+0000","last_clean":"2024-06-06T03:38:37.630421+0000","last_became_active":"2024-06-06T03:35:49.004788+0000","last_became_peered":"2024-06-06T03:35:49.004788+0000","last_unstale":"2024-06-06T03:38:37.630421+0000","last_undegraded":"2024-06-06T03:38:37.630421+0000","last_fullsized":"2024-06-06T03:38:37.630421+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:20:06.961454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":225,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:42.210803+0000","last_change":"2024-06-06T03:35:10.278559+0000","last_active":"2024-06-06T03:38:42.210803+0000","last_peered":"2024-06-06T03:38:42.210803+0000","last_clean":"2024-06-06T03:38:42.210803+0000","last_became_active":"2024-06-06T03:35:10.278104+0000","last_became_peered":"2024-06-06T03:35:10.278104+0000","last_unstale":"2024-06-06T03:38:42.210803+0000","last_undegraded":"2024-06-06T03:38:42.210803+0000","last_fullsized":"2024-06-06T03:38:42.210803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:23:47.131731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717376+0000","last_change":"2024-06-06T03:35:13.862303+0000","last_active":"2024-06-06T03:38:37.717376+0000","last_peered":"2024-06-06T03:38:37.717376+0000","last_clean":"2024-06-06T03:38:37.717376+0000","last_became_active":"2024-06-06T03:35:13.861978+0000","last_became_peered":"2024-06-06T03:35:13.861978+0000","last_unstale":"2024-06-06T03:38:37.717376+0000","last_undegraded":"2024-06-06T03:38:37.717376+0000","last_fullsized":"2024-06-06T03:38:37.717376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:24:00.824646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.629723+0000","last_change":"2024-06-06T03:35:09.825365+0000","last_active":"2024-06-06T03:38:37.629723+0000","last_peered":"2024-06-06T03:38:37.629723+0000","last_clean":"2024-06-06T03:38:37.629723+0000","last_became_active":"2024-06-06T03:35:09.824948+0000","last_became_peered":"2024-06-06T03:35:09.824948+0000","last_unstale":"2024-06-06T03:38:37.629723+0000","last_undegraded":"2024-06-06T03:38:37.629723+0000","last_fullsized":"2024-06-06T03:38:37.629723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:23:48.189292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'148","reported_seq":406,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:39.739025+0000","last_change":"2024-06-06T03:33:44.124626+0000","last_active":"2024-06-06T03:38:39.739025+0000","last_peered":"2024-06-06T03:38:39.739025+0000","last_clean":"2024-06-06T03:38:39.739025+0000","last_became_active":"2024-06-06T03:33:44.124046+0000","last_became_peered":"2024-06-06T03:33:44.124046+0000","last_unstale":"2024-06-06T03:38:39.739025+0000","last_undegraded":"2024-06-06T03:38:39.739025+0000","last_fullsized":"2024-06-06T03:38:39.739025+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:30:14.586719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:30:14.586719+0000","last_clean_scrub_stamp":"2024-06-06T03:30:14.586719+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:17.604180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890344+0000","last_change":"2024-06-06T03:35:13.852044+0000","last_active":"2024-06-06T03:38:37.890344+0000","last_peered":"2024-06-06T03:38:37.890344+0000","last_clean":"2024-06-06T03:38:37.890344+0000","last_became_active":"2024-06-06T03:35:13.851715+0000","last_became_peered":"2024-06-06T03:35:13.851715+0000","last_unstale":"2024-06-06T03:38:37.890344+0000","last_undegraded":"2024-06-06T03:38:37.890344+0000","last_fullsized":"2024-06-06T03:38:37.890344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:19:23.778367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890093+0000","last_change":"2024-06-06T03:35:09.828463+0000","last_active":"2024-06-06T03:38:37.890093+0000","last_peered":"2024-06-06T03:38:37.890093+0000","last_clean":"2024-06-06T03:38:37.890093+0000","last_became_active":"2024-06-06T03:35:09.828121+0000","last_became_peered":"2024-06-06T03:35:09.828121+0000","last_unstale":"2024-06-06T03:38:37.890093+0000","last_undegraded":"2024-06-06T03:38:37.890093+0000","last_fullsized":"2024-06-06T03:38:37.890093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:54:25.095741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.728370+0000","last_change":"2024-06-06T03:35:10.277901+0000","last_active":"2024-06-06T03:38:37.728370+0000","last_peered":"2024-06-06T03:38:37.728370+0000","last_clean":"2024-06-06T03:38:37.728370+0000","last_became_active":"2024-06-06T03:35:10.277447+0000","last_became_peered":"2024-06-06T03:35:10.277447+0000","last_unstale":"2024-06-06T03:38:37.728370+0000","last_undegraded":"2024-06-06T03:38:37.728370+0000","last_fullsized":"2024-06-06T03:38:37.728370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:22:50.550287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891033+0000","last_change":"2024-06-06T03:35:13.864323+0000","last_active":"2024-06-06T03:38:37.891033+0000","last_peered":"2024-06-06T03:38:37.891033+0000","last_clean":"2024-06-06T03:38:37.891033+0000","last_became_active":"2024-06-06T03:35:13.863101+0000","last_became_peered":"2024-06-06T03:35:13.863101+0000","last_unstale":"2024-06-06T03:38:37.891033+0000","last_undegraded":"2024-06-06T03:38:37.891033+0000","last_fullsized":"2024-06-06T03:38:37.891033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:32.399325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'105","reported_seq":226,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:39.879130+0000","last_change":"2024-06-06T03:35:09.822280+0000","last_active":"2024-06-06T03:38:39.879130+0000","last_peered":"2024-06-06T03:38:39.879130+0000","last_clean":"2024-06-06T03:38:39.879130+0000","last_became_active":"2024-06-06T03:35:09.821922+0000","last_became_peered":"2024-06-06T03:35:09.821922+0000","last_unstale":"2024-06-06T03:38:39.879130+0000","last_undegraded":"2024-06-06T03:38:39.879130+0000","last_fullsized":"2024-06-06T03:38:39.879130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:00:03.328747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":192,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717835+0000","last_change":"2024-06-06T03:35:13.853538+0000","last_active":"2024-06-06T03:38:37.717835+0000","last_peered":"2024-06-06T03:38:37.717835+0000","last_clean":"2024-06-06T03:38:37.717835+0000","last_became_active":"2024-06-06T03:35:13.853200+0000","last_became_peered":"2024-06-06T03:35:13.853200+0000","last_unstale":"2024-06-06T03:38:37.717835+0000","last_undegraded":"2024-06-06T03:38:37.717835+0000","last_fullsized":"2024-06-06T03:38:37.717835+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:13:06.574272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.630204+0000","last_change":"2024-06-06T03:35:10.761675+0000","last_active":"2024-06-06T03:38:37.630204+0000","last_peered":"2024-06-06T03:38:37.630204+0000","last_clean":"2024-06-06T03:38:37.630204+0000","last_became_active":"2024-06-06T03:35:10.761287+0000","last_became_peered":"2024-06-06T03:35:10.761287+0000","last_unstale":"2024-06-06T03:38:37.630204+0000","last_undegraded":"2024-06-06T03:38:37.630204+0000","last_fullsized":"2024-06-06T03:38:37.630204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:58:18.524273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.892143+0000","last_change":"2024-06-06T03:35:13.847680+0000","last_active":"2024-06-06T03:38:37.892143+0000","last_peered":"2024-06-06T03:38:37.892143+0000","last_clean":"2024-06-06T03:38:37.892143+0000","last_became_active":"2024-06-06T03:35:13.847359+0000","last_became_peered":"2024-06-06T03:35:13.847359+0000","last_unstale":"2024-06-06T03:38:37.892143+0000","last_undegraded":"2024-06-06T03:38:37.892143+0000","last_fullsized":"2024-06-06T03:38:37.892143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:29:18.041078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.644794+0000","last_change":"2024-06-06T03:35:10.277491+0000","last_active":"2024-06-06T03:38:37.644794+0000","last_peered":"2024-06-06T03:38:37.644794+0000","last_clean":"2024-06-06T03:38:37.644794+0000","last_became_active":"2024-06-06T03:35:10.277135+0000","last_became_peered":"2024-06-06T03:35:10.277135+0000","last_unstale":"2024-06-06T03:38:37.644794+0000","last_undegraded":"2024-06-06T03:38:37.644794+0000","last_fullsized":"2024-06-06T03:38:37.644794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:55.198789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.727917+0000","last_change":"2024-06-06T03:35:13.846013+0000","last_active":"2024-06-06T03:38:37.727917+0000","last_peered":"2024-06-06T03:38:37.727917+0000","last_clean":"2024-06-06T03:38:37.727917+0000","last_became_active":"2024-06-06T03:35:13.845650+0000","last_became_peered":"2024-06-06T03:35:13.845650+0000","last_unstale":"2024-06-06T03:38:37.727917+0000","last_undegraded":"2024-06-06T03:38:37.727917+0000","last_fullsized":"2024-06-06T03:38:37.727917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:04:51.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.716995+0000","last_change":"2024-06-06T03:35:09.826380+0000","last_active":"2024-06-06T03:38:37.716995+0000","last_peered":"2024-06-06T03:38:37.716995+0000","last_clean":"2024-06-06T03:38:37.716995+0000","last_became_active":"2024-06-06T03:35:09.825813+0000","last_became_peered":"2024-06-06T03:35:09.825813+0000","last_unstale":"2024-06-06T03:38:37.716995+0000","last_undegraded":"2024-06-06T03:38:37.716995+0000","last_fullsized":"2024-06-06T03:38:37.716995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:24.792398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890127+0000","last_change":"2024-06-06T03:35:13.855051+0000","last_active":"2024-06-06T03:38:37.890127+0000","last_peered":"2024-06-06T03:38:37.890127+0000","last_clean":"2024-06-06T03:38:37.890127+0000","last_became_active":"2024-06-06T03:35:13.853781+0000","last_became_peered":"2024-06-06T03:35:13.853781+0000","last_unstale":"2024-06-06T03:38:37.890127+0000","last_undegraded":"2024-06-06T03:38:37.890127+0000","last_fullsized":"2024-06-06T03:38:37.890127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:17:56.174924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890090+0000","last_change":"2024-06-06T03:35:49.009583+0000","last_active":"2024-06-06T03:38:37.890090+0000","last_peered":"2024-06-06T03:38:37.890090+0000","last_clean":"2024-06-06T03:38:37.890090+0000","last_became_active":"2024-06-06T03:35:48.994275+0000","last_became_peered":"2024-06-06T03:35:48.994275+0000","last_unstale":"2024-06-06T03:38:37.890090+0000","last_undegraded":"2024-06-06T03:38:37.890090+0000","last_fullsized":"2024-06-06T03:38:37.890090+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:24:54.985854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717547+0000","last_change":"2024-06-06T03:35:13.855503+0000","last_active":"2024-06-06T03:38:37.717547+0000","last_peered":"2024-06-06T03:38:37.717547+0000","last_clean":"2024-06-06T03:38:37.717547+0000","last_became_active":"2024-06-06T03:35:13.855069+0000","last_became_peered":"2024-06-06T03:35:13.855069+0000","last_unstale":"2024-06-06T03:38:37.717547+0000","last_undegraded":"2024-06-06T03:38:37.717547+0000","last_fullsized":"2024-06-06T03:38:37.717547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:43.778258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.727997+0000","last_change":"2024-06-06T03:35:09.827116+0000","last_active":"2024-06-06T03:38:37.727997+0000","last_peered":"2024-06-06T03:38:37.727997+0000","last_clean":"2024-06-06T03:38:37.727997+0000","last_became_active":"2024-06-06T03:35:09.826756+0000","last_became_peered":"2024-06-06T03:35:09.826756+0000","last_unstale":"2024-06-06T03:38:37.727997+0000","last_undegraded":"2024-06-06T03:38:37.727997+0000","last_fullsized":"2024-06-06T03:38:37.727997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:31.501411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890818+0000","last_change":"2024-06-06T03:35:13.861189+0000","last_active":"2024-06-06T03:38:37.890818+0000","last_peered":"2024-06-06T03:38:37.890818+0000","last_clean":"2024-06-06T03:38:37.890818+0000","last_became_active":"2024-06-06T03:35:13.860751+0000","last_became_peered":"2024-06-06T03:35:13.860751+0000","last_unstale":"2024-06-06T03:38:37.890818+0000","last_undegraded":"2024-06-06T03:38:37.890818+0000","last_fullsized":"2024-06-06T03:38:37.890818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:16:34.036420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":228,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:42.244899+0000","last_change":"2024-06-06T03:35:10.761606+0000","last_active":"2024-06-06T03:38:42.244899+0000","last_peered":"2024-06-06T03:38:42.244899+0000","last_clean":"2024-06-06T03:38:42.244899+0000","last_became_active":"2024-06-06T03:35:10.761104+0000","last_became_peered":"2024-06-06T03:35:10.761104+0000","last_unstale":"2024-06-06T03:38:42.244899+0000","last_undegraded":"2024-06-06T03:38:42.244899+0000","last_fullsized":"2024-06-06T03:38:42.244899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:51:39.867088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1950,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890803+0000","last_change":"2024-06-06T03:35:13.856047+0000","last_active":"2024-06-06T03:38:37.890803+0000","last_peered":"2024-06-06T03:38:37.890803+0000","last_clean":"2024-06-06T03:38:37.890803+0000","last_became_active":"2024-06-06T03:35:13.855718+0000","last_became_peered":"2024-06-06T03:35:13.855718+0000","last_unstale":"2024-06-06T03:38:37.890803+0000","last_undegraded":"2024-06-06T03:38:37.890803+0000","last_fullsized":"2024-06-06T03:38:37.890803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:18:40.926872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'67","reported_seq":127,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:39.477847+0000","last_change":"2024-06-06T03:35:09.827283+0000","last_active":"2024-06-06T03:38:39.477847+0000","last_peered":"2024-06-06T03:38:39.477847+0000","last_clean":"2024-06-06T03:38:39.477847+0000","last_became_active":"2024-06-06T03:35:09.826595+0000","last_became_peered":"2024-06-06T03:35:09.826595+0000","last_unstale":"2024-06-06T03:38:39.477847+0000","last_undegraded":"2024-06-06T03:38:39.477847+0000","last_fullsized":"2024-06-06T03:38:39.477847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:34:16.398847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890527+0000","last_change":"2024-06-06T03:35:13.851900+0000","last_active":"2024-06-06T03:38:37.890527+0000","last_peered":"2024-06-06T03:38:37.890527+0000","last_clean":"2024-06-06T03:38:37.890527+0000","last_became_active":"2024-06-06T03:35:13.851477+0000","last_became_peered":"2024-06-06T03:35:13.851477+0000","last_unstale":"2024-06-06T03:38:37.890527+0000","last_undegraded":"2024-06-06T03:38:37.890527+0000","last_fullsized":"2024-06-06T03:38:37.890527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:31:54.001226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890429+0000","last_change":"2024-06-06T03:35:09.824675+0000","last_active":"2024-06-06T03:38:37.890429+0000","last_peered":"2024-06-06T03:38:37.890429+0000","last_clean":"2024-06-06T03:38:37.890429+0000","last_became_active":"2024-06-06T03:35:09.824347+0000","last_became_peered":"2024-06-06T03:35:09.824347+0000","last_unstale":"2024-06-06T03:38:37.890429+0000","last_undegraded":"2024-06-06T03:38:37.890429+0000","last_fullsized":"2024-06-06T03:38:37.890429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:08.805263+0000","last_clean_scrub_stamp":"2024-06-06T03:35:08.805263+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:01:43.080552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891036+0000","last_change":"2024-06-06T03:35:13.854706+0000","last_active":"2024-06-06T03:38:37.891036+0000","last_peered":"2024-06-06T03:38:37.891036+0000","last_clean":"2024-06-06T03:38:37.891036+0000","last_became_active":"2024-06-06T03:35:13.854392+0000","last_became_peered":"2024-06-06T03:35:13.854392+0000","last_unstale":"2024-06-06T03:38:37.891036+0000","last_undegraded":"2024-06-06T03:38:37.891036+0000","last_fullsized":"2024-06-06T03:38:37.891036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:58:10.102367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890213+0000","last_change":"2024-06-06T03:35:13.853594+0000","last_active":"2024-06-06T03:38:37.890213+0000","last_peered":"2024-06-06T03:38:37.890213+0000","last_clean":"2024-06-06T03:38:37.890213+0000","last_became_active":"2024-06-06T03:35:13.852962+0000","last_became_peered":"2024-06-06T03:35:13.852962+0000","last_unstale":"2024-06-06T03:38:37.890213+0000","last_undegraded":"2024-06-06T03:38:37.890213+0000","last_fullsized":"2024-06-06T03:38:37.890213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:02:44.233868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632115+0000","last_change":"2024-06-06T03:35:13.848790+0000","last_active":"2024-06-06T03:38:37.632115+0000","last_peered":"2024-06-06T03:38:37.632115+0000","last_clean":"2024-06-06T03:38:37.632115+0000","last_became_active":"2024-06-06T03:35:13.848442+0000","last_became_peered":"2024-06-06T03:35:13.848442+0000","last_unstale":"2024-06-06T03:38:37.632115+0000","last_undegraded":"2024-06-06T03:38:37.632115+0000","last_fullsized":"2024-06-06T03:38:37.632115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:51.382210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632024+0000","last_change":"2024-06-06T03:35:13.866999+0000","last_active":"2024-06-06T03:38:37.632024+0000","last_peered":"2024-06-06T03:38:37.632024+0000","last_clean":"2024-06-06T03:38:37.632024+0000","last_became_active":"2024-06-06T03:35:13.866664+0000","last_became_peered":"2024-06-06T03:35:13.866664+0000","last_unstale":"2024-06-06T03:38:37.632024+0000","last_undegraded":"2024-06-06T03:38:37.632024+0000","last_fullsized":"2024-06-06T03:38:37.632024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:31:14.166122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.891400+0000","last_change":"2024-06-06T03:35:13.849062+0000","last_active":"2024-06-06T03:38:37.891400+0000","last_peered":"2024-06-06T03:38:37.891400+0000","last_clean":"2024-06-06T03:38:37.891400+0000","last_became_active":"2024-06-06T03:35:13.848750+0000","last_became_peered":"2024-06-06T03:35:13.848750+0000","last_unstale":"2024-06-06T03:38:37.891400+0000","last_undegraded":"2024-06-06T03:38:37.891400+0000","last_fullsized":"2024-06-06T03:38:37.891400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:55:58.084908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890931+0000","last_change":"2024-06-06T03:35:13.857601+0000","last_active":"2024-06-06T03:38:37.890931+0000","last_peered":"2024-06-06T03:38:37.890931+0000","last_clean":"2024-06-06T03:38:37.890931+0000","last_became_active":"2024-06-06T03:35:13.857270+0000","last_became_peered":"2024-06-06T03:35:13.857270+0000","last_unstale":"2024-06-06T03:38:37.890931+0000","last_undegraded":"2024-06-06T03:38:37.890931+0000","last_fullsized":"2024-06-06T03:38:37.890931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:56:29.970210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890367+0000","last_change":"2024-06-06T03:35:13.846699+0000","last_active":"2024-06-06T03:38:37.890367+0000","last_peered":"2024-06-06T03:38:37.890367+0000","last_clean":"2024-06-06T03:38:37.890367+0000","last_became_active":"2024-06-06T03:35:13.846355+0000","last_became_peered":"2024-06-06T03:35:13.846355+0000","last_unstale":"2024-06-06T03:38:37.890367+0000","last_undegraded":"2024-06-06T03:38:37.890367+0000","last_fullsized":"2024-06-06T03:38:37.890367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:50:22.537339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.632406+0000","last_change":"2024-06-06T03:35:13.854013+0000","last_active":"2024-06-06T03:38:37.632406+0000","last_peered":"2024-06-06T03:38:37.632406+0000","last_clean":"2024-06-06T03:38:37.632406+0000","last_became_active":"2024-06-06T03:35:13.853739+0000","last_became_peered":"2024-06-06T03:35:13.853739+0000","last_unstale":"2024-06-06T03:38:37.632406+0000","last_undegraded":"2024-06-06T03:38:37.632406+0000","last_fullsized":"2024-06-06T03:38:37.632406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:27:12.330943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.717814+0000","last_change":"2024-06-06T03:35:13.854251+0000","last_active":"2024-06-06T03:38:37.717814+0000","last_peered":"2024-06-06T03:38:37.717814+0000","last_clean":"2024-06-06T03:38:37.717814+0000","last_became_active":"2024-06-06T03:35:13.853915+0000","last_became_peered":"2024-06-06T03:35:13.853915+0000","last_unstale":"2024-06-06T03:38:37.717814+0000","last_undegraded":"2024-06-06T03:38:37.717814+0000","last_fullsized":"2024-06-06T03:38:37.717814+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:58:08.813316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.733345+0000","last_change":"2024-06-06T03:35:13.849998+0000","last_active":"2024-06-06T03:38:37.733345+0000","last_peered":"2024-06-06T03:38:37.733345+0000","last_clean":"2024-06-06T03:38:37.733345+0000","last_became_active":"2024-06-06T03:35:13.848789+0000","last_became_peered":"2024-06-06T03:35:13.848789+0000","last_unstale":"2024-06-06T03:38:37.733345+0000","last_undegraded":"2024-06-06T03:38:37.733345+0000","last_fullsized":"2024-06-06T03:38:37.733345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:53:39.043356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.716965+0000","last_change":"2024-06-06T03:35:13.858020+0000","last_active":"2024-06-06T03:38:37.716965+0000","last_peered":"2024-06-06T03:38:37.716965+0000","last_clean":"2024-06-06T03:38:37.716965+0000","last_became_active":"2024-06-06T03:35:13.857723+0000","last_became_peered":"2024-06-06T03:35:13.857723+0000","last_unstale":"2024-06-06T03:38:37.716965+0000","last_undegraded":"2024-06-06T03:38:37.716965+0000","last_fullsized":"2024-06-06T03:38:37.716965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:08:08.651213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.644929+0000","last_change":"2024-06-06T03:35:13.867546+0000","last_active":"2024-06-06T03:38:37.644929+0000","last_peered":"2024-06-06T03:38:37.644929+0000","last_clean":"2024-06-06T03:38:37.644929+0000","last_became_active":"2024-06-06T03:35:13.867182+0000","last_became_peered":"2024-06-06T03:35:13.867182+0000","last_unstale":"2024-06-06T03:38:37.644929+0000","last_undegraded":"2024-06-06T03:38:37.644929+0000","last_fullsized":"2024-06-06T03:38:37.644929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:25:55.831943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890895+0000","last_change":"2024-06-06T03:35:13.849080+0000","last_active":"2024-06-06T03:38:37.890895+0000","last_peered":"2024-06-06T03:38:37.890895+0000","last_clean":"2024-06-06T03:38:37.890895+0000","last_became_active":"2024-06-06T03:35:13.848476+0000","last_became_peered":"2024-06-06T03:35:13.848476+0000","last_unstale":"2024-06-06T03:38:37.890895+0000","last_undegraded":"2024-06-06T03:38:37.890895+0000","last_fullsized":"2024-06-06T03:38:37.890895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:48:37.306528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-06-06T03:38:37.890161+0000","last_change":"2024-06-06T03:35:13.848442+0000","last_active":"2024-06-06T03:38:37.890161+0000","last_peered":"2024-06-06T03:38:37.890161+0000","last_clean":"2024-06-06T03:38:37.890161+0000","last_became_active":"2024-06-06T03:35:13.848074+0000","last_became_peered":"2024-06-06T03:35:13.848074+0000","last_unstale":"2024-06-06T03:38:37.890161+0000","last_undegraded":"2024-06-06T03:38:37.890161+0000","last_fullsized":"2024-06-06T03:38:37.890161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T03:35:12.825531+0000","last_clean_scrub_stamp":"2024-06-06T03:35:12.825531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:07:00.197671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107001,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799175,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2284,"num_read_kb":1363,"num_write":510,"num_write_kb":2386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":2397525,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":387,"ondisk_log_size":387,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383738,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":489954,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645375,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28196,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716956,"statfs":{"total":95995035648,"available":95966162944,"internally_reserved":0,"allocated":1208320,"data_stored":1477088,"data_compressed":53749,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808900,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":882208,"data_compressed":22169,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070537,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":882474,"data_compressed":22169,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332174,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":1084037,"data_compressed":38547,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495699,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":489548,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":558446,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":490828,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1097728,"data_stored":559528,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":1209982,"data_compressed":37371,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920862,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":489350,"data_compressed":2880,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757337,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1130496,"data_stored":1410328,"data_compressed":53749,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35667,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331973,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":398733,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T03:38:48.093 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:38:48.344 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:48 smithi131 ceph-mon[36687]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:38:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:48 smithi131 ceph-mon[36687]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-06-06T03:38:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:48 smithi040 ceph-mon[32220]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:38:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:48 smithi040 ceph-mon[32220]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-06-06T03:38:49.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:48 smithi104 ceph-mon[36569]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T03:38:49.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:48 smithi104 ceph-mon[36569]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-06-06T03:38:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.942+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/731290477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f341073d0 msgr2=0x7f4f341077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.942+0000 7f4f3a4b5700 1 --2- 172.21.15.40:0/731290477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f341073d0 0x7f4f341077b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009990 tx=0x7f4f24009ca0 comp rx=0 tx=0).stop 2024-06-06T03:38:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.942+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/731290477 shutdown_connections 2024-06-06T03:38:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.942+0000 7f4f3a4b5700 1 --2- 172.21.15.40:0/731290477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f34101d30 0x7f4f3410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.942+0000 7f4f3a4b5700 1 --2- 172.21.15.40:0/731290477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f341073d0 0x7f4f341077b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.942+0000 7f4f3a4b5700 1 --2- 172.21.15.40:0/731290477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f34101390 0x7f4f341017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.942+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/731290477 >> 172.21.15.40:0/731290477 conn(0x7f4f340782c0 msgr2=0x7f4f340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.943+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/731290477 shutdown_connections 2024-06-06T03:38:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.943+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/731290477 wait complete. 2024-06-06T03:38:49.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.943+0000 7f4f3a4b5700 1 Processor -- start 2024-06-06T03:38:49.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.943+0000 7f4f3a4b5700 1 -- start start 2024-06-06T03:38:49.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.943+0000 7f4f3a4b5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f34101390 0x7f4f3419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:49.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.943+0000 7f4f3a4b5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f34101d30 0x7f4f3419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:49.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.943+0000 7f4f3a4b5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f341073d0 0x7f4f3419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:49.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.943+0000 7f4f3a4b5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4f34111630 con 0x7f4f341073d0 2024-06-06T03:38:49.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f3a4b5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4f341114b0 con 0x7f4f34101d30 2024-06-06T03:38:49.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f3a4b5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f341117b0 con 0x7f4f34101390 2024-06-06T03:38:49.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f38a52700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f341073d0 0x7f4f3419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:49.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f38a52700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f341073d0 0x7f4f3419f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45002/0 (socket says 172.21.15.40:45002) 2024-06-06T03:38:49.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f38a52700 1 -- 172.21.15.40:0/1961232458 learned_addr learned my addr 172.21.15.40:0/1961232458 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:49.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f33fff700 1 --2- 172.21.15.40:0/1961232458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f34101390 0x7f4f3419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f2bfff700 1 --2- 172.21.15.40:0/1961232458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f34101d30 0x7f4f3419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f38a52700 1 -- 172.21.15.40:0/1961232458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f34101390 msgr2=0x7f4f3419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f38a52700 1 --2- 172.21.15.40:0/1961232458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f34101390 0x7f4f3419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f38a52700 1 -- 172.21.15.40:0/1961232458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f34101d30 msgr2=0x7f4f3419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f38a52700 1 --2- 172.21.15.40:0/1961232458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f34101d30 0x7f4f3419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f38a52700 1 -- 172.21.15.40:0/1961232458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3419fe10 con 0x7f4f341073d0 2024-06-06T03:38:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f38a52700 1 --2- 172.21.15.40:0/1961232458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f341073d0 0x7f4f3419f6d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00ef30 tx=0x7f4f2c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f31ffb700 1 -- 172.21.15.40:0/1961232458 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c0092e0 con 0x7f4f341073d0 2024-06-06T03:38:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f31ffb700 1 -- 172.21.15.40:0/1961232458 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f2c004bb0 con 0x7f4f341073d0 2024-06-06T03:38:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/1961232458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f341a00a0 con 0x7f4f341073d0 2024-06-06T03:38:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f31ffb700 1 -- 172.21.15.40:0/1961232458 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c004280 con 0x7f4f341073d0 2024-06-06T03:38:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.944+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/1961232458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f341a7980 con 0x7f4f341073d0 2024-06-06T03:38:49.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.945+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/1961232458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f34069180 con 0x7f4f341073d0 2024-06-06T03:38:49.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.947+0000 7f4f31ffb700 1 -- 172.21.15.40:0/1961232458 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f4f2c01b030 con 0x7f4f341073d0 2024-06-06T03:38:49.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.947+0000 7f4f31ffb700 1 --2- 172.21.15.40:0/1961232458 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4f200a2e20 0x7f4f200a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:49.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.947+0000 7f4f33fff700 1 --2- 172.21.15.40:0/1961232458 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4f200a2e20 0x7f4f200a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:49.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.947+0000 7f4f31ffb700 1 -- 172.21.15.40:0/1961232458 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f4f2c0cf950 con 0x7f4f341073d0 2024-06-06T03:38:49.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.948+0000 7f4f33fff700 1 --2- 172.21.15.40:0/1961232458 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4f200a2e20 0x7f4f200a52e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400b2e0 tx=0x7f4f24005960 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:49.951+0000 7f4f31ffb700 1 -- 172.21.15.40:0/1961232458 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4f2c022020 con 0x7f4f341073d0 2024-06-06T03:38:50.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:50.202+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/1961232458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f341a7b50 con 0x7f4f341073d0 2024-06-06T03:38:50.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:50.202+0000 7f4f31ffb700 1 -- 172.21.15.40:0/1961232458 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11933 (secure 0 0 0) 0x7f4f2c010040 con 0x7f4f341073d0 2024-06-06T03:38:50.202 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:38:50.203 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:38:35.323771+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24443,24425,24398,24403]},"id":1}]} 2024-06-06T03:38:50.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:50.204+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/1961232458 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4f200a2e20 msgr2=0x7f4f200a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:50.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:50.204+0000 7f4f3a4b5700 1 --2- 172.21.15.40:0/1961232458 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4f200a2e20 0x7f4f200a52e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400b2e0 tx=0x7f4f24005960 comp rx=0 tx=0).stop 2024-06-06T03:38:50.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:50.204+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/1961232458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f341073d0 msgr2=0x7f4f3419f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:50.204+0000 7f4f3a4b5700 1 --2- 172.21.15.40:0/1961232458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f341073d0 0x7f4f3419f6d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00ef30 tx=0x7f4f2c00c3a0 comp rx=0 tx=0).stop 2024-06-06T03:38:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:50.204+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/1961232458 shutdown_connections 2024-06-06T03:38:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:50.204+0000 7f4f3a4b5700 1 --2- 172.21.15.40:0/1961232458 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4f200a2e20 0x7f4f200a52e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:50.204+0000 7f4f3a4b5700 1 --2- 172.21.15.40:0/1961232458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f34101390 0x7f4f3419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:50.204+0000 7f4f3a4b5700 1 --2- 172.21.15.40:0/1961232458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f341073d0 0x7f4f3419f6d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:50.204+0000 7f4f3a4b5700 1 --2- 172.21.15.40:0/1961232458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f34101d30 0x7f4f3419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:50.204+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/1961232458 >> 172.21.15.40:0/1961232458 conn(0x7f4f340782c0 msgr2=0x7f4f340ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:50.205+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/1961232458 shutdown_connections 2024-06-06T03:38:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:50.205+0000 7f4f3a4b5700 1 -- 172.21.15.40:0/1961232458 wait complete. 2024-06-06T03:38:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 29 2024-06-06T03:38:50.826 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:38:51.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:50 smithi040 ceph-mon[32220]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-06-06T03:38:51.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:50 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1961232458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:38:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:51.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:50 smithi131 ceph-mon[36687]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-06-06T03:38:51.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:50 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1961232458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:38:51.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:50 smithi104 ceph-mon[36569]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-06-06T03:38:51.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:50 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1961232458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:38:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.582+0000 7f291481e700 1 -- 172.21.15.40:0/3850154852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2910102770 msgr2=0x7f2910102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.582+0000 7f291481e700 1 --2- 172.21.15.40:0/3850154852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2910102770 0x7f2910102b50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f290000b0d0 tx=0x7f290000b3e0 comp rx=0 tx=0).stop 2024-06-06T03:38:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.582+0000 7f291481e700 1 -- 172.21.15.40:0/3850154852 shutdown_connections 2024-06-06T03:38:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.582+0000 7f291481e700 1 --2- 172.21.15.40:0/3850154852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2910107860 0x7f291010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.582+0000 7f291481e700 1 --2- 172.21.15.40:0/3850154852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2910102770 0x7f2910102b50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.582+0000 7f291481e700 1 --2- 172.21.15.40:0/3850154852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2910103120 0x7f2910107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.582+0000 7f291481e700 1 -- 172.21.15.40:0/3850154852 >> 172.21.15.40:0/3850154852 conn(0x7f2910079600 msgr2=0x7f2910079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.582+0000 7f291481e700 1 -- 172.21.15.40:0/3850154852 shutdown_connections 2024-06-06T03:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.582+0000 7f291481e700 1 -- 172.21.15.40:0/3850154852 wait complete. 2024-06-06T03:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.583+0000 7f291481e700 1 Processor -- start 2024-06-06T03:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.583+0000 7f291481e700 1 -- start start 2024-06-06T03:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.583+0000 7f291481e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2910102770 0x7f2910074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.583+0000 7f291481e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2910103120 0x7f2910074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.583+0000 7f291481e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2910107860 0x7f2910075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.583+0000 7f291481e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2910112780 con 0x7f2910103120 2024-06-06T03:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.583+0000 7f291481e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2910112600 con 0x7f2910102770 2024-06-06T03:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.583+0000 7f291481e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2910112900 con 0x7f2910107860 2024-06-06T03:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.584+0000 7f290dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2910103120 0x7f2910074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.584+0000 7f290dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2910103120 0x7f2910074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45014/0 (socket says 172.21.15.40:45014) 2024-06-06T03:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.584+0000 7f290dd9b700 1 -- 172.21.15.40:0/252524084 learned_addr learned my addr 172.21.15.40:0/252524084 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.584+0000 7f290e59c700 1 --2- 172.21.15.40:0/252524084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2910102770 0x7f2910074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.584+0000 7f290ed9d700 1 --2- 172.21.15.40:0/252524084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2910107860 0x7f2910075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.584+0000 7f290dd9b700 1 -- 172.21.15.40:0/252524084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2910107860 msgr2=0x7f2910075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.584+0000 7f290dd9b700 1 --2- 172.21.15.40:0/252524084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2910107860 0x7f2910075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.584+0000 7f290dd9b700 1 -- 172.21.15.40:0/252524084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2910102770 msgr2=0x7f2910074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.584+0000 7f290dd9b700 1 --2- 172.21.15.40:0/252524084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2910102770 0x7f2910074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.584+0000 7f290dd9b700 1 -- 172.21.15.40:0/252524084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29101a70e0 con 0x7f2910103120 2024-06-06T03:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.584+0000 7f290dd9b700 1 --2- 172.21.15.40:0/252524084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2910103120 0x7f2910074cd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f28f800d950 tx=0x7f28f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.584+0000 7f29077fe700 1 -- 172.21.15.40:0/252524084 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8016070 con 0x7f2910103120 2024-06-06T03:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.584+0000 7f29077fe700 1 -- 172.21.15.40:0/252524084 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f8009e20 con 0x7f2910103120 2024-06-06T03:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.584+0000 7f291481e700 1 -- 172.21.15.40:0/252524084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29101a73d0 con 0x7f2910103120 2024-06-06T03:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.585+0000 7f29077fe700 1 -- 172.21.15.40:0/252524084 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f800f620 con 0x7f2910103120 2024-06-06T03:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.585+0000 7f291481e700 1 -- 172.21.15.40:0/252524084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29101a7910 con 0x7f2910103120 2024-06-06T03:38:52.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.586+0000 7f291481e700 1 -- 172.21.15.40:0/252524084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291010aaa0 con 0x7f2910103120 2024-06-06T03:38:52.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.587+0000 7f29077fe700 1 -- 172.21.15.40:0/252524084 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f28f8019070 con 0x7f2910103120 2024-06-06T03:38:52.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.587+0000 7f29077fe700 1 --2- 172.21.15.40:0/252524084 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f28fc0a2e20 0x7f28fc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:52.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.587+0000 7f290e59c700 1 --2- 172.21.15.40:0/252524084 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f28fc0a2e20 0x7f28fc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:52.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.587+0000 7f29077fe700 1 -- 172.21.15.40:0/252524084 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f28f80cf4b0 con 0x7f2910103120 2024-06-06T03:38:52.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.588+0000 7f290e59c700 1 --2- 172.21.15.40:0/252524084 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f28fc0a2e20 0x7f28fc0a52e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2900009fd0 tx=0x7f290000b040 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:52.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.592+0000 7f29077fe700 1 -- 172.21.15.40:0/252524084 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f28f809bff0 con 0x7f2910103120 2024-06-06T03:38:52.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.850+0000 7f291481e700 1 -- 172.21.15.40:0/252524084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2910075cc0 con 0x7f2910103120 2024-06-06T03:38:52.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.851+0000 7f29077fe700 1 -- 172.21.15.40:0/252524084 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11933 (secure 0 0 0) 0x7f28f809b8f0 con 0x7f2910103120 2024-06-06T03:38:52.851 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:38:52.851 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:38:35.323771+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24443,24425,24398,24403]},"id":1}]} 2024-06-06T03:38:52.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.853+0000 7f291481e700 1 -- 172.21.15.40:0/252524084 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f28fc0a2e20 msgr2=0x7f28fc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:52.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.853+0000 7f291481e700 1 --2- 172.21.15.40:0/252524084 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f28fc0a2e20 0x7f28fc0a52e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2900009fd0 tx=0x7f290000b040 comp rx=0 tx=0).stop 2024-06-06T03:38:52.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.853+0000 7f291481e700 1 -- 172.21.15.40:0/252524084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2910103120 msgr2=0x7f2910074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:52.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.853+0000 7f291481e700 1 --2- 172.21.15.40:0/252524084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2910103120 0x7f2910074cd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f28f800d950 tx=0x7f28f800dc60 comp rx=0 tx=0).stop 2024-06-06T03:38:52.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.853+0000 7f291481e700 1 -- 172.21.15.40:0/252524084 shutdown_connections 2024-06-06T03:38:52.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.853+0000 7f291481e700 1 --2- 172.21.15.40:0/252524084 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f28fc0a2e20 0x7f28fc0a52e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:52.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.853+0000 7f291481e700 1 --2- 172.21.15.40:0/252524084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2910107860 0x7f2910075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:52.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.853+0000 7f291481e700 1 --2- 172.21.15.40:0/252524084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2910102770 0x7f2910074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:52.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.853+0000 7f291481e700 1 --2- 172.21.15.40:0/252524084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2910103120 0x7f2910074cd0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:52.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.853+0000 7f291481e700 1 -- 172.21.15.40:0/252524084 >> 172.21.15.40:0/252524084 conn(0x7f2910079600 msgr2=0x7f2910100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:52.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.853+0000 7f291481e700 1 -- 172.21.15.40:0/252524084 shutdown_connections 2024-06-06T03:38:52.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:52.853+0000 7f291481e700 1 -- 172.21.15.40:0/252524084 wait complete. 2024-06-06T03:38:52.854 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 29 2024-06-06T03:38:53.181 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:52 smithi040 ceph-mon[32220]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-06-06T03:38:53.181 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:52 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/252524084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:38:53.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:52 smithi131 ceph-mon[36687]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-06-06T03:38:53.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:52 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/252524084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:38:53.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:52 smithi104 ceph-mon[36569]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-06-06T03:38:53.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:52 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/252524084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:38:53.464 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T03:38:53.720 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:55.154 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:54 smithi040 ceph-mon[32220]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 52 op/s 2024-06-06T03:38:55.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:54 smithi131 ceph-mon[36687]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 52 op/s 2024-06-06T03:38:55.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:54 smithi104 ceph-mon[36569]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 52 op/s 2024-06-06T03:38:55.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.411+0000 7f3eb54db700 1 -- 172.21.15.40:0/3398894773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb0101d40 msgr2=0x7f3eb010e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.411+0000 7f3eb54db700 1 --2- 172.21.15.40:0/3398894773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb0101d40 0x7f3eb010e230 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009990 tx=0x7f3ea8009ca0 comp rx=0 tx=0).stop 2024-06-06T03:38:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.411+0000 7f3eb54db700 1 -- 172.21.15.40:0/3398894773 shutdown_connections 2024-06-06T03:38:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.411+0000 7f3eb54db700 1 --2- 172.21.15.40:0/3398894773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3eb01073f0 0x7f3eb01077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.411+0000 7f3eb54db700 1 --2- 172.21.15.40:0/3398894773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb0101d40 0x7f3eb010e230 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.411+0000 7f3eb54db700 1 --2- 172.21.15.40:0/3398894773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eb0101380 0x7f3eb0101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.411+0000 7f3eb54db700 1 -- 172.21.15.40:0/3398894773 >> 172.21.15.40:0/3398894773 conn(0x7f3eb0078300 msgr2=0x7f3eb0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.411+0000 7f3eb54db700 1 -- 172.21.15.40:0/3398894773 shutdown_connections 2024-06-06T03:38:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.411+0000 7f3eb54db700 1 -- 172.21.15.40:0/3398894773 wait complete. 2024-06-06T03:38:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.412+0000 7f3eb54db700 1 Processor -- start 2024-06-06T03:38:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.412+0000 7f3eb54db700 1 -- start start 2024-06-06T03:38:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.412+0000 7f3eb54db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb0101380 0x7f3eb010ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.412+0000 7f3eb54db700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3eb0101d40 0x7f3eb010f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.412+0000 7f3eb54db700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eb01073f0 0x7f3eb010f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.412+0000 7f3eb54db700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb01127e0 con 0x7f3eb0101380 2024-06-06T03:38:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.412+0000 7f3eb54db700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb0112660 con 0x7f3eb01073f0 2024-06-06T03:38:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.412+0000 7f3eb54db700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb0112960 con 0x7f3eb0101d40 2024-06-06T03:38:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3eaeffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb0101380 0x7f3eb010ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3eaeffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb0101380 0x7f3eb010ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45032/0 (socket says 172.21.15.40:45032) 2024-06-06T03:38:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3eaeffd700 1 -- 172.21.15.40:0/3753846039 learned_addr learned my addr 172.21.15.40:0/3753846039 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:38:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3eae7fc700 1 --2- 172.21.15.40:0/3753846039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3eb0101d40 0x7f3eb010f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3eaf7fe700 1 --2- 172.21.15.40:0/3753846039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eb01073f0 0x7f3eb010f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3eaeffd700 1 -- 172.21.15.40:0/3753846039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3eb0101d40 msgr2=0x7f3eb010f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3eaeffd700 1 --2- 172.21.15.40:0/3753846039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3eb0101d40 0x7f3eb010f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3eaeffd700 1 -- 172.21.15.40:0/3753846039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eb01073f0 msgr2=0x7f3eb010f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3eaeffd700 1 --2- 172.21.15.40:0/3753846039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eb01073f0 0x7f3eb010f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3eaeffd700 1 -- 172.21.15.40:0/3753846039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb01a7100 con 0x7f3eb0101380 2024-06-06T03:38:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3eaeffd700 1 --2- 172.21.15.40:0/3753846039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb0101380 0x7f3eb010ecc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00060b0 tx=0x7f3ea000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3ea7fff700 1 -- 172.21.15.40:0/3753846039 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea001a070 con 0x7f3eb0101380 2024-06-06T03:38:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3ea7fff700 1 -- 172.21.15.40:0/3753846039 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea000dd20 con 0x7f3eb0101380 2024-06-06T03:38:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3ea7fff700 1 -- 172.21.15.40:0/3753846039 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0014720 con 0x7f3eb0101380 2024-06-06T03:38:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.413+0000 7f3eb54db700 1 -- 172.21.15.40:0/3753846039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb01a73f0 con 0x7f3eb0101380 2024-06-06T03:38:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.414+0000 7f3eb54db700 1 -- 172.21.15.40:0/3753846039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb01a78e0 con 0x7f3eb0101380 2024-06-06T03:38:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.415+0000 7f3eb54db700 1 -- 172.21.15.40:0/3753846039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb0069180 con 0x7f3eb0101380 2024-06-06T03:38:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.416+0000 7f3ea7fff700 1 -- 172.21.15.40:0/3753846039 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f3ea001d070 con 0x7f3eb0101380 2024-06-06T03:38:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.417+0000 7f3ea7fff700 1 --2- 172.21.15.40:0/3753846039 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f3e9c0a2e20 0x7f3e9c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:38:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.417+0000 7f3eae7fc700 1 --2- 172.21.15.40:0/3753846039 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f3e9c0a2e20 0x7f3e9c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:38:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.417+0000 7f3ea7fff700 1 -- 172.21.15.40:0/3753846039 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f3ea00cf1b0 con 0x7f3eb0101380 2024-06-06T03:38:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.417+0000 7f3eae7fc700 1 --2- 172.21.15.40:0/3753846039 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f3e9c0a2e20 0x7f3e9c0a52e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0106300 tx=0x7f3e9800a380 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:38:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.421+0000 7f3ea7fff700 1 -- 172.21.15.40:0/3753846039 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3ea0010070 con 0x7f3eb0101380 2024-06-06T03:38:55.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.626+0000 7f3eb54db700 1 -- 172.21.15.40:0/3753846039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3eb002de60 con 0x7f3eb0101380 2024-06-06T03:38:55.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.627+0000 7f3ea7fff700 1 -- 172.21.15.40:0/3753846039 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22909 (secure 0 0 0) 0x7f3ea009bcf0 con 0x7f3eb0101380 2024-06-06T03:38:55.627 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:38:55.627 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":91,"fsid":"77c25d68-23b4-11ef-bc9b-c7b262605968","created":"2024-06-06T03:26:53.165489+0000","modified":"2024-06-06T03:38:37.622522+0000","last_up_change":"2024-06-06T03:34:07.114448+0000","last_in_change":"2024-06-06T03:33:46.152948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T03:30:14.102785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T03:35:08.521025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T03:35:12.369035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63c0a320-a2f2-4b84-87e7-68501d335285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6803","nonce":3453526793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6805","nonce":3453526793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6809","nonce":3453526793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6807","nonce":3453526793}]},"public_addr":"172.21.15.40:6803/3453526793","cluster_addr":"172.21.15.40:6805/3453526793","heartbeat_back_addr":"172.21.15.40:6809/3453526793","heartbeat_front_addr":"172.21.15.40:6807/3453526793","state":["exists","up"]},{"osd":1,"uuid":"70a31534-06a1-4ffa-a54d-577d45bccf98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6801","nonce":405854252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6803","nonce":405854252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6807","nonce":405854252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6805","nonce":405854252}]},"public_addr":"172.21.15.104:6801/405854252","cluster_addr":"172.21.15.104:6803/405854252","heartbeat_back_addr":"172.21.15.104:6807/405854252","heartbeat_front_addr":"172.21.15.104:6805/405854252","state":["exists","up"]},{"osd":2,"uuid":"f5a89f43-e533-45f0-a777-bd3e9bfa8072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6801","nonce":3919221141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6803","nonce":3919221141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6807","nonce":3919221141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6805","nonce":3919221141}]},"public_addr":"172.21.15.131:6801/3919221141","cluster_addr":"172.21.15.131:6803/3919221141","heartbeat_back_addr":"172.21.15.131:6807/3919221141","heartbeat_front_addr":"172.21.15.131:6805/3919221141","state":["exists","up"]},{"osd":3,"uuid":"4df19390-6378-4346-a2be-29bce1dd2380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6811","nonce":2008541522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6813","nonce":2008541522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6817","nonce":2008541522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6815","nonce":2008541522}]},"public_addr":"172.21.15.40:6811/2008541522","cluster_addr":"172.21.15.40:6813/2008541522","heartbeat_back_addr":"172.21.15.40:6817/2008541522","heartbeat_front_addr":"172.21.15.40:6815/2008541522","state":["exists","up"]},{"osd":4,"uuid":"508de408-1cb1-4423-8199-db6bde9034f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6809","nonce":1153394847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6811","nonce":1153394847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6815","nonce":1153394847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6813","nonce":1153394847}]},"public_addr":"172.21.15.104:6809/1153394847","cluster_addr":"172.21.15.104:6811/1153394847","heartbeat_back_addr":"172.21.15.104:6815/1153394847","heartbeat_front_addr":"172.21.15.104:6813/1153394847","state":["exists","up"]},{"osd":5,"uuid":"a8c0cec9-a8ad-4a5d-800b-8b891ca6d836","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6809","nonce":1051933303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6811","nonce":1051933303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6815","nonce":1051933303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6813","nonce":1051933303}]},"public_addr":"172.21.15.131:6809/1051933303","cluster_addr":"172.21.15.131:6811/1051933303","heartbeat_back_addr":"172.21.15.131:6815/1051933303","heartbeat_front_addr":"172.21.15.131:6813/1051933303","state":["exists","up"]},{"osd":6,"uuid":"5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6819","nonce":2670922011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6821","nonce":2670922011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6825","nonce":2670922011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6823","nonce":2670922011}]},"public_addr":"172.21.15.40:6819/2670922011","cluster_addr":"172.21.15.40:6821/2670922011","heartbeat_back_addr":"172.21.15.40:6825/2670922011","heartbeat_front_addr":"172.21.15.40:6823/2670922011","state":["exists","up"]},{"osd":7,"uuid":"bfc536bb-6ae4-459e-afd9-8197c7420bd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6817","nonce":2353571051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6819","nonce":2353571051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6823","nonce":2353571051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6821","nonce":2353571051}]},"public_addr":"172.21.15.104:6817/2353571051","cluster_addr":"172.21.15.104:6819/2353571051","heartbeat_back_addr":"172.21.15.104:6823/2353571051","heartbeat_front_addr":"172.21.15.104:6821/2353571051","state":["exists","up"]},{"osd":8,"uuid":"1fe214f2-5ea3-4fc5-83bf-d9bccafe1529","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6817","nonce":1364533794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6819","nonce":1364533794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6823","nonce":1364533794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6821","nonce":1364533794}]},"public_addr":"172.21.15.131:6817/1364533794","cluster_addr":"172.21.15.131:6819/1364533794","heartbeat_back_addr":"172.21.15.131:6823/1364533794","heartbeat_front_addr":"172.21.15.131:6821/1364533794","state":["exists","up"]},{"osd":9,"uuid":"4b2b3555-6870-4ead-be50-b635799265a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6827","nonce":2604133190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6829","nonce":2604133190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6833","nonce":2604133190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6831","nonce":2604133190}]},"public_addr":"172.21.15.40:6827/2604133190","cluster_addr":"172.21.15.40:6829/2604133190","heartbeat_back_addr":"172.21.15.40:6833/2604133190","heartbeat_front_addr":"172.21.15.40:6831/2604133190","state":["exists","up"]},{"osd":10,"uuid":"70049e24-94ce-458e-97e3-dbcf9de05444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6825","nonce":2231164832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6827","nonce":2231164832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6831","nonce":2231164832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6829","nonce":2231164832}]},"public_addr":"172.21.15.104:6825/2231164832","cluster_addr":"172.21.15.104:6827/2231164832","heartbeat_back_addr":"172.21.15.104:6831/2231164832","heartbeat_front_addr":"172.21.15.104:6829/2231164832","state":["exists","up"]},{"osd":11,"uuid":"80447548-1fbe-4892-ac13-eb4bb6acf10d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6825","nonce":4128185670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6827","nonce":4128185670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6831","nonce":4128185670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6829","nonce":4128185670}]},"public_addr":"172.21.15.131:6825/4128185670","cluster_addr":"172.21.15.131:6827/4128185670","heartbeat_back_addr":"172.21.15.131:6831/4128185670","heartbeat_front_addr":"172.21.15.131:6829/4128185670","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:23.289784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:47.271677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:10.866792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:36.781924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:01.995848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:26.904606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:52.997438+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:19.221567+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:44.543977+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:11.948545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:38.402943+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:34:04.712200+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/889893607":"2024-06-07T03:38:37.622492+0000","172.21.15.40:6801/869712339":"2024-06-07T03:38:37.622492+0000","172.21.15.40:0/1294024108":"2024-06-07T03:38:37.622492+0000","172.21.15.40:0/23942578":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/1915158516":"2024-06-07T03:38:37.622492+0000","172.21.15.40:6800/869712339":"2024-06-07T03:38:37.622492+0000","172.21.15.40:0/2234980855":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6801/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6800/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2301695357":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/1676880727":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/2237379034":"2024-06-07T03:38:37.622492+0000","172.21.15.40:6801/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6800/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/1597665640":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/513375601":"2024-06-07T03:27:19.799499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T03:38:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.630+0000 7f3eb54db700 1 -- 172.21.15.40:0/3753846039 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f3e9c0a2e20 msgr2=0x7f3e9c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.630+0000 7f3eb54db700 1 --2- 172.21.15.40:0/3753846039 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f3e9c0a2e20 0x7f3e9c0a52e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0106300 tx=0x7f3e9800a380 comp rx=0 tx=0).stop 2024-06-06T03:38:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.630+0000 7f3eb54db700 1 -- 172.21.15.40:0/3753846039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb0101380 msgr2=0x7f3eb010ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:38:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.630+0000 7f3eb54db700 1 --2- 172.21.15.40:0/3753846039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb0101380 0x7f3eb010ecc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00060b0 tx=0x7f3ea000d3b0 comp rx=0 tx=0).stop 2024-06-06T03:38:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.630+0000 7f3eb54db700 1 -- 172.21.15.40:0/3753846039 shutdown_connections 2024-06-06T03:38:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.630+0000 7f3eb54db700 1 --2- 172.21.15.40:0/3753846039 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f3e9c0a2e20 0x7f3e9c0a52e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.630+0000 7f3eb54db700 1 --2- 172.21.15.40:0/3753846039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eb01073f0 0x7f3eb010f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.630+0000 7f3eb54db700 1 --2- 172.21.15.40:0/3753846039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb0101380 0x7f3eb010ecc0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.630+0000 7f3eb54db700 1 --2- 172.21.15.40:0/3753846039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3eb0101d40 0x7f3eb010f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:38:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.630+0000 7f3eb54db700 1 -- 172.21.15.40:0/3753846039 >> 172.21.15.40:0/3753846039 conn(0x7f3eb0078300 msgr2=0x7f3eb0100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:38:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.630+0000 7f3eb54db700 1 -- 172.21.15.40:0/3753846039 shutdown_connections 2024-06-06T03:38:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:38:55.630+0000 7f3eb54db700 1 -- 172.21.15.40:0/3753846039 wait complete. 2024-06-06T03:38:56.183 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:55 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3753846039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:38:56.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-06-06T03:38:56.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:55 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3753846039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:38:56.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:55 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3753846039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:38:56.463 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:57.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:56 smithi131 ceph-mon[36687]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:38:57.326 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:56 smithi040 ceph-mon[32220]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:38:57.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:56 smithi104 ceph-mon[36569]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:38:58.859 INFO:teuthology.orchestra.run.smithi040.stdout:{"run_dir":"/var/run/ceph"} 2024-06-06T03:38:58.859 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-06-06T03:38:58.859 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 5050 2024-06-06T03:38:58.859 DEBUG:tasks.quiescer:Yielding 2024-06-06T03:38:58.860 INFO:teuthology.run_tasks:Running task workunit... 2024-06-06T03:38:58.873 INFO:tasks.workunit:Pulling workunits from ref 3d746ca5346f34eef7e826a1eb71e59e02b682ae 2024-06-06T03:38:58.873 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-06T03:38:58.873 INFO:tasks.workunit:timeout=3h 2024-06-06T03:38:58.873 INFO:tasks.workunit:cleanup=True 2024-06-06T03:38:58.874 DEBUG:teuthology.orchestra.run.smithi040:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-06T03:38:58.874 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:38:58.892 INFO:teuthology.orchestra.run.smithi040.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-06T03:38:58.892 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-06-06T03:38:58.893 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 6bh/107d Inode: 1099511628283 Links: 2 2024-06-06T03:38:58.893 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T03:38:58.893 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-06-06T03:38:58.893 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-06-06 03:35:51.052794137 +0000 2024-06-06T03:38:58.893 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-06-06 03:35:51.052794137 +0000 2024-06-06T03:38:58.893 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-06-06 03:38:34.615521404 +0000 2024-06-06T03:38:58.893 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-06-06 03:35:51.052794137 +0000 2024-06-06T03:38:58.893 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-06T03:38:58.893 DEBUG:teuthology.orchestra.run.smithi040:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-06T03:38:58.964 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3d746ca5346f34eef7e826a1eb71e59e02b682ae 2024-06-06T03:38:59.021 INFO:tasks.workunit.client.0.smithi040.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-06T03:38:59.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:38:58 smithi040 ceph-mon[32220]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:38:59.130 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:38:59.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:38:58 smithi131 ceph-mon[36687]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:38:59.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:38:58 smithi104 ceph-mon[36569]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:39:00.426 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:39:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:39:00.128+0000 7f67c0a81700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:39:00.426 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:39:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:39:00.128+0000 7f67c0a81700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:39:00.426 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:39:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:39:00.128+0000 7f67c0a81700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:39:00.426 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:39:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:39:00.128+0000 7f67c0a81700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:39:00.427 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:39:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:39:00.128+0000 7f67c0a81700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T03:39:00.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.808+0000 7fd8bf82d700 1 -- 172.21.15.40:0/4081018458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b8102ea0 msgr2=0x7fd8b810f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:00.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.808+0000 7fd8bf82d700 1 --2- 172.21.15.40:0/4081018458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b8102ea0 0x7fd8b810f380 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000b0a0 tx=0x7fd8b000b3b0 comp rx=0 tx=0).stop 2024-06-06T03:39:00.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.808+0000 7fd8bf82d700 1 -- 172.21.15.40:0/4081018458 shutdown_connections 2024-06-06T03:39:00.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.808+0000 7fd8bf82d700 1 --2- 172.21.15.40:0/4081018458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8b81085f0 0x7fd8b81089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.808+0000 7fd8bf82d700 1 --2- 172.21.15.40:0/4081018458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b8102ea0 0x7fd8b810f380 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.808+0000 7fd8bf82d700 1 --2- 172.21.15.40:0/4081018458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8b81024e0 0x7fd8b8102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.808+0000 7fd8bf82d700 1 -- 172.21.15.40:0/4081018458 >> 172.21.15.40:0/4081018458 conn(0x7fd8b8079640 msgr2=0x7fd8b8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.808+0000 7fd8bf82d700 1 -- 172.21.15.40:0/4081018458 shutdown_connections 2024-06-06T03:39:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.808+0000 7fd8bf82d700 1 -- 172.21.15.40:0/4081018458 wait complete. 2024-06-06T03:39:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.808+0000 7fd8bf82d700 1 Processor -- start 2024-06-06T03:39:00.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bf82d700 1 -- start start 2024-06-06T03:39:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bf82d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b81024e0 0x7fd8b8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bf82d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8b8102ea0 0x7fd8b8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bf82d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8b81085f0 0x7fd8b8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bf82d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b81127b0 con 0x7fd8b81024e0 2024-06-06T03:39:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bf82d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b8112630 con 0x7fd8b81085f0 2024-06-06T03:39:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bf82d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b8112930 con 0x7fd8b8102ea0 2024-06-06T03:39:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bd5c9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b81024e0 0x7fd8b8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bd5c9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b81024e0 0x7fd8b8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44630/0 (socket says 172.21.15.40:44630) 2024-06-06T03:39:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bd5c9700 1 -- 172.21.15.40:0/3332315709 learned_addr learned my addr 172.21.15.40:0/3332315709 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bcdc8700 1 --2- 172.21.15.40:0/3332315709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8b8102ea0 0x7fd8b8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:00.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bddca700 1 --2- 172.21.15.40:0/3332315709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8b81085f0 0x7fd8b8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:00.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bd5c9700 1 -- 172.21.15.40:0/3332315709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8b8102ea0 msgr2=0x7fd8b8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:00.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bd5c9700 1 --2- 172.21.15.40:0/3332315709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8b8102ea0 0x7fd8b8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:00.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bd5c9700 1 -- 172.21.15.40:0/3332315709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8b81085f0 msgr2=0x7fd8b8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:00.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bd5c9700 1 --2- 172.21.15.40:0/3332315709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8b81085f0 0x7fd8b8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:00.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bd5c9700 1 -- 172.21.15.40:0/3332315709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b81a70a0 con 0x7fd8b81024e0 2024-06-06T03:39:00.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.809+0000 7fd8bd5c9700 1 --2- 172.21.15.40:0/3332315709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b81024e0 0x7fd8b8074790 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400d980 tx=0x7fd8b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:00.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.810+0000 7fd8ae7fc700 1 -- 172.21.15.40:0/3332315709 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4017070 con 0x7fd8b81024e0 2024-06-06T03:39:00.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.810+0000 7fd8ae7fc700 1 -- 172.21.15.40:0/3332315709 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8b4005020 con 0x7fd8b81024e0 2024-06-06T03:39:00.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.810+0000 7fd8bf82d700 1 -- 172.21.15.40:0/3332315709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b81a7390 con 0x7fd8b81024e0 2024-06-06T03:39:00.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.810+0000 7fd8ae7fc700 1 -- 172.21.15.40:0/3332315709 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b400fbf0 con 0x7fd8b81024e0 2024-06-06T03:39:00.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.810+0000 7fd8bf82d700 1 -- 172.21.15.40:0/3332315709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b81a78d0 con 0x7fd8b81024e0 2024-06-06T03:39:00.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.811+0000 7fd8bf82d700 1 -- 172.21.15.40:0/3332315709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b8069180 con 0x7fd8b81024e0 2024-06-06T03:39:00.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.812+0000 7fd8ae7fc700 1 -- 172.21.15.40:0/3332315709 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fd8b40051d0 con 0x7fd8b81024e0 2024-06-06T03:39:00.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.813+0000 7fd8ae7fc700 1 --2- 172.21.15.40:0/3332315709 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fd8a40a2ea0 0x7fd8a40a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:00.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.813+0000 7fd8bcdc8700 1 --2- 172.21.15.40:0/3332315709 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fd8a40a2ea0 0x7fd8a40a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:00.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.813+0000 7fd8ae7fc700 1 -- 172.21.15.40:0/3332315709 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fd8b40cf6a0 con 0x7fd8b81024e0 2024-06-06T03:39:00.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.813+0000 7fd8bcdc8700 1 --2- 172.21.15.40:0/3332315709 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fd8a40a2ea0 0x7fd8a40a5360 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8079ad0 tx=0x7fd8a800b410 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:00.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:00.816+0000 7fd8ae7fc700 1 -- 172.21.15.40:0/3332315709 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd8b4020020 con 0x7fd8b81024e0 2024-06-06T03:39:01.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:01.073+0000 7fd8bf82d700 1 -- 172.21.15.40:0/3332315709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8b802dca0 con 0x7fd8b81024e0 2024-06-06T03:39:01.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:01.074+0000 7fd8ae7fc700 1 -- 172.21.15.40:0/3332315709 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11933 (secure 0 0 0) 0x7fd8b409c200 con 0x7fd8b81024e0 2024-06-06T03:39:01.073 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:39:01.073 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:38:35.323771+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24443,24425,24398,24403]},"id":1}]} 2024-06-06T03:39:01.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:01.076+0000 7fd8bf82d700 1 -- 172.21.15.40:0/3332315709 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fd8a40a2ea0 msgr2=0x7fd8a40a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:01.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:01.076+0000 7fd8bf82d700 1 --2- 172.21.15.40:0/3332315709 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fd8a40a2ea0 0x7fd8a40a5360 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8079ad0 tx=0x7fd8a800b410 comp rx=0 tx=0).stop 2024-06-06T03:39:01.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:01.076+0000 7fd8bf82d700 1 -- 172.21.15.40:0/3332315709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b81024e0 msgr2=0x7fd8b8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:01.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:01.076+0000 7fd8bf82d700 1 --2- 172.21.15.40:0/3332315709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b81024e0 0x7fd8b8074790 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400d980 tx=0x7fd8b400dc90 comp rx=0 tx=0).stop 2024-06-06T03:39:01.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:01.076+0000 7fd8bf82d700 1 -- 172.21.15.40:0/3332315709 shutdown_connections 2024-06-06T03:39:01.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:01.076+0000 7fd8bf82d700 1 --2- 172.21.15.40:0/3332315709 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fd8a40a2ea0 0x7fd8a40a5360 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:01.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:01.076+0000 7fd8bf82d700 1 --2- 172.21.15.40:0/3332315709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8b81085f0 0x7fd8b8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:01.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:01.076+0000 7fd8bf82d700 1 --2- 172.21.15.40:0/3332315709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b81024e0 0x7fd8b8074790 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:01.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:01.076+0000 7fd8bf82d700 1 --2- 172.21.15.40:0/3332315709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8b8102ea0 0x7fd8b8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:01.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:01.076+0000 7fd8bf82d700 1 -- 172.21.15.40:0/3332315709 >> 172.21.15.40:0/3332315709 conn(0x7fd8b8079640 msgr2=0x7fd8b8100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:01.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:01.076+0000 7fd8bf82d700 1 -- 172.21.15.40:0/3332315709 shutdown_connections 2024-06-06T03:39:01.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:01.076+0000 7fd8bf82d700 1 -- 172.21.15.40:0/3332315709 wait complete. 2024-06-06T03:39:01.076 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 29 2024-06-06T03:39:01.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:00 smithi131 ceph-mon[36687]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-06T03:39:01.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:00 smithi040 ceph-mon[32220]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-06T03:39:01.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:00 smithi104 ceph-mon[36569]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-06T03:39:01.674 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-06T03:35:15.961853+0000', 'modified': '2024-06-06T03:38:35.323771+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24470, 'mds_1': 24443, 'mds_2': 24425, 'mds_3': 24403, 'mds_4': 24398}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14580': {'gid': 14580, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6839/1930069210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6838', 'nonce': 1930069210}, {'type': 'v1', 'addr': '172.21.15.40:6839', 'nonce': 1930069210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14586': {'gid': 14586, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6841/2460301066', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6840', 'nonce': 2460301066}, {'type': 'v1', 'addr': '172.21.15.40:6841', 'nonce': 2460301066}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24358': {'gid': 24358, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6833/1833802668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 1833802668}, {'type': 'v1', 'addr': '172.21.15.131:6833', 'nonce': 1833802668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24376': {'gid': 24376, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/1221528906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 1221528906}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1221528906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24388': {'gid': 24388, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6837/526159320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6836', 'nonce': 526159320}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 526159320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24398': {'gid': 24398, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.104:6833/1414464971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6832', 'nonce': 1414464971}, {'type': 'v1', 'addr': '172.21.15.104:6833', 'nonce': 1414464971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.131:6839/782268782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 782268782}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 782268782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.104:6835/75462915', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 75462915}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 75462915}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.104:6837/2498466212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 2498466212}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 2498466212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.104:6839/489480648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 489480648}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 489480648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24470, 'qdb_cluster': [24470, 24443, 24425, 24398, 24403]} 2024-06-06T03:39:01.674 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-06T03:39:01.675 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.k status 2024-06-06T03:39:02.017 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:39:02.257 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:02 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3332315709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:39:02.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:02 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3332315709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:39:02.338 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:02 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3332315709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:39:03.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:03 smithi040 ceph-mon[32220]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:39:03.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:03 smithi040 ceph-mon[32220]: mgrmap e25: x(active, since 24s), standbys: z, y 2024-06-06T03:39:03.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:03 smithi131 ceph-mon[36687]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:39:03.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:03 smithi131 ceph-mon[36687]: mgrmap e25: x(active, since 24s), standbys: z, y 2024-06-06T03:39:03.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:03 smithi104 ceph-mon[36569]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:39:03.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:03 smithi104 ceph-mon[36569]: mgrmap e25: x(active, since 24s), standbys: z, y 2024-06-06T03:39:03.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.685+0000 7f4f02d60700 1 -- 172.21.15.40:0/1760221326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4efc06a170 msgr2=0x7f4efc06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:03.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.685+0000 7f4f02d60700 1 --2- 172.21.15.40:0/1760221326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4efc06a170 0x7f4efc06a5f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009990 tx=0x7f4eec009ca0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.685+0000 7f4f02d60700 1 -- 172.21.15.40:0/1760221326 shutdown_connections 2024-06-06T03:39:03.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.685+0000 7f4f02d60700 1 --2- 172.21.15.40:0/1760221326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4efc108500 0x7f4efc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.685+0000 7f4f02d60700 1 --2- 172.21.15.40:0/1760221326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4efc06ab30 0x7f4efc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.685+0000 7f4f02d60700 1 --2- 172.21.15.40:0/1760221326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4efc06a170 0x7f4efc06a5f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.685+0000 7f4f02d60700 1 -- 172.21.15.40:0/1760221326 >> 172.21.15.40:0/1760221326 conn(0x7f4efc078340 msgr2=0x7f4efc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:03.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.685+0000 7f4f02d60700 1 -- 172.21.15.40:0/1760221326 shutdown_connections 2024-06-06T03:39:03.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.685+0000 7f4f02d60700 1 -- 172.21.15.40:0/1760221326 wait complete. 2024-06-06T03:39:03.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.686+0000 7f4f02d60700 1 Processor -- start 2024-06-06T03:39:03.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.686+0000 7f4f02d60700 1 -- start start 2024-06-06T03:39:03.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.686+0000 7f4f02d60700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4efc06a170 0x7f4efc19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.686+0000 7f4f02d60700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4efc06ab30 0x7f4efc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.686+0000 7f4f02d60700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4efc108500 0x7f4efc198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.686+0000 7f4f02d60700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc112780 con 0x7f4efc108500 2024-06-06T03:39:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.686+0000 7f4f02d60700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc112600 con 0x7f4efc06ab30 2024-06-06T03:39:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.686+0000 7f4f02d60700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc112900 con 0x7f4efc06a170 2024-06-06T03:39:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.686+0000 7f4f012fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4efc108500 0x7f4efc198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.686+0000 7f4f012fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4efc108500 0x7f4efc198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44654/0 (socket says 172.21.15.40:44654) 2024-06-06T03:39:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.686+0000 7f4f012fd700 1 -- 172.21.15.40:0/3056232434 learned_addr learned my addr 172.21.15.40:0/3056232434 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:03.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.686+0000 7f4ef3fff700 1 --2- 172.21.15.40:0/3056232434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4efc06ab30 0x7f4efc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.687+0000 7f4f00afc700 1 --2- 172.21.15.40:0/3056232434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4efc06a170 0x7f4efc19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.687+0000 7f4f012fd700 1 -- 172.21.15.40:0/3056232434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4efc06a170 msgr2=0x7f4efc19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:03.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.687+0000 7f4f012fd700 1 --2- 172.21.15.40:0/3056232434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4efc06a170 0x7f4efc19ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.687+0000 7f4f012fd700 1 -- 172.21.15.40:0/3056232434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4efc06ab30 msgr2=0x7f4efc19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:03.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.687+0000 7f4f012fd700 1 --2- 172.21.15.40:0/3056232434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4efc06ab30 0x7f4efc19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.687+0000 7f4f012fd700 1 -- 172.21.15.40:0/3056232434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4efc199440 con 0x7f4efc108500 2024-06-06T03:39:03.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.687+0000 7f4f012fd700 1 --2- 172.21.15.40:0/3056232434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4efc108500 0x7f4efc198b60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400d980 tx=0x7f4ef400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:03.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.687+0000 7f4ef1ffb700 1 -- 172.21.15.40:0/3056232434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4016070 con 0x7f4efc108500 2024-06-06T03:39:03.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.687+0000 7f4ef1ffb700 1 -- 172.21.15.40:0/3056232434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef4004d10 con 0x7f4efc108500 2024-06-06T03:39:03.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.687+0000 7f4f02d60700 1 -- 172.21.15.40:0/3056232434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4efc199730 con 0x7f4efc108500 2024-06-06T03:39:03.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.687+0000 7f4ef1ffb700 1 -- 172.21.15.40:0/3056232434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40055e0 con 0x7f4efc108500 2024-06-06T03:39:03.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.687+0000 7f4f02d60700 1 -- 172.21.15.40:0/3056232434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4efc1a59c0 con 0x7f4efc108500 2024-06-06T03:39:03.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.689+0000 7f4ef1ffb700 1 -- 172.21.15.40:0/3056232434 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f4ef4004750 con 0x7f4efc108500 2024-06-06T03:39:03.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.690+0000 7f4ef1ffb700 1 --2- 172.21.15.40:0/3056232434 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4ee40a2f30 0x7f4ee40a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.691+0000 7f4f00afc700 1 --2- 172.21.15.40:0/3056232434 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4ee40a2f30 0x7f4ee40a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.691+0000 7f4ef1ffb700 1 -- 172.21.15.40:0/3056232434 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f4ef40cf430 con 0x7f4efc108500 2024-06-06T03:39:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.691+0000 7f4f0d5e3b80 1 Processor -- start 2024-06-06T03:39:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.691+0000 7f4f0d5e3b80 1 -- start start 2024-06-06T03:39:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.691+0000 7f4f00afc700 1 --2- 172.21.15.40:0/3056232434 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4ee40a2f30 0x7f4ee40a53f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009c90 tx=0x7f4ef8009040 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.691+0000 7f4f0d5e3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9c4f10 0x56468a9c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.691+0000 7f4f0d5e3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c5830 0x56468a9bf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.691+0000 7f4f0d5e3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9bf750 0x56468a9bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.691+0000 7f4f0d5e3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56468a8b1f10 con 0x56468a9bf750 2024-06-06T03:39:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.691+0000 7f4f0d5e3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56468a910800 con 0x56468a9c4f10 2024-06-06T03:39:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.691+0000 7f4f0d5e3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56468a878410 con 0x56468a9c5830 2024-06-06T03:39:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4f012fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9bf750 0x56468a9bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4f00afc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9c4f10 0x56468a9c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4f012fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9bf750 0x56468a9bfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44668/0 (socket says 172.21.15.40:44668) 2024-06-06T03:39:03.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4f012fd700 1 -- 172.21.15.40:0/2002996099 learned_addr learned my addr 172.21.15.40:0/2002996099 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:03.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4ef3fff700 1 --2- 172.21.15.40:0/2002996099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c5830 0x56468a9bf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4f012fd700 1 -- 172.21.15.40:0/2002996099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c5830 msgr2=0x56468a9bf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:03.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4f012fd700 1 --2- 172.21.15.40:0/2002996099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c5830 0x56468a9bf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4f012fd700 1 -- 172.21.15.40:0/2002996099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9c4f10 msgr2=0x56468a9c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:03.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4f012fd700 1 --2- 172.21.15.40:0/2002996099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9c4f10 0x56468a9c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4f012fd700 1 -- 172.21.15.40:0/2002996099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468a85f880 con 0x56468a9bf750 2024-06-06T03:39:03.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4f00afc700 1 --2- 172.21.15.40:0/2002996099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9c4f10 0x56468a9c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:39:03.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4f012fd700 1 --2- 172.21.15.40:0/2002996099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9bf750 0x56468a9bfb70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4ef409b720 tx=0x7f4ef409b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d98e055e8fe9386 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:03.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4ee2ffd700 1 -- 172.21.15.40:0/2002996099 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40b4070 con 0x56468a9bf750 2024-06-06T03:39:03.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4ee2ffd700 1 -- 172.21.15.40:0/2002996099 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef409b790 con 0x56468a9bf750 2024-06-06T03:39:03.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.692+0000 7f4ee2ffd700 1 -- 172.21.15.40:0/2002996099 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef400fed0 con 0x56468a9bf750 2024-06-06T03:39:03.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.693+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/2002996099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9bf750 msgr2=0x56468a9bfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:03.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.693+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/2002996099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9bf750 0x56468a9bfb70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4ef409b720 tx=0x7f4ef409b3a0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.693+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/2002996099 shutdown_connections 2024-06-06T03:39:03.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.693+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/2002996099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9bf750 0x56468a9bfb70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.693+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/2002996099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9c4f10 0x56468a9c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.693+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/2002996099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c5830 0x56468a9bf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.693+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/2002996099 >> 172.21.15.40:0/2002996099 conn(0x56468a8ec030 msgr2=0x56468a9ba1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:03.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.693+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/2002996099 shutdown_connections 2024-06-06T03:39:03.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.693+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/2002996099 wait complete. 2024-06-06T03:39:03.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.694+0000 7f4f0d5e3b80 1 -- start start 2024-06-06T03:39:03.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.694+0000 7f4f0d5e3b80 1 Processor -- start 2024-06-06T03:39:03.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.694+0000 7f4f0d5e3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9bf750 0x56468a9cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.694+0000 7f4f0d5e3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c4f10 0x56468a9c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.694+0000 7f4f0d5e3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c5830 0x56468a9c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.694+0000 7f4f0d5e3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56468a8b0f70 con 0x56468a9c4f10 2024-06-06T03:39:03.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.694+0000 7f4f0d5e3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56468a90c7c0 con 0x56468a9bf750 2024-06-06T03:39:03.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.694+0000 7f4f0d5e3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56468a9cc210 con 0x56468a9c5830 2024-06-06T03:39:03.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.694+0000 7f4f0d5e3b80 10 client.0 _ll_register_callbacks cb 0x56468a8a6560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:39:03.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.694+0000 7f4ef3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c4f10 0x56468a9c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.694+0000 7f4ef3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c4f10 0x56468a9c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44676/0 (socket says 172.21.15.40:44676) 2024-06-06T03:39:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.694+0000 7f4ef3fff700 1 -- 172.21.15.40:0/1016777047 learned_addr learned my addr 172.21.15.40:0/1016777047 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.694+0000 7f4f012fd700 1 --2- 172.21.15.40:0/1016777047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c5830 0x56468a9c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.694+0000 7f4f00afc700 1 --2- 172.21.15.40:0/1016777047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9bf750 0x56468a9cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.695+0000 7f4ef3fff700 1 -- 172.21.15.40:0/1016777047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c5830 msgr2=0x56468a9c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.695+0000 7f4ef3fff700 1 --2- 172.21.15.40:0/1016777047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c5830 0x56468a9c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.695+0000 7f4ef3fff700 1 -- 172.21.15.40:0/1016777047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9bf750 msgr2=0x56468a9cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.695+0000 7f4ef3fff700 1 --2- 172.21.15.40:0/1016777047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9bf750 0x56468a9cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.695+0000 7f4ef3fff700 1 -- 172.21.15.40:0/1016777047 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468a8d6700 con 0x56468a9c4f10 2024-06-06T03:39:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.695+0000 7f4f012fd700 1 --2- 172.21.15.40:0/1016777047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c5830 0x56468a9c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:39:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.695+0000 7f4ef3fff700 1 --2- 172.21.15.40:0/1016777047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c4f10 0x56468a9c88c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009990 tx=0x7f4eec0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38cb27105fec0bd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.695+0000 7f4edbfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:39:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.695+0000 7f4edbfff700 1 -- 172.21.15.40:0/1016777047 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec028060 con 0x56468a9c4f10 2024-06-06T03:39:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.695+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1016777047 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef409b790 con 0x56468a9c4f10 2024-06-06T03:39:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.695+0000 7f4edbfff700 1 -- 172.21.15.40:0/1016777047 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f4eec0054d0 con 0x56468a9c4f10 2024-06-06T03:39:03.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.695+0000 7f4edbfff700 1 -- 172.21.15.40:0/1016777047 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4eec009590 con 0x56468a9c4f10 2024-06-06T03:39:03.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.695+0000 7f4f0d5e3b80 10 client.15348 fetch_fsmap learned FSMap version 29 2024-06-06T03:39:03.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.695+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1016777047 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56468a9d6a40 con 0x56468a9c4f10 2024-06-06T03:39:03.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.696+0000 7f4edbfff700 1 -- 172.21.15.40:0/1016777047 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef409b790 con 0x56468a9c4f10 2024-06-06T03:39:03.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.696+0000 7f4edbfff700 1 -- 172.21.15.40:0/1016777047 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4eec00f620 con 0x56468a9c4f10 2024-06-06T03:39:03.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.696+0000 7f4f0d5e3b80 10 client.15348 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T03:39:03.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.696+0000 7f4f0d5e3b80 10 client.15348 resolve_mds: resolved name 'k' to daemon mds.k 2024-06-06T03:39:03.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.696+0000 7f4f0d5e3b80 20 client.15348 populate_metadata read hostname 'smithi040' 2024-06-06T03:39:03.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.696+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/1016777047 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x56468a9d9060 0x56468a9da4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.696+0000 7f4f0d5e3b80 4 client.15348 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:39:03.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.696+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1016777047 --> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56468a91dd00 con 0x56468a9d9060 2024-06-06T03:39:03.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.696+0000 7f4f00afc700 1 --2- 172.21.15.40:0/1016777047 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x56468a9d9060 0x56468a9da4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.697+0000 7f4edbfff700 10 client.15348 ms_handle_connect on v2:172.21.15.104:6832/1414464971 2024-06-06T03:39:03.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.697+0000 7f4f00afc700 1 --2- 172.21.15.40:0/1016777047 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x56468a9d9060 0x56468a9da4d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d95530806a753610 server_cookie=97c0b0f378422817 in_seq=0 out_seq=0 2024-06-06T03:39:03.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4edbfff700 1 -- 172.21.15.40:0/1016777047 <== mds.4 v2:172.21.15.104:6832/1414464971 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56468a91dd00 con 0x56468a9d9060 2024-06-06T03:39:03.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4edbfff700 10 client.15348 handle_command_reply: tid=0 2024-06-06T03:39:03.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4f0d5e3b80 1 client.15348 shutdown 2024-06-06T03:39:03.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1016777047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c4f10 msgr2=0x56468a9c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:03.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/1016777047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c4f10 0x56468a9c88c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009990 tx=0x7f4eec0038b0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1016777047 shutdown_connections 2024-06-06T03:39:03.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/1016777047 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x56468a9d9060 0x56468a9da4d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4edbfff700 0 client.15348 ms_handle_reset on v2:172.21.15.104:6832/1414464971 2024-06-06T03:39:03.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/1016777047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c5830 0x56468a9c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/1016777047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9bf750 0x56468a9cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/1016777047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c4f10 0x56468a9c88c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1016777047 >> 172.21.15.40:0/1016777047 conn(0x56468a8ec030 msgr2=0x56468a9b96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:03.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1016777047 shutdown_connections 2024-06-06T03:39:03.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1016777047 wait complete. 2024-06-06T03:39:03.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.698+0000 7f4f0d5e3b80 20 client.15348 trim_cache size 0 max 16384 2024-06-06T03:39:03.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f0d5e3b80 1 Processor -- start 2024-06-06T03:39:03.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f0d5e3b80 1 -- start start 2024-06-06T03:39:03.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f0d5e3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9bf750 0x56468a9bf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f0d5e3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c4f10 0x56468a9c6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f0d5e3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c5830 0x56468a9b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f0d5e3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56468a9d8810 con 0x56468a9c5830 2024-06-06T03:39:03.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f0d5e3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56468a9d8ee0 con 0x56468a9bf750 2024-06-06T03:39:03.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f0d5e3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56468a9d95b0 con 0x56468a9c4f10 2024-06-06T03:39:03.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f012fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c5830 0x56468a9b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f012fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c5830 0x56468a9b9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44688/0 (socket says 172.21.15.40:44688) 2024-06-06T03:39:03.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f012fd700 1 -- 172.21.15.40:0/3003259198 learned_addr learned my addr 172.21.15.40:0/3003259198 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:03.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f00afc700 1 --2- 172.21.15.40:0/3003259198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9bf750 0x56468a9bf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4ef3fff700 1 --2- 172.21.15.40:0/3003259198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c4f10 0x56468a9c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f012fd700 1 -- 172.21.15.40:0/3003259198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c4f10 msgr2=0x56468a9c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:03.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f012fd700 1 --2- 172.21.15.40:0/3003259198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c4f10 0x56468a9c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f012fd700 1 -- 172.21.15.40:0/3003259198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9bf750 msgr2=0x56468a9bf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:03.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f012fd700 1 --2- 172.21.15.40:0/3003259198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9bf750 0x56468a9bf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.720+0000 7f4f012fd700 1 -- 172.21.15.40:0/3003259198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468a9c6e90 con 0x56468a9c5830 2024-06-06T03:39:03.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.721+0000 7f4f012fd700 1 --2- 172.21.15.40:0/3003259198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c5830 0x56468a9b9e30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40ccba0 tx=0x7f4ef409e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a7661353a1412b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:03.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.721+0000 7f4ee27fc700 1 -- 172.21.15.40:0/3003259198 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40a9070 con 0x56468a9c5830 2024-06-06T03:39:03.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.721+0000 7f4ee27fc700 1 -- 172.21.15.40:0/3003259198 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef40ba070 con 0x56468a9c5830 2024-06-06T03:39:03.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.721+0000 7f4ee27fc700 1 -- 172.21.15.40:0/3003259198 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40b4040 con 0x56468a9c5830 2024-06-06T03:39:03.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.721+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/3003259198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c5830 msgr2=0x56468a9b9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:03.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.721+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/3003259198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c5830 0x56468a9b9e30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40ccba0 tx=0x7f4ef409e040 comp rx=0 tx=0).stop 2024-06-06T03:39:03.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.721+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/3003259198 shutdown_connections 2024-06-06T03:39:03.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.721+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/3003259198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c5830 0x56468a9b9e30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.722+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/3003259198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9bf750 0x56468a9bf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.722+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/3003259198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9c4f10 0x56468a9c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.722+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/3003259198 >> 172.21.15.40:0/3003259198 conn(0x56468a8ec030 msgr2=0x56468a9c5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:03.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.722+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/3003259198 shutdown_connections 2024-06-06T03:39:03.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.722+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/3003259198 wait complete. 2024-06-06T03:39:03.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.722+0000 7f4f0d5e3b80 1 -- start start 2024-06-06T03:39:03.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.722+0000 7f4f0d5e3b80 1 Processor -- start 2024-06-06T03:39:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4f0d5e3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9bf750 0x56468a9e3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4f0d5e3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c4f10 0x56468a9e3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4f0d5e3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9c5830 0x56468a9de900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4f0d5e3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56468a9da350 con 0x56468a9c4f10 2024-06-06T03:39:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4f0d5e3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56468a9d9c80 con 0x56468a9c5830 2024-06-06T03:39:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4f0d5e3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56468a9daa20 con 0x56468a9bf750 2024-06-06T03:39:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4f0d5e3b80 10 client.0 _ll_register_callbacks cb 0x56468a8a6560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:39:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4ef3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c4f10 0x56468a9e3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4ef3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c4f10 0x56468a9e3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44692/0 (socket says 172.21.15.40:44692) 2024-06-06T03:39:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4ef3fff700 1 -- 172.21.15.40:0/1970457555 learned_addr learned my addr 172.21.15.40:0/1970457555 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4f00afc700 1 --2- 172.21.15.40:0/1970457555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9bf750 0x56468a9e3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4f012fd700 1 --2- 172.21.15.40:0/1970457555 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9c5830 0x56468a9de900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4ef3fff700 1 -- 172.21.15.40:0/1970457555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9bf750 msgr2=0x56468a9e3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4ef3fff700 1 --2- 172.21.15.40:0/1970457555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9bf750 0x56468a9e3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4ef3fff700 1 -- 172.21.15.40:0/1970457555 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9c5830 msgr2=0x56468a9de900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4ef3fff700 1 --2- 172.21.15.40:0/1970457555 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9c5830 0x56468a9de900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4ef3fff700 1 -- 172.21.15.40:0/1970457555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468a9dee40 con 0x56468a9c4f10 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4ef3fff700 1 --2- 172.21.15.40:0/1970457555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c4f10 0x56468a9e3e90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4eec005090 tx=0x7f4eec005580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1779280ec8859b11 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4edbfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4edbfff700 1 -- 172.21.15.40:0/1970457555 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec003e60 con 0x56468a9c4f10 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4edbfff700 1 -- 172.21.15.40:0/1970457555 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f4eec017940 con 0x56468a9c4f10 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.723+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1970457555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef40ba070 con 0x56468a9c4f10 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.724+0000 7f4edbfff700 1 -- 172.21.15.40:0/1970457555 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4eec03e050 con 0x56468a9c4f10 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.724+0000 7f4f0d5e3b80 10 client.15354 fetch_fsmap learned FSMap version 29 2024-06-06T03:39:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.724+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1970457555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56468aa45030 con 0x56468a9c4f10 2024-06-06T03:39:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.724+0000 7f4edbfff700 1 -- 172.21.15.40:0/1970457555 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef40ba070 con 0x56468a9c4f10 2024-06-06T03:39:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.724+0000 7f4edbfff700 1 -- 172.21.15.40:0/1970457555 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4eec035070 con 0x56468a9c4f10 2024-06-06T03:39:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.724+0000 7f4f0d5e3b80 10 client.15354 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T03:39:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.724+0000 7f4f0d5e3b80 10 client.15354 resolve_mds: resolved name 'k' to daemon mds.k 2024-06-06T03:39:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.724+0000 7f4f0d5e3b80 20 client.15354 populate_metadata read hostname 'smithi040' 2024-06-06T03:39:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.724+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/1970457555 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x56468aa49c50 0x56468aa48960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.724+0000 7f4f0d5e3b80 4 client.15354 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:39:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.724+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1970457555 --> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] -- command(tid 0: {"prefix": "status"}) -- 0x56468a918280 con 0x56468aa49c50 2024-06-06T03:39:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.724+0000 7f4f00afc700 1 --2- 172.21.15.40:0/1970457555 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x56468aa49c50 0x56468aa48960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4f00afc700 1 --2- 172.21.15.40:0/1970457555 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x56468aa49c50 0x56468aa48960 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6f7c6fd7e735970a server_cookie=1a7abdc47696c51d in_seq=0 out_seq=0 2024-06-06T03:39:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4edbfff700 10 client.15354 ms_handle_connect on v2:172.21.15.104:6832/1414464971 2024-06-06T03:39:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4edbfff700 1 -- 172.21.15.40:0/1970457555 <== mds.4 v2:172.21.15.104:6832/1414464971 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56468a918280 con 0x56468aa49c50 2024-06-06T03:39:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4edbfff700 10 client.15354 handle_command_reply: tid=0 2024-06-06T03:39:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4f0d5e3b80 1 client.15354 shutdown 2024-06-06T03:39:03.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1970457555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c4f10 msgr2=0x56468a9e3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:03.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/1970457555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c4f10 0x56468a9e3e90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4eec005090 tx=0x7f4eec005580 comp rx=0 tx=0).stop 2024-06-06T03:39:03.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1970457555 shutdown_connections 2024-06-06T03:39:03.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/1970457555 >> [v2:172.21.15.104:6832/1414464971,v1:172.21.15.104:6833/1414464971] conn(0x56468aa49c50 0x56468aa48960 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/1970457555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56468a9bf750 0x56468a9e3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/1970457555 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56468a9c5830 0x56468a9de900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4edbfff700 0 client.15354 ms_handle_reset on v2:172.21.15.104:6832/1414464971 2024-06-06T03:39:03.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4f0d5e3b80 1 --2- 172.21.15.40:0/1970457555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56468a9c4f10 0x56468a9e3e90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1970457555 >> 172.21.15.40:0/1970457555 conn(0x56468a8ec030 msgr2=0x56468a9c64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:03.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1970457555 shutdown_connections 2024-06-06T03:39:03.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.725+0000 7f4f0d5e3b80 1 -- 172.21.15.40:0/1970457555 wait complete. 2024-06-06T03:39:03.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.726+0000 7f4f0d5e3b80 20 client.15354 trim_cache size 0 max 16384 2024-06-06T03:39:03.732 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:39:03.732 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:39:03.732 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 4, 2024-06-06T03:39:03.732 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24398, 2024-06-06T03:39:03.732 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:39:03.732 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:39:03.732 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:39:03.732 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 209.79558547799999, 2024-06-06T03:39:03.732 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 29, 2024-06-06T03:39:03.732 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 91, 2024-06-06T03:39:03.732 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 90, 2024-06-06T03:39:03.733 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 277.52715016899998 2024-06-06T03:39:03.733 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:39:03.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.726+0000 7f4f02d60700 1 -- 172.21.15.40:0/3056232434 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4ee40a2f30 msgr2=0x7f4ee40a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:03.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.726+0000 7f4f02d60700 1 --2- 172.21.15.40:0/3056232434 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4ee40a2f30 0x7f4ee40a53f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009c90 tx=0x7f4ef8009040 comp rx=0 tx=0).stop 2024-06-06T03:39:03.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.726+0000 7f4f02d60700 1 -- 172.21.15.40:0/3056232434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4efc108500 msgr2=0x7f4efc198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:03.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.726+0000 7f4f02d60700 1 --2- 172.21.15.40:0/3056232434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4efc108500 0x7f4efc198b60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400d980 tx=0x7f4ef400dc90 comp rx=0 tx=0).stop 2024-06-06T03:39:03.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.726+0000 7f4f02d60700 1 -- 172.21.15.40:0/3056232434 shutdown_connections 2024-06-06T03:39:03.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.726+0000 7f4f02d60700 1 --2- 172.21.15.40:0/3056232434 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4ee40a2f30 0x7f4ee40a53f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.726+0000 7f4f02d60700 1 --2- 172.21.15.40:0/3056232434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4efc06a170 0x7f4efc19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.726+0000 7f4f02d60700 1 --2- 172.21.15.40:0/3056232434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4efc108500 0x7f4efc198b60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.726+0000 7f4f02d60700 1 --2- 172.21.15.40:0/3056232434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4efc06ab30 0x7f4efc19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:03.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.726+0000 7f4f02d60700 1 -- 172.21.15.40:0/3056232434 >> 172.21.15.40:0/3056232434 conn(0x7f4efc078340 msgr2=0x7f4efc100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:03.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.726+0000 7f4f02d60700 1 -- 172.21.15.40:0/3056232434 shutdown_connections 2024-06-06T03:39:03.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:03.726+0000 7f4f02d60700 1 -- 172.21.15.40:0/3056232434 wait complete. 2024-06-06T03:39:04.383 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-06T03:39:04.630 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:39:05.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:05 smithi131 ceph-mon[36687]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-06-06T03:39:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:05 smithi040 ceph-mon[32220]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-06-06T03:39:05.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:05 smithi104 ceph-mon[36569]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-06-06T03:39:06.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.561+0000 7fadca6b6700 1 -- 172.21.15.40:0/1140745516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc4101d50 msgr2=0x7fadc410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:06.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.561+0000 7fadca6b6700 1 --2- 172.21.15.40:0/1140745516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc4101d50 0x7fadc410e240 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00b0a0 tx=0x7fadbc00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.561+0000 7fadca6b6700 1 -- 172.21.15.40:0/1140745516 shutdown_connections 2024-06-06T03:39:06.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.561+0000 7fadca6b6700 1 --2- 172.21.15.40:0/1140745516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc4107400 0x7fadc41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.561+0000 7fadca6b6700 1 --2- 172.21.15.40:0/1140745516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc4101d50 0x7fadc410e240 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.561+0000 7fadca6b6700 1 --2- 172.21.15.40:0/1140745516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadc4101390 0x7fadc4101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.561+0000 7fadca6b6700 1 -- 172.21.15.40:0/1140745516 >> 172.21.15.40:0/1140745516 conn(0x7fadc4078340 msgr2=0x7fadc4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:06.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.561+0000 7fadca6b6700 1 -- 172.21.15.40:0/1140745516 shutdown_connections 2024-06-06T03:39:06.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.561+0000 7fadca6b6700 1 -- 172.21.15.40:0/1140745516 wait complete. 2024-06-06T03:39:06.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.562+0000 7fadca6b6700 1 Processor -- start 2024-06-06T03:39:06.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.562+0000 7fadca6b6700 1 -- start start 2024-06-06T03:39:06.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.562+0000 7fadca6b6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc4101390 0x7fadc419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.562+0000 7fadca6b6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc4101d50 0x7fadc419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.562+0000 7fadca6b6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadc4107400 0x7fadc4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.562+0000 7fadca6b6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4111690 con 0x7fadc4101d50 2024-06-06T03:39:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.562+0000 7fadca6b6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4111510 con 0x7fadc4107400 2024-06-06T03:39:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.562+0000 7fadca6b6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4111810 con 0x7fadc4101390 2024-06-06T03:39:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.562+0000 7fadc37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc4101d50 0x7fadc419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.562+0000 7fadc37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc4101d50 0x7fadc419ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44714/0 (socket says 172.21.15.40:44714) 2024-06-06T03:39:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.562+0000 7fadc37fe700 1 -- 172.21.15.40:0/1773047547 learned_addr learned my addr 172.21.15.40:0/1773047547 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.563+0000 7fadc3fff700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc4101390 0x7fadc419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.563+0000 7fadc8c53700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadc4107400 0x7fadc4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.563+0000 7fadc37fe700 1 -- 172.21.15.40:0/1773047547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc4101390 msgr2=0x7fadc419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.563+0000 7fadc37fe700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc4101390 0x7fadc419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.563+0000 7fadc37fe700 1 -- 172.21.15.40:0/1773047547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadc4107400 msgr2=0x7fadc4198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.563+0000 7fadc37fe700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadc4107400 0x7fadc4198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.563+0000 7fadc37fe700 1 -- 172.21.15.40:0/1773047547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc41990f0 con 0x7fadc4101d50 2024-06-06T03:39:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.563+0000 7fadc8c53700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadc4107400 0x7fadc4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:39:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.563+0000 7fadc37fe700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc4101d50 0x7fadc419ed10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fadb800d950 tx=0x7fadb800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.563+0000 7fadc17fa700 1 -- 172.21.15.40:0/1773047547 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8016070 con 0x7fadc4101d50 2024-06-06T03:39:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.563+0000 7fadca6b6700 1 -- 172.21.15.40:0/1773047547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc41993e0 con 0x7fadc4101d50 2024-06-06T03:39:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.563+0000 7fadc17fa700 1 -- 172.21.15.40:0/1773047547 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb8009b40 con 0x7fadc4101d50 2024-06-06T03:39:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.563+0000 7fadc17fa700 1 -- 172.21.15.40:0/1773047547 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb800f620 con 0x7fadc4101d50 2024-06-06T03:39:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.563+0000 7fadca6b6700 1 -- 172.21.15.40:0/1773047547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc41a56a0 con 0x7fadc4101d50 2024-06-06T03:39:06.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.566+0000 7fadc17fa700 1 -- 172.21.15.40:0/1773047547 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7fadb8019070 con 0x7fadc4101d50 2024-06-06T03:39:06.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.566+0000 7fadc17fa700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fadac0a2f30 0x7fadac0a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.566+0000 7fadc3fff700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fadac0a2f30 0x7fadac0a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.566+0000 7fadc17fa700 1 -- 172.21.15.40:0/1773047547 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fadb80cf640 con 0x7fadc4101d50 2024-06-06T03:39:06.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.567+0000 7fadc3fff700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fadac0a2f30 0x7fadac0a53f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fadb4005e10 tx=0x7fadb4005d40 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:06.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.567+0000 7fadd4f39b80 1 Processor -- start 2024-06-06T03:39:06.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.567+0000 7fadd4f39b80 1 -- start start 2024-06-06T03:39:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.567+0000 7fadd4f39b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682ff10 0x5643768302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.567+0000 7fadd4f39b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564376830830 0x56437682a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.567+0000 7fadd4f39b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56437682a750 0x56437682ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.567+0000 7fadd4f39b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56437671cf10 con 0x564376830830 2024-06-06T03:39:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.567+0000 7fadd4f39b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56437677b800 con 0x56437682ff10 2024-06-06T03:39:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.567+0000 7fadd4f39b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5643766e3410 con 0x56437682a750 2024-06-06T03:39:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.568+0000 7fadc37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564376830830 0x56437682a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.568+0000 7fadc37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564376830830 0x56437682a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44724/0 (socket says 172.21.15.40:44724) 2024-06-06T03:39:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.568+0000 7fadc37fe700 1 -- 172.21.15.40:0/2435505042 learned_addr learned my addr 172.21.15.40:0/2435505042 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.568+0000 7fadc3fff700 1 --2- 172.21.15.40:0/2435505042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682ff10 0x5643768302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.568+0000 7fadc8c53700 1 --2- 172.21.15.40:0/2435505042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56437682a750 0x56437682ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.568+0000 7fadc37fe700 1 -- 172.21.15.40:0/2435505042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56437682a750 msgr2=0x56437682ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.568+0000 7fadc37fe700 1 --2- 172.21.15.40:0/2435505042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56437682a750 0x56437682ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.568+0000 7fadc37fe700 1 -- 172.21.15.40:0/2435505042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682ff10 msgr2=0x5643768302f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:06.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.568+0000 7fadc37fe700 1 --2- 172.21.15.40:0/2435505042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682ff10 0x5643768302f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.568+0000 7fadc37fe700 1 -- 172.21.15.40:0/2435505042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643766ca880 con 0x564376830830 2024-06-06T03:39:06.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.568+0000 7fadc37fe700 1 --2- 172.21.15.40:0/2435505042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564376830830 0x56437682a210 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fadb8098370 tx=0x7fadb809b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290b098f8315df61 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:06.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.568+0000 7fadaa7fc700 1 -- 172.21.15.40:0/2435505042 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8020070 con 0x564376830830 2024-06-06T03:39:06.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.568+0000 7fadaa7fc700 1 -- 172.21.15.40:0/2435505042 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb80a4610 con 0x564376830830 2024-06-06T03:39:06.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.568+0000 7fadaa7fc700 1 -- 172.21.15.40:0/2435505042 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb80a43e0 con 0x564376830830 2024-06-06T03:39:06.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.569+0000 7fadd4f39b80 1 -- 172.21.15.40:0/2435505042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564376830830 msgr2=0x56437682a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:06.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.569+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/2435505042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564376830830 0x56437682a210 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fadb8098370 tx=0x7fadb809b5f0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.569+0000 7fadd4f39b80 1 -- 172.21.15.40:0/2435505042 shutdown_connections 2024-06-06T03:39:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.569+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/2435505042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56437682a750 0x56437682ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.569+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/2435505042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682ff10 0x5643768302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.569+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/2435505042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564376830830 0x56437682a210 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.569+0000 7fadd4f39b80 1 -- 172.21.15.40:0/2435505042 >> 172.21.15.40:0/2435505042 conn(0x564376757030 msgr2=0x5643768251d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.569+0000 7fadd4f39b80 1 -- 172.21.15.40:0/2435505042 shutdown_connections 2024-06-06T03:39:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.569+0000 7fadd4f39b80 1 -- 172.21.15.40:0/2435505042 wait complete. 2024-06-06T03:39:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.569+0000 7fadd4f39b80 1 -- start start 2024-06-06T03:39:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadd4f39b80 1 Processor -- start 2024-06-06T03:39:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadd4f39b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 0x564376836660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadd4f39b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x564376833960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadd4f39b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 0x564376833ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadd4f39b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56437671bf70 con 0x56437682ff10 2024-06-06T03:39:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadd4f39b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5643767777c0 con 0x56437682a750 2024-06-06T03:39:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadd4f39b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564376837150 con 0x564376830830 2024-06-06T03:39:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadd4f39b80 10 client.0 _ll_register_callbacks cb 0x564376711560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:39:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadc37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x564376833960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadc37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x564376833960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44738/0 (socket says 172.21.15.40:44738) 2024-06-06T03:39:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadc37fe700 1 -- 172.21.15.40:0/3605197015 learned_addr learned my addr 172.21.15.40:0/3605197015 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadc3fff700 1 --2- 172.21.15.40:0/3605197015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 0x564376836660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadc8c53700 1 --2- 172.21.15.40:0/3605197015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 0x564376833ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadc37fe700 1 -- 172.21.15.40:0/3605197015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 msgr2=0x564376833ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadc37fe700 1 --2- 172.21.15.40:0/3605197015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 0x564376833ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadc37fe700 1 -- 172.21.15.40:0/3605197015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 msgr2=0x564376836660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadc37fe700 1 --2- 172.21.15.40:0/3605197015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 0x564376836660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.570+0000 7fadc37fe700 1 -- 172.21.15.40:0/3605197015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564376741700 con 0x56437682ff10 2024-06-06T03:39:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.571+0000 7fadc37fe700 1 --2- 172.21.15.40:0/3605197015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x564376833960 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fadb8012040 tx=0x7fadb8005ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72422a68e2d809b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.571+0000 7fada37fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:39:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.571+0000 7fada37fe700 1 -- 172.21.15.40:0/3605197015 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8020050 con 0x56437682ff10 2024-06-06T03:39:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.571+0000 7fada37fe700 1 -- 172.21.15.40:0/3605197015 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fadb809be10 con 0x56437682ff10 2024-06-06T03:39:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.571+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3605197015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb80a4610 con 0x56437682ff10 2024-06-06T03:39:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.571+0000 7fada37fe700 1 -- 172.21.15.40:0/3605197015 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fadb8081050 con 0x56437682ff10 2024-06-06T03:39:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.571+0000 7fadd4f39b80 10 client.15366 fetch_fsmap learned FSMap version 29 2024-06-06T03:39:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.571+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3605197015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5643768419a0 con 0x56437682ff10 2024-06-06T03:39:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.571+0000 7fada37fe700 1 -- 172.21.15.40:0/3605197015 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb80a4610 con 0x56437682ff10 2024-06-06T03:39:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.571+0000 7fada37fe700 1 -- 172.21.15.40:0/3605197015 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fadb80b4ba0 con 0x56437682ff10 2024-06-06T03:39:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.571+0000 7fadd4f39b80 10 client.15366 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T03:39:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.571+0000 7fadd4f39b80 10 client.15366 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T03:39:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.572+0000 7fadd4f39b80 20 client.15366 populate_metadata read hostname 'smithi040' 2024-06-06T03:39:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.572+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3605197015 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x564376843af0 0x564376844f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.572+0000 7fadd4f39b80 4 client.15366 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:39:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.572+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3605197015 --> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564376788d00 con 0x564376843af0 2024-06-06T03:39:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.572+0000 7fadc3fff700 1 --2- 172.21.15.40:0/3605197015 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x564376843af0 0x564376844f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.572+0000 7fadc3fff700 1 --2- 172.21.15.40:0/3605197015 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x564376843af0 0x564376844f60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1a5982b9ebc55472 server_cookie=23dd726caf9140f3 in_seq=0 out_seq=0 2024-06-06T03:39:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.572+0000 7fada37fe700 10 client.15366 ms_handle_connect on v2:172.21.15.131:6838/782268782 2024-06-06T03:39:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.573+0000 7fada37fe700 1 -- 172.21.15.40:0/3605197015 <== mds.3 v2:172.21.15.131:6838/782268782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564376788d00 con 0x564376843af0 2024-06-06T03:39:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.573+0000 7fada37fe700 10 client.15366 handle_command_reply: tid=0 2024-06-06T03:39:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.573+0000 7fadd4f39b80 1 client.15366 shutdown 2024-06-06T03:39:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.573+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3605197015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 msgr2=0x564376833960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.573+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3605197015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x564376833960 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fadb8012040 tx=0x7fadb8005ba0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.573+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3605197015 shutdown_connections 2024-06-06T03:39:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.573+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3605197015 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x564376843af0 0x564376844f60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.573+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3605197015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 0x564376833ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.573+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3605197015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 0x564376836660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.573+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3605197015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x564376833960 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.573+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3605197015 >> 172.21.15.40:0/3605197015 conn(0x564376757030 msgr2=0x5643768246b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.573+0000 7fada37fe700 0 client.15366 ms_handle_reset on v2:172.21.15.131:6838/782268782 2024-06-06T03:39:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.573+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3605197015 shutdown_connections 2024-06-06T03:39:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.573+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3605197015 wait complete. 2024-06-06T03:39:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.574+0000 7fadd4f39b80 20 client.15366 trim_cache size 0 max 16384 2024-06-06T03:39:06.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.595+0000 7fadd4f39b80 1 Processor -- start 2024-06-06T03:39:06.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.595+0000 7fadd4f39b80 1 -- start start 2024-06-06T03:39:06.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.595+0000 7fadd4f39b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 0x564376831360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.595+0000 7fadd4f39b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x5643768361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.595+0000 7fadd4f39b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 0x56437684e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.595+0000 7fadd4f39b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5643768432a0 con 0x56437682ff10 2024-06-06T03:39:06.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.595+0000 7fadd4f39b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564376843970 con 0x56437682a750 2024-06-06T03:39:06.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.595+0000 7fadd4f39b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564376844040 con 0x564376830830 2024-06-06T03:39:06.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fadc37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x5643768361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fadc37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x5643768361a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44742/0 (socket says 172.21.15.40:44742) 2024-06-06T03:39:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fadc37fe700 1 -- 172.21.15.40:0/3782936777 learned_addr learned my addr 172.21.15.40:0/3782936777 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fadc8c53700 1 --2- 172.21.15.40:0/3782936777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 0x56437684e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fadc3fff700 1 --2- 172.21.15.40:0/3782936777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 0x564376831360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fadc37fe700 1 -- 172.21.15.40:0/3782936777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 msgr2=0x56437684e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fadc37fe700 1 --2- 172.21.15.40:0/3782936777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 0x56437684e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fadc37fe700 1 -- 172.21.15.40:0/3782936777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 msgr2=0x564376831360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fadc37fe700 1 --2- 172.21.15.40:0/3782936777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 0x564376831360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fadc37fe700 1 -- 172.21.15.40:0/3782936777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643768430b0 con 0x56437682ff10 2024-06-06T03:39:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fadc3fff700 1 --2- 172.21.15.40:0/3782936777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 0x564376831360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:39:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fadc8c53700 1 --2- 172.21.15.40:0/3782936777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 0x56437684e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:39:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fadc37fe700 1 --2- 172.21.15.40:0/3782936777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x5643768361a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fadb8097e00 tx=0x7fadb8097c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94e4f272327e9c8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:06.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fada9ffb700 1 -- 172.21.15.40:0/3782936777 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8020070 con 0x56437682ff10 2024-06-06T03:39:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fada9ffb700 1 -- 172.21.15.40:0/3782936777 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb809aec0 con 0x56437682ff10 2024-06-06T03:39:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.596+0000 7fada9ffb700 1 -- 172.21.15.40:0/3782936777 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb80b4ba0 con 0x56437682ff10 2024-06-06T03:39:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.597+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3782936777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 msgr2=0x5643768361a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.597+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3782936777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x5643768361a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fadb8097e00 tx=0x7fadb8097c40 comp rx=0 tx=0).stop 2024-06-06T03:39:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.597+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3782936777 shutdown_connections 2024-06-06T03:39:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.597+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3782936777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 0x56437684e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.597+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3782936777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 0x564376831360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.597+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3782936777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x5643768361a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.597+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3782936777 >> 172.21.15.40:0/3782936777 conn(0x564376757030 msgr2=0x564376830c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.597+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3782936777 shutdown_connections 2024-06-06T03:39:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.597+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3782936777 wait complete. 2024-06-06T03:39:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.597+0000 7fadd4f39b80 1 -- start start 2024-06-06T03:39:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadd4f39b80 1 Processor -- start 2024-06-06T03:39:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadd4f39b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 0x5643768a7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadd4f39b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x5643768a7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadd4f39b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 0x5643768a7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadd4f39b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564376844de0 con 0x56437682ff10 2024-06-06T03:39:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadd4f39b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564376844710 con 0x56437682a750 2024-06-06T03:39:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadd4f39b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5643768454b0 con 0x564376830830 2024-06-06T03:39:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadd4f39b80 10 client.0 _ll_register_callbacks cb 0x564376711560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:39:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadc37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x5643768a7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadc37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x5643768a7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44750/0 (socket says 172.21.15.40:44750) 2024-06-06T03:39:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadc37fe700 1 -- 172.21.15.40:0/3591607012 learned_addr learned my addr 172.21.15.40:0/3591607012 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadc8c53700 1 --2- 172.21.15.40:0/3591607012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 0x5643768a7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadc3fff700 1 --2- 172.21.15.40:0/3591607012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 0x5643768a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadc37fe700 1 -- 172.21.15.40:0/3591607012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 msgr2=0x5643768a7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadc37fe700 1 --2- 172.21.15.40:0/3591607012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 0x5643768a7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadc37fe700 1 -- 172.21.15.40:0/3591607012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 msgr2=0x5643768a7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadc37fe700 1 --2- 172.21.15.40:0/3591607012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 0x5643768a7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadc37fe700 1 -- 172.21.15.40:0/3591607012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643768b2500 con 0x56437682ff10 2024-06-06T03:39:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.598+0000 7fadc3fff700 1 --2- 172.21.15.40:0/3591607012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 0x5643768a7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:39:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fadc37fe700 1 --2- 172.21.15.40:0/3591607012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x5643768a7880 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fadb809b930 tx=0x7fadb809c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fada61c24715ce server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fada37fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:39:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fada37fe700 1 -- 172.21.15.40:0/3591607012 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8020070 con 0x56437682ff10 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fada37fe700 1 -- 172.21.15.40:0/3591607012 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fadb80b4ba0 con 0x56437682ff10 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3591607012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb809aec0 con 0x56437682ff10 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fada37fe700 1 -- 172.21.15.40:0/3591607012 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fadb80ccf80 con 0x56437682ff10 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fadd4f39b80 10 client.15372 fetch_fsmap learned FSMap version 29 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3591607012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5643768b2500 con 0x56437682ff10 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fada37fe700 1 -- 172.21.15.40:0/3591607012 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb809aec0 con 0x56437682ff10 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fada37fe700 1 -- 172.21.15.40:0/3591607012 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fadb8086120 con 0x56437682ff10 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fadd4f39b80 10 client.15372 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fadd4f39b80 10 client.15372 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fadd4f39b80 20 client.15372 populate_metadata read hostname 'smithi040' 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3591607012 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x5643768b7ba0 0x5643768b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fadd4f39b80 4 client.15372 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.599+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3591607012 --> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] -- command(tid 0: {"prefix": "status"}) -- 0x564376783280 con 0x5643768b7ba0 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.600+0000 7fadc3fff700 1 --2- 172.21.15.40:0/3591607012 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x5643768b7ba0 0x5643768b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.600+0000 7fadc3fff700 1 --2- 172.21.15.40:0/3591607012 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x5643768b7ba0 0x5643768b68b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=30ef8faf3bc6cb14 server_cookie=1351751544762e2f in_seq=0 out_seq=0 2024-06-06T03:39:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.600+0000 7fada37fe700 10 client.15372 ms_handle_connect on v2:172.21.15.131:6838/782268782 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.600+0000 7fada37fe700 1 -- 172.21.15.40:0/3591607012 <== mds.3 v2:172.21.15.131:6838/782268782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564376783280 con 0x5643768b7ba0 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.600+0000 7fada37fe700 10 client.15372 handle_command_reply: tid=0 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.600+0000 7fadd4f39b80 1 client.15372 shutdown 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3591607012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 msgr2=0x5643768a7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3591607012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x5643768a7880 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fadb809b930 tx=0x7fadb809c150 comp rx=0 tx=0).stop 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3591607012 shutdown_connections 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3591607012 >> [v2:172.21.15.131:6838/782268782,v1:172.21.15.131:6839/782268782] conn(0x5643768b7ba0 0x5643768b68b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3591607012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564376830830 0x5643768a7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3591607012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56437682a750 0x5643768a7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fada37fe700 0 client.15372 ms_handle_reset on v2:172.21.15.131:6838/782268782 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadd4f39b80 1 --2- 172.21.15.40:0/3591607012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56437682ff10 0x5643768a7880 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3591607012 >> 172.21.15.40:0/3591607012 conn(0x564376757030 msgr2=0x5643768314e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3591607012 shutdown_connections 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadd4f39b80 1 -- 172.21.15.40:0/3591607012 wait complete. 2024-06-06T03:39:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadd4f39b80 20 client.15372 trim_cache size 0 max 16384 2024-06-06T03:39:06.606 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:39:06.606 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:39:06.606 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 3, 2024-06-06T03:39:06.606 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24403, 2024-06-06T03:39:06.606 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:39:06.606 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:39:06.606 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:39:06.606 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 213.68005932299999, 2024-06-06T03:39:06.606 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 29, 2024-06-06T03:39:06.606 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 91, 2024-06-06T03:39:06.606 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 90, 2024-06-06T03:39:06.607 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 258.20118730799999 2024-06-06T03:39:06.607 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:39:06.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadca6b6700 1 -- 172.21.15.40:0/1773047547 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fadac0a2f30 msgr2=0x7fadac0a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:06.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadca6b6700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fadac0a2f30 0x7fadac0a53f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fadb4005e10 tx=0x7fadb4005d40 comp rx=0 tx=0).stop 2024-06-06T03:39:06.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadca6b6700 1 -- 172.21.15.40:0/1773047547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc4101d50 msgr2=0x7fadc419ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:06.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.601+0000 7fadca6b6700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc4101d50 0x7fadc419ed10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fadb800d950 tx=0x7fadb800dd10 comp rx=0 tx=0).stop 2024-06-06T03:39:06.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.602+0000 7fadca6b6700 1 -- 172.21.15.40:0/1773047547 shutdown_connections 2024-06-06T03:39:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.602+0000 7fadca6b6700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fadac0a2f30 0x7fadac0a53f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.602+0000 7fadca6b6700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc4101390 0x7fadc419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.602+0000 7fadca6b6700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadc4107400 0x7fadc4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.602+0000 7fadca6b6700 1 --2- 172.21.15.40:0/1773047547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadc4101d50 0x7fadc419ed10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.602+0000 7fadca6b6700 1 -- 172.21.15.40:0/1773047547 >> 172.21.15.40:0/1773047547 conn(0x7fadc4078340 msgr2=0x7fadc4100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.602+0000 7fadca6b6700 1 -- 172.21.15.40:0/1773047547 shutdown_connections 2024-06-06T03:39:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:06.602+0000 7fadca6b6700 1 -- 172.21.15.40:0/1773047547 wait complete. 2024-06-06T03:39:07.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:07 smithi131 ceph-mon[36687]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-06-06T03:39:07.317 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:07 smithi040 ceph-mon[32220]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-06-06T03:39:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:07 smithi104 ceph-mon[36569]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-06-06T03:39:07.346 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-06-06T03:39:07.597 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:39:09.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.257+0000 7f4099412700 1 -- 172.21.15.40:0/3487028051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4094101f80 msgr2=0x7f4094105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:09.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.257+0000 7f4099412700 1 --2- 172.21.15.40:0/3487028051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4094101f80 0x7f4094105fd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f40880099c0 tx=0x7f4088009cd0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.257+0000 7f4099412700 1 -- 172.21.15.40:0/3487028051 shutdown_connections 2024-06-06T03:39:09.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.257+0000 7f4099412700 1 --2- 172.21.15.40:0/3487028051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40941066b0 0x7f409410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.257+0000 7f4099412700 1 --2- 172.21.15.40:0/3487028051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40941015d0 0x7f40941019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.257+0000 7f4099412700 1 --2- 172.21.15.40:0/3487028051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4094101f80 0x7f4094105fd0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.257+0000 7f4099412700 1 -- 172.21.15.40:0/3487028051 >> 172.21.15.40:0/3487028051 conn(0x7f4094078400 msgr2=0x7f4094078810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:09.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.257+0000 7f4099412700 1 -- 172.21.15.40:0/3487028051 shutdown_connections 2024-06-06T03:39:09.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.257+0000 7f4099412700 1 -- 172.21.15.40:0/3487028051 wait complete. 2024-06-06T03:39:09.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.257+0000 7f4099412700 1 Processor -- start 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.258+0000 7f4099412700 1 -- start start 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.258+0000 7f4099412700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40941015d0 0x7f409419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.258+0000 7f4099412700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4094101f80 0x7f409419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.258+0000 7f4099412700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40941066b0 0x7f4094198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.258+0000 7f4099412700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4094111670 con 0x7f40941015d0 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.258+0000 7f4099412700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f40941114f0 con 0x7f40941066b0 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.258+0000 7f4099412700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f40941117f0 con 0x7f4094101f80 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.258+0000 7f4092ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40941015d0 0x7f409419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.258+0000 7f4092ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40941015d0 0x7f409419e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36004/0 (socket says 172.21.15.40:36004) 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.258+0000 7f4092ffd700 1 -- 172.21.15.40:0/1433048508 learned_addr learned my addr 172.21.15.40:0/1433048508 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.258+0000 7f40927fc700 1 --2- 172.21.15.40:0/1433048508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4094101f80 0x7f409419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.258+0000 7f40937fe700 1 --2- 172.21.15.40:0/1433048508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40941066b0 0x7f4094198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.258+0000 7f4092ffd700 1 -- 172.21.15.40:0/1433048508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4094101f80 msgr2=0x7f409419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.259+0000 7f4092ffd700 1 --2- 172.21.15.40:0/1433048508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4094101f80 0x7f409419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.259+0000 7f4092ffd700 1 -- 172.21.15.40:0/1433048508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40941066b0 msgr2=0x7f4094198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:09.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.259+0000 7f4092ffd700 1 --2- 172.21.15.40:0/1433048508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40941066b0 0x7f4094198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.259+0000 7f4092ffd700 1 -- 172.21.15.40:0/1433048508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40941991c0 con 0x7f40941015d0 2024-06-06T03:39:09.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.259+0000 7f4092ffd700 1 --2- 172.21.15.40:0/1433048508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40941015d0 0x7f409419e8a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f408400d950 tx=0x7f408400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:09.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.259+0000 7f407bfff700 1 -- 172.21.15.40:0/1433048508 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084016070 con 0x7f40941015d0 2024-06-06T03:39:09.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.259+0000 7f4099412700 1 -- 172.21.15.40:0/1433048508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40941994b0 con 0x7f40941015d0 2024-06-06T03:39:09.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.259+0000 7f407bfff700 1 -- 172.21.15.40:0/1433048508 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4084009e20 con 0x7f40941015d0 2024-06-06T03:39:09.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.259+0000 7f4099412700 1 -- 172.21.15.40:0/1433048508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40941a5770 con 0x7f40941015d0 2024-06-06T03:39:09.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.259+0000 7f407bfff700 1 -- 172.21.15.40:0/1433048508 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408400f620 con 0x7f40941015d0 2024-06-06T03:39:09.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.261+0000 7f407bfff700 1 -- 172.21.15.40:0/1433048508 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f4084019070 con 0x7f40941015d0 2024-06-06T03:39:09.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.262+0000 7f407bfff700 1 --2- 172.21.15.40:0/1433048508 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f407c0a2eb0 0x7f407c0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.262+0000 7f407bfff700 1 -- 172.21.15.40:0/1433048508 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f40840cf270 con 0x7f40941015d0 2024-06-06T03:39:09.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.262+0000 7f40927fc700 1 --2- 172.21.15.40:0/1433048508 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f407c0a2eb0 0x7f407c0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.262+0000 7f40927fc700 1 --2- 172.21.15.40:0/1433048508 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f407c0a2eb0 0x7f407c0a5370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4088000c00 tx=0x7f4088005fb0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:09.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f40a3c95b80 1 Processor -- start 2024-06-06T03:39:09.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f40a3c95b80 1 -- start start 2024-06-06T03:39:09.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f40a3c95b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616a5f0 0x55b9a616a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f40a3c95b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616af10 0x55b9a6164990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f40a3c95b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 0x55b9a61652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f40a3c95b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b9a60b67c0 con 0x55b9a616a5f0 2024-06-06T03:39:09.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f40a3c95b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b9a605e6d0 con 0x55b9a6164ed0 2024-06-06T03:39:09.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f40a3c95b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b9a60b2780 con 0x55b9a616af10 2024-06-06T03:39:09.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f4092ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616a5f0 0x55b9a616a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f4092ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616a5f0 0x55b9a616a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36018/0 (socket says 172.21.15.40:36018) 2024-06-06T03:39:09.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f4092ffd700 1 -- 172.21.15.40:0/4134265129 learned_addr learned my addr 172.21.15.40:0/4134265129 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:09.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f40927fc700 1 --2- 172.21.15.40:0/4134265129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616af10 0x55b9a6164990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f40937fe700 1 --2- 172.21.15.40:0/4134265129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 0x55b9a61652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f4092ffd700 1 -- 172.21.15.40:0/4134265129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616af10 msgr2=0x55b9a6164990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f4092ffd700 1 --2- 172.21.15.40:0/4134265129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616af10 0x55b9a6164990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f4092ffd700 1 -- 172.21.15.40:0/4134265129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 msgr2=0x55b9a61652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f4092ffd700 1 --2- 172.21.15.40:0/4134265129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 0x55b9a61652f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.263+0000 7f4092ffd700 1 -- 172.21.15.40:0/4134265129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a6054f00 con 0x55b9a616a5f0 2024-06-06T03:39:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.264+0000 7f4092ffd700 1 --2- 172.21.15.40:0/4134265129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616a5f0 0x55b9a616a9d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f40840a3d10 tx=0x7f408409b220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ab91e2b1406e5e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.264+0000 7f40797fa700 1 -- 172.21.15.40:0/4134265129 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084020070 con 0x55b9a616a5f0 2024-06-06T03:39:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.264+0000 7f40797fa700 1 -- 172.21.15.40:0/4134265129 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f408409f980 con 0x55b9a616a5f0 2024-06-06T03:39:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.264+0000 7f40797fa700 1 -- 172.21.15.40:0/4134265129 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40840a3fb0 con 0x55b9a616a5f0 2024-06-06T03:39:09.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.265+0000 7f40a3c95b80 1 -- 172.21.15.40:0/4134265129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616a5f0 msgr2=0x55b9a616a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:09.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.265+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/4134265129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616a5f0 0x55b9a616a9d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f40840a3d10 tx=0x7f408409b220 comp rx=0 tx=0).stop 2024-06-06T03:39:09.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.265+0000 7f40a3c95b80 1 -- 172.21.15.40:0/4134265129 shutdown_connections 2024-06-06T03:39:09.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.265+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/4134265129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 0x55b9a61652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.265+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/4134265129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616a5f0 0x55b9a616a9d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.265+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/4134265129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616af10 0x55b9a6164990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.265+0000 7f40a3c95b80 1 -- 172.21.15.40:0/4134265129 >> 172.21.15.40:0/4134265129 conn(0x55b9a608fc70 msgr2=0x55b9a615f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:09.266 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:09 smithi040 ceph-mon[32220]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T03:39:09.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.265+0000 7f40a3c95b80 1 -- 172.21.15.40:0/4134265129 shutdown_connections 2024-06-06T03:39:09.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.265+0000 7f40a3c95b80 1 -- 172.21.15.40:0/4134265129 wait complete. 2024-06-06T03:39:09.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.265+0000 7f40a3c95b80 1 -- start start 2024-06-06T03:39:09.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.265+0000 7f40a3c95b80 1 Processor -- start 2024-06-06T03:39:09.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f40a3c95b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a6164ed0 0x55b9a6170c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f40a3c95b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616a5f0 0x55b9a616df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f40a3c95b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a616af10 0x55b9a616e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f40a3c95b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b9a6171980 con 0x55b9a6164ed0 2024-06-06T03:39:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f40a3c95b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b9a6056ba0 con 0x55b9a616af10 2024-06-06T03:39:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f40a3c95b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b9a6171b00 con 0x55b9a616a5f0 2024-06-06T03:39:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f40a3c95b80 10 client.0 _ll_register_callbacks cb 0x55b9a60508d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:39:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f4092ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a6164ed0 0x55b9a6170c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f4092ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a6164ed0 0x55b9a6170c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36032/0 (socket says 172.21.15.40:36032) 2024-06-06T03:39:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f4092ffd700 1 -- 172.21.15.40:0/1226040596 learned_addr learned my addr 172.21.15.40:0/1226040596 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f40937fe700 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a616af10 0x55b9a616e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f40927fc700 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616a5f0 0x55b9a616df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f4092ffd700 1 -- 172.21.15.40:0/1226040596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616a5f0 msgr2=0x55b9a616df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f4092ffd700 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616a5f0 0x55b9a616df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f4092ffd700 1 -- 172.21.15.40:0/1226040596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a616af10 msgr2=0x55b9a616e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f4092ffd700 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a616af10 0x55b9a616e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f4092ffd700 1 -- 172.21.15.40:0/1226040596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a6005880 con 0x55b9a6164ed0 2024-06-06T03:39:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f40937fe700 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a616af10 0x55b9a616e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:39:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f40927fc700 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616a5f0 0x55b9a616df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:39:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f4092ffd700 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a6164ed0 0x55b9a6170c80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f40840cc9e0 tx=0x7f4084010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae17c9f87f5fe9b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.266+0000 7f4076ff5700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:39:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.267+0000 7f4076ff5700 1 -- 172.21.15.40:0/1226040596 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40840b8020 con 0x55b9a6164ed0 2024-06-06T03:39:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.267+0000 7f4076ff5700 1 -- 172.21.15.40:0/1226040596 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f408409ba40 con 0x55b9a6164ed0 2024-06-06T03:39:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.267+0000 7f40a3c95b80 1 -- 172.21.15.40:0/1226040596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f408409f980 con 0x55b9a6164ed0 2024-06-06T03:39:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.267+0000 7f4076ff5700 1 -- 172.21.15.40:0/1226040596 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4084020070 con 0x55b9a6164ed0 2024-06-06T03:39:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.267+0000 7f40a3c95b80 10 client.15384 fetch_fsmap learned FSMap version 29 2024-06-06T03:39:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.267+0000 7f40a3c95b80 1 -- 172.21.15.40:0/1226040596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9a6184a90 con 0x55b9a6164ed0 2024-06-06T03:39:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.267+0000 7f4076ff5700 1 -- 172.21.15.40:0/1226040596 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f408409f980 con 0x55b9a6164ed0 2024-06-06T03:39:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.267+0000 7f4076ff5700 1 -- 172.21.15.40:0/1226040596 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f408409ab40 con 0x55b9a6164ed0 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.267+0000 7f40a3c95b80 10 client.15384 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.267+0000 7f40a3c95b80 10 client.15384 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.267+0000 7f40a3c95b80 20 client.15384 populate_metadata read hostname 'smithi040' 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.267+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55b9a6185de0 0x55b9a6187f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.267+0000 7f40a3c95b80 4 client.15384 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.267+0000 7f40a3c95b80 1 -- 172.21.15.40:0/1226040596 --> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9a60c3cc0 con 0x55b9a6185de0 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.268+0000 7f40927fc700 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55b9a6185de0 0x55b9a6187f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.268+0000 7f4076ff5700 10 client.15384 ms_handle_connect on v2:172.21.15.104:6834/75462915 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.268+0000 7f40927fc700 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55b9a6185de0 0x55b9a6187f80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2e1264e2bb4f5e server_cookie=8ab715e4c9229b61 in_seq=0 out_seq=0 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f4076ff5700 1 -- 172.21.15.40:0/1226040596 <== mds.2 v2:172.21.15.104:6834/75462915 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b9a60c3cc0 con 0x55b9a6185de0 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f4076ff5700 10 client.15384 handle_command_reply: tid=0 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f40a3c95b80 1 client.15384 shutdown 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f40a3c95b80 1 -- 172.21.15.40:0/1226040596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a6164ed0 msgr2=0x55b9a6170c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a6164ed0 0x55b9a6170c80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f40840cc9e0 tx=0x7f4084010b50 comp rx=0 tx=0).stop 2024-06-06T03:39:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f40a3c95b80 1 -- 172.21.15.40:0/1226040596 shutdown_connections 2024-06-06T03:39:09.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55b9a6185de0 0x55b9a6187f80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a616af10 0x55b9a616e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a6164ed0 0x55b9a6170c80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/1226040596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616a5f0 0x55b9a616df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f4076ff5700 0 client.15384 ms_handle_reset on v2:172.21.15.104:6834/75462915 2024-06-06T03:39:09.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f40a3c95b80 1 -- 172.21.15.40:0/1226040596 >> 172.21.15.40:0/1226040596 conn(0x55b9a608fc70 msgr2=0x55b9a615f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:09.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f40a3c95b80 1 -- 172.21.15.40:0/1226040596 shutdown_connections 2024-06-06T03:39:09.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f40a3c95b80 1 -- 172.21.15.40:0/1226040596 wait complete. 2024-06-06T03:39:09.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.269+0000 7f40a3c95b80 20 client.15384 trim_cache size 0 max 16384 2024-06-06T03:39:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:09 smithi131 ceph-mon[36687]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T03:39:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.291+0000 7f40a3c95b80 1 Processor -- start 2024-06-06T03:39:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.291+0000 7f40a3c95b80 1 -- start start 2024-06-06T03:39:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.291+0000 7f40a3c95b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 0x55b9a6164990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.291+0000 7f40a3c95b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616a5f0 0x55b9a6186440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.291+0000 7f40a3c95b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616af10 0x55b9a616ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.291+0000 7f40a3c95b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b9a61862c0 con 0x55b9a616af10 2024-06-06T03:39:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.291+0000 7f40a3c95b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b9a6186990 con 0x55b9a6164ed0 2024-06-06T03:39:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.291+0000 7f40a3c95b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b9a6187060 con 0x55b9a616a5f0 2024-06-06T03:39:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.292+0000 7f40937fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616af10 0x55b9a616ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.292+0000 7f40937fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616af10 0x55b9a616ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36046/0 (socket says 172.21.15.40:36046) 2024-06-06T03:39:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.292+0000 7f4092ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 0x55b9a6164990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.292+0000 7f40937fe700 1 -- 172.21.15.40:0/473638206 learned_addr learned my addr 172.21.15.40:0/473638206 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.292+0000 7f40927fc700 1 --2- 172.21.15.40:0/473638206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616a5f0 0x55b9a6186440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.292+0000 7f40937fe700 1 -- 172.21.15.40:0/473638206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616a5f0 msgr2=0x55b9a6186440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.292+0000 7f40937fe700 1 --2- 172.21.15.40:0/473638206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616a5f0 0x55b9a6186440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.292+0000 7f40937fe700 1 -- 172.21.15.40:0/473638206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 msgr2=0x55b9a6164990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.292+0000 7f40937fe700 1 --2- 172.21.15.40:0/473638206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 0x55b9a6164990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.292+0000 7f40937fe700 1 -- 172.21.15.40:0/473638206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a6187c20 con 0x55b9a616af10 2024-06-06T03:39:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.292+0000 7f40937fe700 1 --2- 172.21.15.40:0/473638206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616af10 0x55b9a616ba00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f408c007630 tx=0x7f408c007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220a76d7b1d74702 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.292+0000 7f4078ff9700 1 -- 172.21.15.40:0/473638206 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c017610 con 0x55b9a616af10 2024-06-06T03:39:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.292+0000 7f4078ff9700 1 -- 172.21.15.40:0/473638206 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f408c007e10 con 0x55b9a616af10 2024-06-06T03:39:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.292+0000 7f4078ff9700 1 -- 172.21.15.40:0/473638206 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c01fcc0 con 0x55b9a616af10 2024-06-06T03:39:09.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.293+0000 7f40a3c95b80 1 -- 172.21.15.40:0/473638206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616af10 msgr2=0x55b9a616ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:09.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.293+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/473638206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616af10 0x55b9a616ba00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f408c007630 tx=0x7f408c007660 comp rx=0 tx=0).stop 2024-06-06T03:39:09.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.293+0000 7f40a3c95b80 1 -- 172.21.15.40:0/473638206 shutdown_connections 2024-06-06T03:39:09.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.293+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/473638206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616af10 0x55b9a616ba00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.293+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/473638206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 0x55b9a6164990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.293+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/473638206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616a5f0 0x55b9a6186440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.293+0000 7f40a3c95b80 1 -- 172.21.15.40:0/473638206 >> 172.21.15.40:0/473638206 conn(0x55b9a608fc70 msgr2=0x55b9a616b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.293+0000 7f40a3c95b80 1 -- 172.21.15.40:0/473638206 shutdown_connections 2024-06-06T03:39:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.293+0000 7f40a3c95b80 1 -- 172.21.15.40:0/473638206 wait complete. 2024-06-06T03:39:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.293+0000 7f40a3c95b80 1 -- start start 2024-06-06T03:39:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40a3c95b80 1 Processor -- start 2024-06-06T03:39:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40a3c95b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 0x55b9a61917c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40a3c95b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616a5f0 0x55b9a6163980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40a3c95b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616af10 0x55b9a6163ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40a3c95b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b9a6187e00 con 0x55b9a616a5f0 2024-06-06T03:39:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40a3c95b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b9a6187730 con 0x55b9a6164ed0 2024-06-06T03:39:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40a3c95b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b9a61884d0 con 0x55b9a616af10 2024-06-06T03:39:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40a3c95b80 10 client.0 _ll_register_callbacks cb 0x55b9a60508d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:39:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40927fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616a5f0 0x55b9a6163980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40927fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616a5f0 0x55b9a6163980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36060/0 (socket says 172.21.15.40:36060) 2024-06-06T03:39:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40927fc700 1 -- 172.21.15.40:0/3604674910 learned_addr learned my addr 172.21.15.40:0/3604674910 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f4092ffd700 1 --2- 172.21.15.40:0/3604674910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 0x55b9a61917c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40927fc700 1 -- 172.21.15.40:0/3604674910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616af10 msgr2=0x55b9a6163ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40937fe700 1 --2- 172.21.15.40:0/3604674910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616af10 0x55b9a6163ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40927fc700 1 --2- 172.21.15.40:0/3604674910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616af10 0x55b9a6163ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40927fc700 1 -- 172.21.15.40:0/3604674910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 msgr2=0x55b9a61917c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40927fc700 1 --2- 172.21.15.40:0/3604674910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 0x55b9a61917c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40927fc700 1 -- 172.21.15.40:0/3604674910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a6164400 con 0x55b9a616a5f0 2024-06-06T03:39:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40937fe700 1 --2- 172.21.15.40:0/3604674910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616af10 0x55b9a6163ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:39:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f40927fc700 1 --2- 172.21.15.40:0/3604674910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616a5f0 0x55b9a6163980 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4088009680 tx=0x7f408801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf1c16309e731 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.294+0000 7f4076ff5700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:39:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.295+0000 7f4076ff5700 1 -- 172.21.15.40:0/3604674910 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408800fdd0 con 0x55b9a616a5f0 2024-06-06T03:39:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.295+0000 7f40a3c95b80 1 -- 172.21.15.40:0/3604674910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f408c007e10 con 0x55b9a616a5f0 2024-06-06T03:39:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.295+0000 7f4076ff5700 1 -- 172.21.15.40:0/3604674910 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f4088024410 con 0x55b9a616a5f0 2024-06-06T03:39:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.295+0000 7f4076ff5700 1 -- 172.21.15.40:0/3604674910 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4088024610 con 0x55b9a616a5f0 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.295+0000 7f40a3c95b80 10 client.15390 fetch_fsmap learned FSMap version 29 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.295+0000 7f40a3c95b80 1 -- 172.21.15.40:0/3604674910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9a6191250 con 0x55b9a616a5f0 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.295+0000 7f4076ff5700 1 -- 172.21.15.40:0/3604674910 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f408c007e10 con 0x55b9a616a5f0 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.295+0000 7f4076ff5700 1 -- 172.21.15.40:0/3604674910 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4088045860 con 0x55b9a616a5f0 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.295+0000 7f40a3c95b80 10 client.15390 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.295+0000 7f40a3c95b80 10 client.15390 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.295+0000 7f40a3c95b80 20 client.15390 populate_metadata read hostname 'smithi040' 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.295+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/3604674910 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55b9a61f6d60 0x55b9a61f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.295+0000 7f40a3c95b80 4 client.15390 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.295+0000 7f40a3c95b80 1 -- 172.21.15.40:0/3604674910 --> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] -- command(tid 0: {"prefix": "status"}) -- 0x55b9a60be240 con 0x55b9a61f6d60 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f4092ffd700 1 --2- 172.21.15.40:0/3604674910 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55b9a61f6d60 0x55b9a61f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f4092ffd700 1 --2- 172.21.15.40:0/3604674910 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55b9a61f6d60 0x55b9a61f5a70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6df5e6be1e29a5f5 server_cookie=acf11c635f3bddd9 in_seq=0 out_seq=0 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f4076ff5700 10 client.15390 ms_handle_connect on v2:172.21.15.104:6834/75462915 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f4076ff5700 1 -- 172.21.15.40:0/3604674910 <== mds.2 v2:172.21.15.104:6834/75462915 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b9a60be240 con 0x55b9a61f6d60 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f4076ff5700 10 client.15390 handle_command_reply: tid=0 2024-06-06T03:39:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f40a3c95b80 1 client.15390 shutdown 2024-06-06T03:39:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f40a3c95b80 1 -- 172.21.15.40:0/3604674910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616a5f0 msgr2=0x55b9a6163980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/3604674910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616a5f0 0x55b9a6163980 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4088009680 tx=0x7f408801a050 comp rx=0 tx=0).stop 2024-06-06T03:39:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f40a3c95b80 1 -- 172.21.15.40:0/3604674910 shutdown_connections 2024-06-06T03:39:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/3604674910 >> [v2:172.21.15.104:6834/75462915,v1:172.21.15.104:6835/75462915] conn(0x55b9a61f6d60 0x55b9a61f5a70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f4076ff5700 0 client.15390 ms_handle_reset on v2:172.21.15.104:6834/75462915 2024-06-06T03:39:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/3604674910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b9a616af10 0x55b9a6163ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/3604674910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b9a6164ed0 0x55b9a61917c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f40a3c95b80 1 --2- 172.21.15.40:0/3604674910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b9a616a5f0 0x55b9a6163980 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.296+0000 7f40a3c95b80 1 -- 172.21.15.40:0/3604674910 >> 172.21.15.40:0/3604674910 conn(0x55b9a608fc70 msgr2=0x55b9a616b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.297+0000 7f40a3c95b80 1 -- 172.21.15.40:0/3604674910 shutdown_connections 2024-06-06T03:39:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.297+0000 7f40a3c95b80 1 -- 172.21.15.40:0/3604674910 wait complete. 2024-06-06T03:39:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.297+0000 7f40a3c95b80 20 client.15390 trim_cache size 0 max 16384 2024-06-06T03:39:09.300 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:39:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:39:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 2, 2024-06-06T03:39:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24425, 2024-06-06T03:39:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:39:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:39:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:39:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 217.38336928999999, 2024-06-06T03:39:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 29, 2024-06-06T03:39:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 91, 2024-06-06T03:39:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 90, 2024-06-06T03:39:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 272.32746580200001 2024-06-06T03:39:09.301 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:39:09.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.297+0000 7f4099412700 1 -- 172.21.15.40:0/1433048508 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f407c0a2eb0 msgr2=0x7f407c0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:09.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.297+0000 7f4099412700 1 --2- 172.21.15.40:0/1433048508 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f407c0a2eb0 0x7f407c0a5370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4088000c00 tx=0x7f4088005fb0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.297+0000 7f4099412700 1 -- 172.21.15.40:0/1433048508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40941015d0 msgr2=0x7f409419e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:09.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.297+0000 7f4099412700 1 --2- 172.21.15.40:0/1433048508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40941015d0 0x7f409419e8a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f408400d950 tx=0x7f408400dc60 comp rx=0 tx=0).stop 2024-06-06T03:39:09.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.297+0000 7f4099412700 1 -- 172.21.15.40:0/1433048508 shutdown_connections 2024-06-06T03:39:09.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.297+0000 7f4099412700 1 --2- 172.21.15.40:0/1433048508 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f407c0a2eb0 0x7f407c0a5370 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.297+0000 7f4099412700 1 --2- 172.21.15.40:0/1433048508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40941066b0 0x7f4094198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.297+0000 7f4099412700 1 --2- 172.21.15.40:0/1433048508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40941015d0 0x7f409419e8a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.297+0000 7f4099412700 1 --2- 172.21.15.40:0/1433048508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4094101f80 0x7f409419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:09.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.297+0000 7f4099412700 1 -- 172.21.15.40:0/1433048508 >> 172.21.15.40:0/1433048508 conn(0x7f4094078400 msgr2=0x7f4094105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:09.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.297+0000 7f4099412700 1 -- 172.21.15.40:0/1433048508 shutdown_connections 2024-06-06T03:39:09.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:09.298+0000 7f4099412700 1 -- 172.21.15.40:0/1433048508 wait complete. 2024-06-06T03:39:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:09 smithi104 ceph-mon[36569]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T03:39:10.023 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-06T03:39:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:39:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:11 smithi131 ceph-mon[36687]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-06T03:39:11.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:11 smithi040 ceph-mon[32220]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-06T03:39:11.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:11 smithi104 ceph-mon[36569]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-06T03:39:11.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.994+0000 7f291aa50700 1 -- 172.21.15.40:0/1578356289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914101d30 msgr2=0x7f291410e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:11.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.994+0000 7f291aa50700 1 --2- 172.21.15.40:0/1578356289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914101d30 0x7f291410e1d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f290c00b0a0 tx=0x7f290c00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:39:11.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.994+0000 7f291aa50700 1 -- 172.21.15.40:0/1578356289 shutdown_connections 2024-06-06T03:39:11.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.994+0000 7f291aa50700 1 --2- 172.21.15.40:0/1578356289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2914107420 0x7f2914107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:11.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.994+0000 7f291aa50700 1 --2- 172.21.15.40:0/1578356289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914101d30 0x7f291410e1d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:11.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.994+0000 7f291aa50700 1 --2- 172.21.15.40:0/1578356289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2914101370 0x7f29141017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:11.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.994+0000 7f291aa50700 1 -- 172.21.15.40:0/1578356289 >> 172.21.15.40:0/1578356289 conn(0x7f2914078340 msgr2=0x7f2914078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:11.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.994+0000 7f291aa50700 1 -- 172.21.15.40:0/1578356289 shutdown_connections 2024-06-06T03:39:11.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.994+0000 7f291aa50700 1 -- 172.21.15.40:0/1578356289 wait complete. 2024-06-06T03:39:11.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.995+0000 7f291aa50700 1 Processor -- start 2024-06-06T03:39:11.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.995+0000 7f291aa50700 1 -- start start 2024-06-06T03:39:11.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.995+0000 7f291aa50700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914101370 0x7f2914199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:11.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.995+0000 7f291aa50700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2914101d30 0x7f2914199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:11.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.995+0000 7f291aa50700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2914107420 0x7f29141a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.995+0000 7f291aa50700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2914111650 con 0x7f2914101370 2024-06-06T03:39:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.995+0000 7f291aa50700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f29141114d0 con 0x7f2914107420 2024-06-06T03:39:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.995+0000 7f291aa50700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f29141117d0 con 0x7f2914101d30 2024-06-06T03:39:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.995+0000 7f2913fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914101370 0x7f2914199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.995+0000 7f2913fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914101370 0x7f2914199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36080/0 (socket says 172.21.15.40:36080) 2024-06-06T03:39:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.995+0000 7f2913fff700 1 -- 172.21.15.40:0/46355613 learned_addr learned my addr 172.21.15.40:0/46355613 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.995+0000 7f2918fed700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2914107420 0x7f29141a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.995+0000 7f290bfff700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2914101d30 0x7f2914199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.996+0000 7f2913fff700 1 -- 172.21.15.40:0/46355613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2914101d30 msgr2=0x7f2914199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.996+0000 7f2913fff700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2914101d30 0x7f2914199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.996+0000 7f2913fff700 1 -- 172.21.15.40:0/46355613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2914107420 msgr2=0x7f29141a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.996+0000 7f2913fff700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2914107420 0x7f29141a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.996+0000 7f2913fff700 1 -- 172.21.15.40:0/46355613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29141a4f50 con 0x7f2914101370 2024-06-06T03:39:11.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.996+0000 7f290bfff700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2914101d30 0x7f2914199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:39:11.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.996+0000 7f2913fff700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914101370 0x7f2914199990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f290400e9c0 tx=0x7f290400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:11.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.996+0000 7f2911ffb700 1 -- 172.21.15.40:0/46355613 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290400cbd0 con 0x7f2914101370 2024-06-06T03:39:11.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.996+0000 7f291aa50700 1 -- 172.21.15.40:0/46355613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29141a5180 con 0x7f2914101370 2024-06-06T03:39:11.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.996+0000 7f2911ffb700 1 -- 172.21.15.40:0/46355613 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f290400ee60 con 0x7f2914101370 2024-06-06T03:39:11.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.996+0000 7f2911ffb700 1 -- 172.21.15.40:0/46355613 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904005630 con 0x7f2914101370 2024-06-06T03:39:11.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.996+0000 7f291aa50700 1 -- 172.21.15.40:0/46355613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29141a5690 con 0x7f2914101370 2024-06-06T03:39:12.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:11.999+0000 7f2911ffb700 1 -- 172.21.15.40:0/46355613 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f2904004750 con 0x7f2914101370 2024-06-06T03:39:12.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.000+0000 7f2911ffb700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f29000a2eb0 0x7f29000a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.000+0000 7f290bfff700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f29000a2eb0 0x7f29000a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.000+0000 7f2911ffb700 1 -- 172.21.15.40:0/46355613 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f29040cf8c0 con 0x7f2914101370 2024-06-06T03:39:12.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.000+0000 7f29252d3b80 1 Processor -- start 2024-06-06T03:39:12.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.000+0000 7f29252d3b80 1 -- start start 2024-06-06T03:39:12.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.000+0000 7f290bfff700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f29000a2eb0 0x7f29000a5370 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2914078a20 tx=0x7f28fc00b410 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:12.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.000+0000 7f29252d3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8cf10 0x55902ff8d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.000+0000 7f29252d3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff8d830 0x55902ff87210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f29252d3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff87750 0x55902ff87b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f29252d3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55902fe79f10 con 0x55902ff8d830 2024-06-06T03:39:12.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f29252d3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55902fed8800 con 0x55902ff8cf10 2024-06-06T03:39:12.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f29252d3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55902fe40410 con 0x55902ff87750 2024-06-06T03:39:12.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f290bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff8d830 0x55902ff87210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f290bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff8d830 0x55902ff87210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36090/0 (socket says 172.21.15.40:36090) 2024-06-06T03:39:12.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f290bfff700 1 -- 172.21.15.40:0/2767420339 learned_addr learned my addr 172.21.15.40:0/2767420339 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:12.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f2913fff700 1 --2- 172.21.15.40:0/2767420339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8cf10 0x55902ff8d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f290bfff700 1 -- 172.21.15.40:0/2767420339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff87750 msgr2=0x55902ff87b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:12.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f2918fed700 1 --2- 172.21.15.40:0/2767420339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff87750 0x55902ff87b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f290bfff700 1 --2- 172.21.15.40:0/2767420339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff87750 0x55902ff87b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f290bfff700 1 -- 172.21.15.40:0/2767420339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8cf10 msgr2=0x55902ff8d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:12.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f290bfff700 1 --2- 172.21.15.40:0/2767420339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8cf10 0x55902ff8d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f290bfff700 1 -- 172.21.15.40:0/2767420339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55902fe27880 con 0x55902ff8d830 2024-06-06T03:39:12.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f2913fff700 1 --2- 172.21.15.40:0/2767420339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8cf10 0x55902ff8d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:39:12.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f2918fed700 1 --2- 172.21.15.40:0/2767420339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff87750 0x55902ff87b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:39:12.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f290bfff700 1 --2- 172.21.15.40:0/2767420339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff8d830 0x55902ff87210 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f28fc011760 tx=0x7f28fc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5203882983b3f338 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:12.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f290a7fc700 1 -- 172.21.15.40:0/2767420339 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc01e620 con 0x55902ff8d830 2024-06-06T03:39:12.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f290a7fc700 1 -- 172.21.15.40:0/2767420339 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28fc010e70 con 0x55902ff8d830 2024-06-06T03:39:12.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.001+0000 7f290a7fc700 1 -- 172.21.15.40:0/2767420339 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc01cc20 con 0x55902ff8d830 2024-06-06T03:39:12.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.002+0000 7f29252d3b80 1 -- 172.21.15.40:0/2767420339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff8d830 msgr2=0x55902ff87210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:12.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.002+0000 7f29252d3b80 1 --2- 172.21.15.40:0/2767420339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff8d830 0x55902ff87210 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f28fc011760 tx=0x7f28fc013000 comp rx=0 tx=0).stop 2024-06-06T03:39:12.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.002+0000 7f29252d3b80 1 -- 172.21.15.40:0/2767420339 shutdown_connections 2024-06-06T03:39:12.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.002+0000 7f29252d3b80 1 --2- 172.21.15.40:0/2767420339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff87750 0x55902ff87b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.002+0000 7f29252d3b80 1 --2- 172.21.15.40:0/2767420339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8cf10 0x55902ff8d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.002+0000 7f29252d3b80 1 --2- 172.21.15.40:0/2767420339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff8d830 0x55902ff87210 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.002+0000 7f29252d3b80 1 -- 172.21.15.40:0/2767420339 >> 172.21.15.40:0/2767420339 conn(0x55902feb4030 msgr2=0x55902ff821d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:12.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.002+0000 7f29252d3b80 1 -- 172.21.15.40:0/2767420339 shutdown_connections 2024-06-06T03:39:12.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.002+0000 7f29252d3b80 1 -- 172.21.15.40:0/2767420339 wait complete. 2024-06-06T03:39:12.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.003+0000 7f29252d3b80 1 -- start start 2024-06-06T03:39:12.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.003+0000 7f29252d3b80 1 Processor -- start 2024-06-06T03:39:12.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.003+0000 7f29252d3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff87750 0x55902ff935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.003+0000 7f29252d3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8cf10 0x55902ff908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.003+0000 7f29252d3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff8d830 0x55902ff90e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.003+0000 7f29252d3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55902fe78f70 con 0x55902ff8d830 2024-06-06T03:39:12.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.003+0000 7f29252d3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55902fed47c0 con 0x55902ff87750 2024-06-06T03:39:12.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.003+0000 7f29252d3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55902ff94210 con 0x55902ff8cf10 2024-06-06T03:39:12.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.003+0000 7f29252d3b80 10 client.0 _ll_register_callbacks cb 0x55902fe6e560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:39:12.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.003+0000 7f2918fed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff8d830 0x55902ff90e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.003+0000 7f2918fed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff8d830 0x55902ff90e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36104/0 (socket says 172.21.15.40:36104) 2024-06-06T03:39:12.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.003+0000 7f2918fed700 1 -- 172.21.15.40:0/736865757 learned_addr learned my addr 172.21.15.40:0/736865757 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:12.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.003+0000 7f2913fff700 1 --2- 172.21.15.40:0/736865757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff87750 0x55902ff935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.004+0000 7f290bfff700 1 --2- 172.21.15.40:0/736865757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8cf10 0x55902ff908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.004+0000 7f2918fed700 1 -- 172.21.15.40:0/736865757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8cf10 msgr2=0x55902ff908e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:12.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.004+0000 7f2918fed700 1 --2- 172.21.15.40:0/736865757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8cf10 0x55902ff908e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.004+0000 7f2918fed700 1 -- 172.21.15.40:0/736865757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff87750 msgr2=0x55902ff935e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:12.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.004+0000 7f2918fed700 1 --2- 172.21.15.40:0/736865757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff87750 0x55902ff935e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.004+0000 7f2918fed700 1 -- 172.21.15.40:0/736865757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55902fe9e700 con 0x55902ff8d830 2024-06-06T03:39:12.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.004+0000 7f2918fed700 1 --2- 172.21.15.40:0/736865757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff8d830 0x55902ff90e20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f290c00bd80 tx=0x7f290c00bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7711a6a97364abe0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:12.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.004+0000 7f28f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:39:12.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.004+0000 7f28f37fe700 1 -- 172.21.15.40:0/736865757 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c0074e0 con 0x55902ff8d830 2024-06-06T03:39:12.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.004+0000 7f28f37fe700 1 -- 172.21.15.40:0/736865757 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f290c007680 con 0x55902ff8d830 2024-06-06T03:39:12.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.004+0000 7f29252d3b80 1 -- 172.21.15.40:0/736865757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28fc010e70 con 0x55902ff8d830 2024-06-06T03:39:12.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.004+0000 7f28f37fe700 1 -- 172.21.15.40:0/736865757 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f290c0078c0 con 0x55902ff8d830 2024-06-06T03:39:12.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.004+0000 7f29252d3b80 10 client.15402 fetch_fsmap learned FSMap version 29 2024-06-06T03:39:12.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.004+0000 7f29252d3b80 1 -- 172.21.15.40:0/736865757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55902ff9e940 con 0x55902ff8d830 2024-06-06T03:39:12.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.005+0000 7f28f37fe700 1 -- 172.21.15.40:0/736865757 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28fc010e70 con 0x55902ff8d830 2024-06-06T03:39:12.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.005+0000 7f28f37fe700 1 -- 172.21.15.40:0/736865757 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f290c011070 con 0x55902ff8d830 2024-06-06T03:39:12.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.005+0000 7f29252d3b80 10 client.15402 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T03:39:12.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.005+0000 7f29252d3b80 10 client.15402 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T03:39:12.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.005+0000 7f29252d3b80 20 client.15402 populate_metadata read hostname 'smithi040' 2024-06-06T03:39:12.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.005+0000 7f29252d3b80 1 --2- 172.21.15.40:0/736865757 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x55902ffa0f40 0x55902ffa23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.005+0000 7f29252d3b80 4 client.15402 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:39:12.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.005+0000 7f29252d3b80 1 -- 172.21.15.40:0/736865757 --> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55902fee5d00 con 0x55902ffa0f40 2024-06-06T03:39:12.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.005+0000 7f2913fff700 1 --2- 172.21.15.40:0/736865757 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x55902ffa0f40 0x55902ffa23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.005+0000 7f2913fff700 1 --2- 172.21.15.40:0/736865757 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x55902ffa0f40 0x55902ffa23b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=adb50737d9087d8f server_cookie=96d4f5471314a1c8 in_seq=0 out_seq=0 2024-06-06T03:39:12.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.005+0000 7f28f37fe700 10 client.15402 ms_handle_connect on v2:172.21.15.104:6836/2498466212 2024-06-06T03:39:12.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.006+0000 7f28f37fe700 1 -- 172.21.15.40:0/736865757 <== mds.1 v2:172.21.15.104:6836/2498466212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55902fee5d00 con 0x55902ffa0f40 2024-06-06T03:39:12.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.006+0000 7f28f37fe700 10 client.15402 handle_command_reply: tid=0 2024-06-06T03:39:12.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.006+0000 7f29252d3b80 1 client.15402 shutdown 2024-06-06T03:39:12.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.006+0000 7f29252d3b80 1 -- 172.21.15.40:0/736865757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff8d830 msgr2=0x55902ff90e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:12.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.006+0000 7f29252d3b80 1 --2- 172.21.15.40:0/736865757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff8d830 0x55902ff90e20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f290c00bd80 tx=0x7f290c00bdb0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.006+0000 7f29252d3b80 1 -- 172.21.15.40:0/736865757 shutdown_connections 2024-06-06T03:39:12.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.006+0000 7f29252d3b80 1 --2- 172.21.15.40:0/736865757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff8d830 0x55902ff90e20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.006+0000 7f29252d3b80 1 --2- 172.21.15.40:0/736865757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff87750 0x55902ff935e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.006+0000 7f29252d3b80 1 --2- 172.21.15.40:0/736865757 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x55902ffa0f40 0x55902ffa23b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.007+0000 7f29252d3b80 1 --2- 172.21.15.40:0/736865757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8cf10 0x55902ff908e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.007+0000 7f28f37fe700 0 client.15402 ms_handle_reset on v2:172.21.15.104:6836/2498466212 2024-06-06T03:39:12.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.007+0000 7f29252d3b80 1 -- 172.21.15.40:0/736865757 >> 172.21.15.40:0/736865757 conn(0x55902feb4030 msgr2=0x55902ff816b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:12.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.007+0000 7f29252d3b80 1 -- 172.21.15.40:0/736865757 shutdown_connections 2024-06-06T03:39:12.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.007+0000 7f29252d3b80 1 -- 172.21.15.40:0/736865757 wait complete. 2024-06-06T03:39:12.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.007+0000 7f29252d3b80 20 client.15402 trim_cache size 0 max 16384 2024-06-06T03:39:12.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f29252d3b80 1 Processor -- start 2024-06-06T03:39:12.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f29252d3b80 1 -- start start 2024-06-06T03:39:12.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f29252d3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff87750 0x55902ff90fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f29252d3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8cf10 0x55902ff81e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f29252d3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8d830 0x55902ff82370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f29252d3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55902ffa06f0 con 0x55902ff87750 2024-06-06T03:39:12.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f29252d3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55902ffa0dc0 con 0x55902ff8d830 2024-06-06T03:39:12.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f29252d3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55902ffa1490 con 0x55902ff8cf10 2024-06-06T03:39:12.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f2913fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff87750 0x55902ff90fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f2913fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff87750 0x55902ff90fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36120/0 (socket says 172.21.15.40:36120) 2024-06-06T03:39:12.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f2913fff700 1 -- 172.21.15.40:0/999988857 learned_addr learned my addr 172.21.15.40:0/999988857 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:12.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f290bfff700 1 --2- 172.21.15.40:0/999988857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8cf10 0x55902ff81e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f2918fed700 1 --2- 172.21.15.40:0/999988857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8d830 0x55902ff82370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f2913fff700 1 -- 172.21.15.40:0/999988857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8cf10 msgr2=0x55902ff81e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:12.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f2913fff700 1 --2- 172.21.15.40:0/999988857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8cf10 0x55902ff81e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f2913fff700 1 -- 172.21.15.40:0/999988857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8d830 msgr2=0x55902ff82370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:12.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f2913fff700 1 --2- 172.21.15.40:0/999988857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8d830 0x55902ff82370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f2913fff700 1 -- 172.21.15.40:0/999988857 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55902ffa3310 con 0x55902ff87750 2024-06-06T03:39:12.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.029+0000 7f2913fff700 1 --2- 172.21.15.40:0/999988857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff87750 0x55902ff90fb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2904009fd0 tx=0x7f290400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f07a5d5a2143c68 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:12.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.030+0000 7f2909ffb700 1 -- 172.21.15.40:0/999988857 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290409ff70 con 0x55902ff87750 2024-06-06T03:39:12.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.030+0000 7f2909ffb700 1 -- 172.21.15.40:0/999988857 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29040a4830 con 0x55902ff87750 2024-06-06T03:39:12.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.030+0000 7f2909ffb700 1 -- 172.21.15.40:0/999988857 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29040a4600 con 0x55902ff87750 2024-06-06T03:39:12.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.030+0000 7f29252d3b80 1 -- 172.21.15.40:0/999988857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff87750 msgr2=0x55902ff90fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:12.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.030+0000 7f29252d3b80 1 --2- 172.21.15.40:0/999988857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff87750 0x55902ff90fb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2904009fd0 tx=0x7f290400cf90 comp rx=0 tx=0).stop 2024-06-06T03:39:12.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.030+0000 7f29252d3b80 1 -- 172.21.15.40:0/999988857 shutdown_connections 2024-06-06T03:39:12.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.030+0000 7f29252d3b80 1 --2- 172.21.15.40:0/999988857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8d830 0x55902ff82370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.030+0000 7f29252d3b80 1 --2- 172.21.15.40:0/999988857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff87750 0x55902ff90fb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.030+0000 7f29252d3b80 1 --2- 172.21.15.40:0/999988857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8cf10 0x55902ff81e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.030+0000 7f29252d3b80 1 -- 172.21.15.40:0/999988857 >> 172.21.15.40:0/999988857 conn(0x55902feb4030 msgr2=0x55902ff908a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:12.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.030+0000 7f29252d3b80 1 -- 172.21.15.40:0/999988857 shutdown_connections 2024-06-06T03:39:12.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.030+0000 7f29252d3b80 1 -- 172.21.15.40:0/999988857 wait complete. 2024-06-06T03:39:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.031+0000 7f29252d3b80 1 -- start start 2024-06-06T03:39:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.031+0000 7f29252d3b80 1 Processor -- start 2024-06-06T03:39:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.031+0000 7f29252d3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff87750 0x55902ffaba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.031+0000 7f29252d3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8cf10 0x55902ffabf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.031+0000 7f29252d3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8d830 0x55902ffa67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.031+0000 7f29252d3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55902ffa2230 con 0x55902ff87750 2024-06-06T03:39:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.031+0000 7f29252d3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55902ffa1b60 con 0x55902ff8cf10 2024-06-06T03:39:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.031+0000 7f29252d3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55902ffa2900 con 0x55902ff8d830 2024-06-06T03:39:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.031+0000 7f29252d3b80 10 client.0 _ll_register_callbacks cb 0x55902fe6e560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:39:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.031+0000 7f2913fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff87750 0x55902ffaba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.031+0000 7f2913fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff87750 0x55902ffaba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36134/0 (socket says 172.21.15.40:36134) 2024-06-06T03:39:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.031+0000 7f2913fff700 1 -- 172.21.15.40:0/1994743428 learned_addr learned my addr 172.21.15.40:0/1994743428 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f290bfff700 1 --2- 172.21.15.40:0/1994743428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8cf10 0x55902ffabf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f2918fed700 1 --2- 172.21.15.40:0/1994743428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8d830 0x55902ffa67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f2913fff700 1 -- 172.21.15.40:0/1994743428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8d830 msgr2=0x55902ffa67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f2913fff700 1 --2- 172.21.15.40:0/1994743428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8d830 0x55902ffa67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f2913fff700 1 -- 172.21.15.40:0/1994743428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8cf10 msgr2=0x55902ffabf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f2913fff700 1 --2- 172.21.15.40:0/1994743428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8cf10 0x55902ffabf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f2913fff700 1 -- 172.21.15.40:0/1994743428 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55902ffa6ff0 con 0x55902ff87750 2024-06-06T03:39:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f2913fff700 1 --2- 172.21.15.40:0/1994743428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff87750 0x55902ffaba10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2904098020 tx=0x7f290409b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a521a54df35dab1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f28f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:39:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f28f37fe700 1 -- 172.21.15.40:0/1994743428 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290409be20 con 0x55902ff87750 2024-06-06T03:39:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f28f37fe700 1 -- 172.21.15.40:0/1994743428 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f290409bfc0 con 0x55902ff87750 2024-06-06T03:39:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f29252d3b80 1 -- 172.21.15.40:0/1994743428 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29040a4830 con 0x55902ff87750 2024-06-06T03:39:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f28f37fe700 1 -- 172.21.15.40:0/1994743428 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29040a43a0 con 0x55902ff87750 2024-06-06T03:39:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f29252d3b80 10 client.15408 fetch_fsmap learned FSMap version 29 2024-06-06T03:39:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.032+0000 7f29252d3b80 1 -- 172.21.15.40:0/1994743428 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5590300116e0 con 0x55902ff87750 2024-06-06T03:39:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.033+0000 7f28f37fe700 1 -- 172.21.15.40:0/1994743428 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29040a4830 con 0x55902ff87750 2024-06-06T03:39:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.033+0000 7f28f37fe700 1 -- 172.21.15.40:0/1994743428 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f29040ba3d0 con 0x55902ff87750 2024-06-06T03:39:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.033+0000 7f29252d3b80 10 client.15408 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T03:39:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.033+0000 7f29252d3b80 10 client.15408 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T03:39:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.033+0000 7f29252d3b80 20 client.15408 populate_metadata read hostname 'smithi040' 2024-06-06T03:39:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.033+0000 7f29252d3b80 1 --2- 172.21.15.40:0/1994743428 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x559030015d60 0x559030014a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.033+0000 7f29252d3b80 4 client.15408 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:39:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.033+0000 7f29252d3b80 1 -- 172.21.15.40:0/1994743428 --> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] -- command(tid 0: {"prefix": "status"}) -- 0x55902fee0280 con 0x559030015d60 2024-06-06T03:39:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.033+0000 7f290bfff700 1 --2- 172.21.15.40:0/1994743428 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x559030015d60 0x559030014a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.033+0000 7f290bfff700 1 --2- 172.21.15.40:0/1994743428 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x559030015d60 0x559030014a70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=987595fcbdc03a10 server_cookie=298ba24276659ca3 in_seq=0 out_seq=0 2024-06-06T03:39:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.033+0000 7f28f37fe700 10 client.15408 ms_handle_connect on v2:172.21.15.104:6836/2498466212 2024-06-06T03:39:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f28f37fe700 1 -- 172.21.15.40:0/1994743428 <== mds.1 v2:172.21.15.104:6836/2498466212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55902fee0280 con 0x559030015d60 2024-06-06T03:39:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f28f37fe700 10 client.15408 handle_command_reply: tid=0 2024-06-06T03:39:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f29252d3b80 1 client.15408 shutdown 2024-06-06T03:39:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f29252d3b80 1 -- 172.21.15.40:0/1994743428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff87750 msgr2=0x55902ffaba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f29252d3b80 1 --2- 172.21.15.40:0/1994743428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff87750 0x55902ffaba10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2904098020 tx=0x7f290409b810 comp rx=0 tx=0).stop 2024-06-06T03:39:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f29252d3b80 1 -- 172.21.15.40:0/1994743428 shutdown_connections 2024-06-06T03:39:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f29252d3b80 1 --2- 172.21.15.40:0/1994743428 >> [v2:172.21.15.104:6836/2498466212,v1:172.21.15.104:6837/2498466212] conn(0x559030015d60 0x559030014a70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f28f37fe700 0 client.15408 ms_handle_reset on v2:172.21.15.104:6836/2498466212 2024-06-06T03:39:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f29252d3b80 1 --2- 172.21.15.40:0/1994743428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55902ff8d830 0x55902ffa67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f29252d3b80 1 --2- 172.21.15.40:0/1994743428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55902ff87750 0x55902ffaba10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f29252d3b80 1 --2- 172.21.15.40:0/1994743428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55902ff8cf10 0x55902ffabf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f29252d3b80 1 -- 172.21.15.40:0/1994743428 >> 172.21.15.40:0/1994743428 conn(0x55902feb4030 msgr2=0x55902ff8dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f29252d3b80 1 -- 172.21.15.40:0/1994743428 shutdown_connections 2024-06-06T03:39:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f29252d3b80 1 -- 172.21.15.40:0/1994743428 wait complete. 2024-06-06T03:39:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.034+0000 7f29252d3b80 20 client.15408 trim_cache size 0 max 16384 2024-06-06T03:39:12.039 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:39:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:39:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 1, 2024-06-06T03:39:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24443, 2024-06-06T03:39:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:39:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:39:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:39:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 221.135944064, 2024-06-06T03:39:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 29, 2024-06-06T03:39:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 91, 2024-06-06T03:39:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 90, 2024-06-06T03:39:12.040 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 266.584422147 2024-06-06T03:39:12.040 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:39:12.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.035+0000 7f291aa50700 1 -- 172.21.15.40:0/46355613 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f29000a2eb0 msgr2=0x7f29000a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:12.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.035+0000 7f291aa50700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f29000a2eb0 0x7f29000a5370 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2914078a20 tx=0x7f28fc00b410 comp rx=0 tx=0).stop 2024-06-06T03:39:12.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.035+0000 7f291aa50700 1 -- 172.21.15.40:0/46355613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914101370 msgr2=0x7f2914199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:12.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.035+0000 7f291aa50700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914101370 0x7f2914199990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f290400e9c0 tx=0x7f290400ecd0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.035+0000 7f291aa50700 1 -- 172.21.15.40:0/46355613 shutdown_connections 2024-06-06T03:39:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.035+0000 7f291aa50700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f29000a2eb0 0x7f29000a5370 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.035+0000 7f291aa50700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2914107420 0x7f29141a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.035+0000 7f291aa50700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914101370 0x7f2914199990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.035+0000 7f291aa50700 1 --2- 172.21.15.40:0/46355613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2914101d30 0x7f2914199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.035+0000 7f291aa50700 1 -- 172.21.15.40:0/46355613 >> 172.21.15.40:0/46355613 conn(0x7f2914078340 msgr2=0x7f29141001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.035+0000 7f291aa50700 1 -- 172.21.15.40:0/46355613 shutdown_connections 2024-06-06T03:39:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:12.035+0000 7f291aa50700 1 -- 172.21.15.40:0/46355613 wait complete. 2024-06-06T03:39:12.841 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T03:39:13.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:13 smithi040 ceph-mon[32220]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T03:39:13.098 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:39:13.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:13 smithi131 ceph-mon[36687]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T03:39:13.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:13 smithi104 ceph-mon[36569]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T03:39:14.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.780+0000 7fad4a110700 1 -- 172.21.15.40:0/237683413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad44101390 msgr2=0x7fad44101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:14.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.780+0000 7fad4a110700 1 --2- 172.21.15.40:0/237683413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad44101390 0x7fad44101810 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fad38009990 tx=0x7fad38009ca0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.780+0000 7fad4a110700 1 -- 172.21.15.40:0/237683413 shutdown_connections 2024-06-06T03:39:14.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.780+0000 7fad4a110700 1 --2- 172.21.15.40:0/237683413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad44101d50 0x7fad4410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.780+0000 7fad4a110700 1 --2- 172.21.15.40:0/237683413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad44107400 0x7fad441077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.780+0000 7fad4a110700 1 --2- 172.21.15.40:0/237683413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad44101390 0x7fad44101810 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.780+0000 7fad4a110700 1 -- 172.21.15.40:0/237683413 >> 172.21.15.40:0/237683413 conn(0x7fad44078340 msgr2=0x7fad44078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.780+0000 7fad4a110700 1 -- 172.21.15.40:0/237683413 shutdown_connections 2024-06-06T03:39:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.781+0000 7fad4a110700 1 -- 172.21.15.40:0/237683413 wait complete. 2024-06-06T03:39:14.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.781+0000 7fad4a110700 1 Processor -- start 2024-06-06T03:39:14.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad4a110700 1 -- start start 2024-06-06T03:39:14.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad4a110700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad44101390 0x7fad44199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad4a110700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad44101d50 0x7fad4419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad4a110700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad44107400 0x7fad441a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad4a110700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fad441116f0 con 0x7fad44101390 2024-06-06T03:39:14.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad4a110700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fad44111570 con 0x7fad44101d50 2024-06-06T03:39:14.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad4a110700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fad44111870 con 0x7fad44107400 2024-06-06T03:39:14.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad437fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad44101390 0x7fad44199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad437fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad44101390 0x7fad44199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36148/0 (socket says 172.21.15.40:36148) 2024-06-06T03:39:14.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad437fe700 1 -- 172.21.15.40:0/224787488 learned_addr learned my addr 172.21.15.40:0/224787488 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:14.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad43fff700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad44107400 0x7fad441a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad42ffd700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad44101d50 0x7fad4419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad437fe700 1 -- 172.21.15.40:0/224787488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad44107400 msgr2=0x7fad441a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:14.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad437fe700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad44107400 0x7fad441a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad437fe700 1 -- 172.21.15.40:0/224787488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad44101d50 msgr2=0x7fad4419a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:14.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad437fe700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad44101d50 0x7fad4419a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad437fe700 1 -- 172.21.15.40:0/224787488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad441a5130 con 0x7fad44101390 2024-06-06T03:39:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.782+0000 7fad43fff700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad44107400 0x7fad441a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:39:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.783+0000 7fad437fe700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad44101390 0x7fad44199b70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fad3400b530 tx=0x7fad3400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.783+0000 7fad40ff9700 1 -- 172.21.15.40:0/224787488 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34015020 con 0x7fad44101390 2024-06-06T03:39:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.783+0000 7fad40ff9700 1 -- 172.21.15.40:0/224787488 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad34011d10 con 0x7fad44101390 2024-06-06T03:39:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.783+0000 7fad40ff9700 1 -- 172.21.15.40:0/224787488 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34010e50 con 0x7fad44101390 2024-06-06T03:39:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.783+0000 7fad4a110700 1 -- 172.21.15.40:0/224787488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad441a5360 con 0x7fad44101390 2024-06-06T03:39:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.783+0000 7fad4a110700 1 -- 172.21.15.40:0/224787488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad441a5820 con 0x7fad44101390 2024-06-06T03:39:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.785+0000 7fad40ff9700 1 -- 172.21.15.40:0/224787488 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7fad34018020 con 0x7fad44101390 2024-06-06T03:39:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.786+0000 7fad40ff9700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fad2c0a30d0 0x7fad2c0a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.786+0000 7fad42ffd700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fad2c0a30d0 0x7fad2c0a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.786+0000 7fad40ff9700 1 -- 172.21.15.40:0/224787488 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fad340ced40 con 0x7fad44101390 2024-06-06T03:39:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.786+0000 7fad42ffd700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fad2c0a30d0 0x7fad2c0a5590 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fad38006010 tx=0x7fad3800b540 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.786+0000 7fad54993b80 1 Processor -- start 2024-06-06T03:39:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad54993b80 1 -- start start 2024-06-06T03:39:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad54993b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2030f80 0x55bdc2031360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad54993b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc20318a0 0x55bdc2035670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad54993b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc2035bb0 0x55bdc2036010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad54993b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bdc1f7e7c0 con 0x55bdc20318a0 2024-06-06T03:39:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad54993b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bdc1f266d0 con 0x55bdc2030f80 2024-06-06T03:39:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad54993b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bdc1f7a780 con 0x55bdc2035bb0 2024-06-06T03:39:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad42ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc20318a0 0x55bdc2035670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad42ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc20318a0 0x55bdc2035670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36158/0 (socket says 172.21.15.40:36158) 2024-06-06T03:39:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad42ffd700 1 -- 172.21.15.40:0/1502490859 learned_addr learned my addr 172.21.15.40:0/1502490859 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad437fe700 1 --2- 172.21.15.40:0/1502490859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2030f80 0x55bdc2031360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad42ffd700 1 -- 172.21.15.40:0/1502490859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc2035bb0 msgr2=0x55bdc2036010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad42ffd700 1 --2- 172.21.15.40:0/1502490859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc2035bb0 0x55bdc2036010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad42ffd700 1 -- 172.21.15.40:0/1502490859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2030f80 msgr2=0x55bdc2031360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad42ffd700 1 --2- 172.21.15.40:0/1502490859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2030f80 0x55bdc2031360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad42ffd700 1 -- 172.21.15.40:0/1502490859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdc1f1cf00 con 0x55bdc20318a0 2024-06-06T03:39:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.787+0000 7fad42ffd700 1 --2- 172.21.15.40:0/1502490859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc20318a0 0x55bdc2035670 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fad38009990 tx=0x7fad3801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e6c20491fecc35 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.788+0000 7fad29ffb700 1 -- 172.21.15.40:0/1502490859 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38003780 con 0x55bdc20318a0 2024-06-06T03:39:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.788+0000 7fad29ffb700 1 -- 172.21.15.40:0/1502490859 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad3800be40 con 0x55bdc20318a0 2024-06-06T03:39:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.788+0000 7fad29ffb700 1 -- 172.21.15.40:0/1502490859 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad380247d0 con 0x55bdc20318a0 2024-06-06T03:39:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.789+0000 7fad54993b80 1 -- 172.21.15.40:0/1502490859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc20318a0 msgr2=0x55bdc2035670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.789+0000 7fad54993b80 1 --2- 172.21.15.40:0/1502490859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc20318a0 0x55bdc2035670 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fad38009990 tx=0x7fad3801f000 comp rx=0 tx=0).stop 2024-06-06T03:39:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.789+0000 7fad54993b80 1 -- 172.21.15.40:0/1502490859 shutdown_connections 2024-06-06T03:39:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.789+0000 7fad54993b80 1 --2- 172.21.15.40:0/1502490859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc2035bb0 0x55bdc2036010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.789+0000 7fad54993b80 1 --2- 172.21.15.40:0/1502490859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2030f80 0x55bdc2031360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.789+0000 7fad54993b80 1 --2- 172.21.15.40:0/1502490859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc20318a0 0x55bdc2035670 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.789+0000 7fad54993b80 1 -- 172.21.15.40:0/1502490859 >> 172.21.15.40:0/1502490859 conn(0x55bdc1f57c70 msgr2=0x55bdc2027800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.789+0000 7fad54993b80 1 -- 172.21.15.40:0/1502490859 shutdown_connections 2024-06-06T03:39:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.789+0000 7fad54993b80 1 -- 172.21.15.40:0/1502490859 wait complete. 2024-06-06T03:39:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.789+0000 7fad54993b80 1 -- start start 2024-06-06T03:39:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.789+0000 7fad54993b80 1 Processor -- start 2024-06-06T03:39:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad54993b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2030f80 0x55bdc203d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad54993b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc20318a0 0x55bdc203a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad54993b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2035bb0 0x55bdc203a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad54993b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bdc203dcb0 con 0x55bdc2035bb0 2024-06-06T03:39:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad54993b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bdc1f1eba0 con 0x55bdc2030f80 2024-06-06T03:39:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad54993b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bdc203de30 con 0x55bdc20318a0 2024-06-06T03:39:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad54993b80 10 client.0 _ll_register_callbacks cb 0x55bdc1f188d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:39:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad437fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2030f80 0x55bdc203d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad43fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2035bb0 0x55bdc203a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad437fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2030f80 0x55bdc203d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:60024/0 (socket says 172.21.15.40:60024) 2024-06-06T03:39:14.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad437fe700 1 -- 172.21.15.40:0/3837377296 learned_addr learned my addr 172.21.15.40:0/3837377296 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:14.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad42ffd700 1 --2- 172.21.15.40:0/3837377296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc20318a0 0x55bdc203a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad43fff700 1 -- 172.21.15.40:0/3837377296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc20318a0 msgr2=0x55bdc203a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:14.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad43fff700 1 --2- 172.21.15.40:0/3837377296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc20318a0 0x55bdc203a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad43fff700 1 -- 172.21.15.40:0/3837377296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2030f80 msgr2=0x55bdc203d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:14.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad43fff700 1 --2- 172.21.15.40:0/3837377296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2030f80 0x55bdc203d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad43fff700 1 -- 172.21.15.40:0/3837377296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdc1ecd880 con 0x55bdc2035bb0 2024-06-06T03:39:14.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad437fe700 1 --2- 172.21.15.40:0/3837377296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2030f80 0x55bdc203d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:39:14.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad43fff700 1 --2- 172.21.15.40:0/3837377296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2035bb0 0x55bdc203a910 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fad3c007920 tx=0x7fad3c007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc6f7790028e369 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:14.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:39:14.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.790+0000 7fad22ffd700 1 -- 172.21.15.40:0/3837377296 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c0147e0 con 0x55bdc2035bb0 2024-06-06T03:39:14.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.791+0000 7fad22ffd700 1 -- 172.21.15.40:0/3837377296 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fad3c014980 con 0x55bdc2035bb0 2024-06-06T03:39:14.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.791+0000 7fad54993b80 1 -- 172.21.15.40:0/3837377296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad3800be40 con 0x55bdc2035bb0 2024-06-06T03:39:14.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.791+0000 7fad22ffd700 1 -- 172.21.15.40:0/3837377296 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fad3c014b80 con 0x55bdc2035bb0 2024-06-06T03:39:14.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.791+0000 7fad54993b80 10 client.15420 fetch_fsmap learned FSMap version 29 2024-06-06T03:39:14.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.791+0000 7fad54993b80 1 -- 172.21.15.40:0/3837377296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bdc2048610 con 0x55bdc2035bb0 2024-06-06T03:39:14.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.791+0000 7fad22ffd700 1 -- 172.21.15.40:0/3837377296 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad3800be40 con 0x55bdc2035bb0 2024-06-06T03:39:14.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.791+0000 7fad22ffd700 1 -- 172.21.15.40:0/3837377296 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fad3c01f4b0 con 0x55bdc2035bb0 2024-06-06T03:39:14.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.791+0000 7fad54993b80 10 client.15420 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T03:39:14.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.791+0000 7fad54993b80 10 client.15420 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T03:39:14.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.791+0000 7fad54993b80 20 client.15420 populate_metadata read hostname 'smithi040' 2024-06-06T03:39:14.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.791+0000 7fad54993b80 1 --2- 172.21.15.40:0/3837377296 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bdc204ac00 0x55bdc204c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.791+0000 7fad54993b80 4 client.15420 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:39:14.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.791+0000 7fad54993b80 1 -- 172.21.15.40:0/3837377296 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdc1f8bcc0 con 0x55bdc204ac00 2024-06-06T03:39:14.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.792+0000 7fad437fe700 1 --2- 172.21.15.40:0/3837377296 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bdc204ac00 0x55bdc204c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.792+0000 7fad22ffd700 10 client.15420 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:39:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.792+0000 7fad437fe700 1 --2- 172.21.15.40:0/3837377296 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bdc204ac00 0x55bdc204c070 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e63345b8c7834f18 server_cookie=34046faace6d8116 in_seq=0 out_seq=0 2024-06-06T03:39:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad22ffd700 1 -- 172.21.15.40:0/3837377296 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bdc1f8bcc0 con 0x55bdc204ac00 2024-06-06T03:39:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad22ffd700 10 client.15420 handle_command_reply: tid=0 2024-06-06T03:39:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad54993b80 1 client.15420 shutdown 2024-06-06T03:39:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad54993b80 1 -- 172.21.15.40:0/3837377296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2035bb0 msgr2=0x55bdc203a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad54993b80 1 --2- 172.21.15.40:0/3837377296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2035bb0 0x55bdc203a910 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fad3c007920 tx=0x7fad3c007c30 comp rx=0 tx=0).stop 2024-06-06T03:39:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad54993b80 1 -- 172.21.15.40:0/3837377296 shutdown_connections 2024-06-06T03:39:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad54993b80 1 --2- 172.21.15.40:0/3837377296 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bdc204ac00 0x55bdc204c070 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad54993b80 1 --2- 172.21.15.40:0/3837377296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2035bb0 0x55bdc203a910 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad54993b80 1 --2- 172.21.15.40:0/3837377296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2030f80 0x55bdc203d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad54993b80 1 --2- 172.21.15.40:0/3837377296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc20318a0 0x55bdc203a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad22ffd700 0 client.15420 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:39:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad54993b80 1 -- 172.21.15.40:0/3837377296 >> 172.21.15.40:0/3837377296 conn(0x55bdc1f57c70 msgr2=0x55bdc20275a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad54993b80 1 -- 172.21.15.40:0/3837377296 shutdown_connections 2024-06-06T03:39:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad54993b80 1 -- 172.21.15.40:0/3837377296 wait complete. 2024-06-06T03:39:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.793+0000 7fad54993b80 20 client.15420 trim_cache size 0 max 16384 2024-06-06T03:39:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.815+0000 7fad54993b80 1 Processor -- start 2024-06-06T03:39:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.815+0000 7fad54993b80 1 -- start start 2024-06-06T03:39:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.815+0000 7fad54993b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2030f80 0x55bdc203a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.815+0000 7fad54993b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc20318a0 0x55bdc2037f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.815+0000 7fad54993b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2035bb0 0x55bdc2020db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.815+0000 7fad54993b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bdc204a3b0 con 0x55bdc2030f80 2024-06-06T03:39:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.815+0000 7fad54993b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bdc204aa80 con 0x55bdc2035bb0 2024-06-06T03:39:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.815+0000 7fad54993b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bdc204b150 con 0x55bdc20318a0 2024-06-06T03:39:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.815+0000 7fad437fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2030f80 0x55bdc203a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.815+0000 7fad437fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2030f80 0x55bdc203a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36176/0 (socket says 172.21.15.40:36176) 2024-06-06T03:39:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.815+0000 7fad437fe700 1 -- 172.21.15.40:0/1647186564 learned_addr learned my addr 172.21.15.40:0/1647186564 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.815+0000 7fad43fff700 1 --2- 172.21.15.40:0/1647186564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2035bb0 0x55bdc2020db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.815+0000 7fad42ffd700 1 --2- 172.21.15.40:0/1647186564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc20318a0 0x55bdc2037f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.816+0000 7fad437fe700 1 -- 172.21.15.40:0/1647186564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc20318a0 msgr2=0x55bdc2037f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.816+0000 7fad437fe700 1 --2- 172.21.15.40:0/1647186564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc20318a0 0x55bdc2037f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.816+0000 7fad437fe700 1 -- 172.21.15.40:0/1647186564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2035bb0 msgr2=0x55bdc2020db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.816+0000 7fad437fe700 1 --2- 172.21.15.40:0/1647186564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2035bb0 0x55bdc2020db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.816+0000 7fad437fe700 1 -- 172.21.15.40:0/1647186564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdc204a8a0 con 0x55bdc2030f80 2024-06-06T03:39:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.816+0000 7fad42ffd700 1 --2- 172.21.15.40:0/1647186564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc20318a0 0x55bdc2037f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:39:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.816+0000 7fad437fe700 1 --2- 172.21.15.40:0/1647186564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2030f80 0x55bdc203a960 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fad34097610 tx=0x7fad3409e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4dd38d15c1f6bc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.816+0000 7fad297fa700 1 -- 172.21.15.40:0/1647186564 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad340a3e20 con 0x55bdc2030f80 2024-06-06T03:39:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.816+0000 7fad297fa700 1 -- 172.21.15.40:0/1647186564 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad340994b0 con 0x55bdc2030f80 2024-06-06T03:39:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.816+0000 7fad297fa700 1 -- 172.21.15.40:0/1647186564 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3409f120 con 0x55bdc2030f80 2024-06-06T03:39:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 -- 172.21.15.40:0/1647186564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2030f80 msgr2=0x55bdc203a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 --2- 172.21.15.40:0/1647186564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2030f80 0x55bdc203a960 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fad34097610 tx=0x7fad3409e040 comp rx=0 tx=0).stop 2024-06-06T03:39:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 -- 172.21.15.40:0/1647186564 shutdown_connections 2024-06-06T03:39:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 --2- 172.21.15.40:0/1647186564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc2035bb0 0x55bdc2020db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 --2- 172.21.15.40:0/1647186564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2030f80 0x55bdc203a960 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 --2- 172.21.15.40:0/1647186564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc20318a0 0x55bdc2037f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 -- 172.21.15.40:0/1647186564 >> 172.21.15.40:0/1647186564 conn(0x55bdc1f57c70 msgr2=0x55bdc2037820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 -- 172.21.15.40:0/1647186564 shutdown_connections 2024-06-06T03:39:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 -- 172.21.15.40:0/1647186564 wait complete. 2024-06-06T03:39:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 -- start start 2024-06-06T03:39:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 Processor -- start 2024-06-06T03:39:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc2030f80 0x55bdc20554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc20318a0 0x55bdc2055a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2035bb0 0x55bdc204e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bdc204bef0 con 0x55bdc2035bb0 2024-06-06T03:39:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bdc204b820 con 0x55bdc20318a0 2024-06-06T03:39:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.817+0000 7fad54993b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bdc204c5c0 con 0x55bdc2030f80 2024-06-06T03:39:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad54993b80 10 client.0 _ll_register_callbacks cb 0x55bdc1f188d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:39:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad43fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2035bb0 0x55bdc204e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad43fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2035bb0 0x55bdc204e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36190/0 (socket says 172.21.15.40:36190) 2024-06-06T03:39:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad43fff700 1 -- 172.21.15.40:0/3973910979 learned_addr learned my addr 172.21.15.40:0/3973910979 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad42ffd700 1 --2- 172.21.15.40:0/3973910979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc20318a0 0x55bdc2055a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad437fe700 1 --2- 172.21.15.40:0/3973910979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc2030f80 0x55bdc20554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad43fff700 1 -- 172.21.15.40:0/3973910979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc2030f80 msgr2=0x55bdc20554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad43fff700 1 --2- 172.21.15.40:0/3973910979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc2030f80 0x55bdc20554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad43fff700 1 -- 172.21.15.40:0/3973910979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc20318a0 msgr2=0x55bdc2055a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad43fff700 1 --2- 172.21.15.40:0/3973910979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc20318a0 0x55bdc2055a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad43fff700 1 -- 172.21.15.40:0/3973910979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdc20560b0 con 0x55bdc2035bb0 2024-06-06T03:39:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad437fe700 1 --2- 172.21.15.40:0/3973910979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc2030f80 0x55bdc20554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:39:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad43fff700 1 --2- 172.21.15.40:0/3973910979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2035bb0 0x55bdc204e490 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009d50 tx=0x7fad3c01fde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5018d381dc175d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:39:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad22ffd700 1 -- 172.21.15.40:0/3973910979 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c02c070 con 0x55bdc2035bb0 2024-06-06T03:39:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad22ffd700 1 -- 172.21.15.40:0/3973910979 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fad3c01f420 con 0x55bdc2035bb0 2024-06-06T03:39:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.818+0000 7fad54993b80 1 -- 172.21.15.40:0/3973910979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad340994b0 con 0x55bdc2035bb0 2024-06-06T03:39:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.819+0000 7fad22ffd700 1 -- 172.21.15.40:0/3973910979 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fad3c038050 con 0x55bdc2035bb0 2024-06-06T03:39:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.819+0000 7fad54993b80 10 client.15426 fetch_fsmap learned FSMap version 29 2024-06-06T03:39:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.819+0000 7fad54993b80 1 -- 172.21.15.40:0/3973910979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bdc20b69c0 con 0x55bdc2035bb0 2024-06-06T03:39:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.819+0000 7fad22ffd700 1 -- 172.21.15.40:0/3973910979 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad340994b0 con 0x55bdc2035bb0 2024-06-06T03:39:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.819+0000 7fad22ffd700 1 -- 172.21.15.40:0/3973910979 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fad3c015500 con 0x55bdc2035bb0 2024-06-06T03:39:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.819+0000 7fad54993b80 10 client.15426 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T03:39:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.819+0000 7fad54993b80 10 client.15426 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T03:39:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.819+0000 7fad54993b80 20 client.15426 populate_metadata read hostname 'smithi040' 2024-06-06T03:39:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.819+0000 7fad54993b80 1 --2- 172.21.15.40:0/3973910979 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bdc20bb5f0 0x55bdc20ba300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.819+0000 7fad54993b80 4 client.15426 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T03:39:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.819+0000 7fad54993b80 1 -- 172.21.15.40:0/3973910979 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "status"}) -- 0x55bdc1f86240 con 0x55bdc20bb5f0 2024-06-06T03:39:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.819+0000 7fad437fe700 1 --2- 172.21.15.40:0/3973910979 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bdc20bb5f0 0x55bdc20ba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad437fe700 1 --2- 172.21.15.40:0/3973910979 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bdc20bb5f0 0x55bdc20ba300 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf2efec0eff6d27 server_cookie=8e0c311e4cab2b91 in_seq=0 out_seq=0 2024-06-06T03:39:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad22ffd700 10 client.15426 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:39:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad22ffd700 1 -- 172.21.15.40:0/3973910979 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bdc1f86240 con 0x55bdc20bb5f0 2024-06-06T03:39:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad22ffd700 10 client.15426 handle_command_reply: tid=0 2024-06-06T03:39:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad54993b80 1 client.15426 shutdown 2024-06-06T03:39:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad54993b80 1 -- 172.21.15.40:0/3973910979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2035bb0 msgr2=0x55bdc204e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad54993b80 1 --2- 172.21.15.40:0/3973910979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2035bb0 0x55bdc204e490 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009d50 tx=0x7fad3c01fde0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad54993b80 1 -- 172.21.15.40:0/3973910979 shutdown_connections 2024-06-06T03:39:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad54993b80 1 --2- 172.21.15.40:0/3973910979 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bdc20bb5f0 0x55bdc20ba300 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad54993b80 1 --2- 172.21.15.40:0/3973910979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bdc2030f80 0x55bdc20554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad54993b80 1 --2- 172.21.15.40:0/3973910979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bdc2035bb0 0x55bdc204e490 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad54993b80 1 --2- 172.21.15.40:0/3973910979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdc20318a0 0x55bdc2055a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad22ffd700 0 client.15426 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:39:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad54993b80 1 -- 172.21.15.40:0/3973910979 >> 172.21.15.40:0/3973910979 conn(0x55bdc1f57c70 msgr2=0x55bdc2020db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad54993b80 1 -- 172.21.15.40:0/3973910979 shutdown_connections 2024-06-06T03:39:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad54993b80 1 -- 172.21.15.40:0/3973910979 wait complete. 2024-06-06T03:39:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.820+0000 7fad54993b80 20 client.15426 trim_cache size 0 max 16384 2024-06-06T03:39:14.860 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:39:14.861 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "77c25d68-23b4-11ef-bc9b-c7b262605968", 2024-06-06T03:39:14.861 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-06-06T03:39:14.861 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24470, 2024-06-06T03:39:14.861 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-06-06T03:39:14.861 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-06-06T03:39:14.861 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-06-06T03:39:14.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 237.96049671200001, 2024-06-06T03:39:14.861 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 29, 2024-06-06T03:39:14.861 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 91, 2024-06-06T03:39:14.861 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 90, 2024-06-06T03:39:14.861 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 255.26565737999999 2024-06-06T03:39:14.861 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:39:14.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.821+0000 7fad4a110700 1 -- 172.21.15.40:0/224787488 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fad2c0a30d0 msgr2=0x7fad2c0a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:14.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.821+0000 7fad4a110700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fad2c0a30d0 0x7fad2c0a5590 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fad38006010 tx=0x7fad3800b540 comp rx=0 tx=0).stop 2024-06-06T03:39:14.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.821+0000 7fad4a110700 1 -- 172.21.15.40:0/224787488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad44101390 msgr2=0x7fad44199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:14.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.821+0000 7fad4a110700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad44101390 0x7fad44199b70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fad3400b530 tx=0x7fad3400b8f0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.821+0000 7fad4a110700 1 -- 172.21.15.40:0/224787488 shutdown_connections 2024-06-06T03:39:14.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.821+0000 7fad4a110700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fad2c0a30d0 0x7fad2c0a5590 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.821+0000 7fad4a110700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad44107400 0x7fad441a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.821+0000 7fad4a110700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad44101390 0x7fad44199b70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.821+0000 7fad4a110700 1 --2- 172.21.15.40:0/224787488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad44101d50 0x7fad4419a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.821+0000 7fad4a110700 1 -- 172.21.15.40:0/224787488 >> 172.21.15.40:0/224787488 conn(0x7fad44078340 msgr2=0x7fad440fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.821+0000 7fad4a110700 1 -- 172.21.15.40:0/224787488 shutdown_connections 2024-06-06T03:39:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:14.821+0000 7fad4a110700 1 -- 172.21.15.40:0/224787488 wait complete. 2024-06-06T03:39:15.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:15 smithi040 ceph-mon[32220]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-06-06T03:39:15.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:15 smithi131 ceph-mon[36687]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-06-06T03:39:15.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:15 smithi104 ceph-mon[36569]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-06-06T03:39:15.559 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-06-06T03:39:17.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:17 smithi131 ceph-mon[36687]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-06-06T03:39:17.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:17 smithi040 ceph-mon[32220]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-06-06T03:39:17.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:17 smithi104 ceph-mon[36569]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-06-06T03:39:19.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:19 smithi131 ceph-mon[36687]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-06T03:39:19.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:19 smithi040 ceph-mon[32220]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-06T03:39:19.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:19 smithi104 ceph-mon[36569]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-06T03:39:21.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:21 smithi131 ceph-mon[36687]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-06T03:39:21.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:21 smithi040 ceph-mon[32220]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-06T03:39:21.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:21 smithi104 ceph-mon[36569]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-06T03:39:23.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:23 smithi040 ceph-mon[32220]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T03:39:23.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:23 smithi104 ceph-mon[36569]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T03:39:23.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:23 smithi131 ceph-mon[36687]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T03:39:25.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:25 smithi040 ceph-mon[32220]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-06T03:39:25.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:25 smithi104 ceph-mon[36569]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-06T03:39:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:25 smithi131 ceph-mon[36687]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-06T03:39:27.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:27 smithi040 ceph-mon[32220]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-06T03:39:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:27 smithi104 ceph-mon[36569]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-06T03:39:27.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:27 smithi131 ceph-mon[36687]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-06T03:39:29.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:29 smithi040 ceph-mon[32220]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:39:29.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:29 smithi104 ceph-mon[36569]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:39:29.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:29 smithi131 ceph-mon[36687]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:39:30.561 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.4 2024-06-06T03:39:30.561 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:39:30.826 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:39:31.200 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:31 smithi040 ceph-mon[32220]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-06T03:39:31.334 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:31 smithi131 ceph-mon[36687]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-06T03:39:31.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:31 smithi104 ceph-mon[36569]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-06T03:39:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.660+0000 7fb1f766d700 1 -- 172.21.15.40:0/1630383054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1f006b620 msgr2=0x7fb1f010f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.660+0000 7fb1f766d700 1 --2- 172.21.15.40:0/1630383054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1f006b620 0x7fb1f010f350 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80099c0 tx=0x7fb1e8009cd0 comp rx=0 tx=0).stop 2024-06-06T03:39:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.660+0000 7fb1f766d700 1 -- 172.21.15.40:0/1630383054 shutdown_connections 2024-06-06T03:39:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.660+0000 7fb1f766d700 1 --2- 172.21.15.40:0/1630383054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1f006b620 0x7fb1f010f350 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.660+0000 7fb1f766d700 1 --2- 172.21.15.40:0/1630383054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1f0108580 0x7fb1f0108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.660+0000 7fb1f766d700 1 --2- 172.21.15.40:0/1630383054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1f006ac60 0x7fb1f006b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.660+0000 7fb1f766d700 1 -- 172.21.15.40:0/1630383054 >> 172.21.15.40:0/1630383054 conn(0x7fb1f0078430 msgr2=0x7fb1f0078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.661+0000 7fb1f766d700 1 -- 172.21.15.40:0/1630383054 shutdown_connections 2024-06-06T03:39:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.661+0000 7fb1f766d700 1 -- 172.21.15.40:0/1630383054 wait complete. 2024-06-06T03:39:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.661+0000 7fb1f766d700 1 Processor -- start 2024-06-06T03:39:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.661+0000 7fb1f766d700 1 -- start start 2024-06-06T03:39:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f766d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1f006ac60 0x7fb1f019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f766d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1f006b620 0x7fb1f019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f766d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1f0108580 0x7fb1f01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f766d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01127f0 con 0x7fb1f0108580 2024-06-06T03:39:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f766d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0112670 con 0x7fb1f006ac60 2024-06-06T03:39:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f766d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0112970 con 0x7fb1f006b620 2024-06-06T03:39:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f5c0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1f0108580 0x7fb1f01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f5c0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1f0108580 0x7fb1f01989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46276/0 (socket says 172.21.15.40:46276) 2024-06-06T03:39:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f5c0a700 1 -- 172.21.15.40:0/660955984 learned_addr learned my addr 172.21.15.40:0/660955984 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f4c08700 1 --2- 172.21.15.40:0/660955984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1f006b620 0x7fb1f019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f5409700 1 --2- 172.21.15.40:0/660955984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1f006ac60 0x7fb1f019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f5c0a700 1 -- 172.21.15.40:0/660955984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1f006b620 msgr2=0x7fb1f019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f5c0a700 1 --2- 172.21.15.40:0/660955984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1f006b620 0x7fb1f019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f5c0a700 1 -- 172.21.15.40:0/660955984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1f006ac60 msgr2=0x7fb1f019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f5c0a700 1 --2- 172.21.15.40:0/660955984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1f006ac60 0x7fb1f019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f5c0a700 1 -- 172.21.15.40:0/660955984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f0199150 con 0x7fb1f0108580 2024-06-06T03:39:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.662+0000 7fb1f5c0a700 1 --2- 172.21.15.40:0/660955984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1f0108580 0x7fb1f01989f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80059d0 tx=0x7fb1e8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.663+0000 7fb1e67fc700 1 -- 172.21.15.40:0/660955984 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8027070 con 0x7fb1f0108580 2024-06-06T03:39:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.663+0000 7fb1f766d700 1 -- 172.21.15.40:0/660955984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f01993e0 con 0x7fb1f0108580 2024-06-06T03:39:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.663+0000 7fb1e67fc700 1 -- 172.21.15.40:0/660955984 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1e8003740 con 0x7fb1f0108580 2024-06-06T03:39:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.663+0000 7fb1e67fc700 1 -- 172.21.15.40:0/660955984 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8022bf0 con 0x7fb1f0108580 2024-06-06T03:39:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.663+0000 7fb1f766d700 1 -- 172.21.15.40:0/660955984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f01a56b0 con 0x7fb1f0108580 2024-06-06T03:39:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.664+0000 7fb1f766d700 1 -- 172.21.15.40:0/660955984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f0069180 con 0x7fb1f0108580 2024-06-06T03:39:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.665+0000 7fb1e67fc700 1 -- 172.21.15.40:0/660955984 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7fb1e802a070 con 0x7fb1f0108580 2024-06-06T03:39:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.666+0000 7fb1e67fc700 1 --2- 172.21.15.40:0/660955984 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fb1dc0a2eb0 0x7fb1dc0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.666+0000 7fb1f5409700 1 --2- 172.21.15.40:0/660955984 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fb1dc0a2eb0 0x7fb1dc0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.666+0000 7fb1e67fc700 1 -- 172.21.15.40:0/660955984 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fb1e80d0670 con 0x7fb1f0108580 2024-06-06T03:39:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.666+0000 7fb1f5409700 1 --2- 172.21.15.40:0/660955984 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fb1dc0a2eb0 0x7fb1dc0a5370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009550 tx=0x7fb1ec0091b0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.670+0000 7fb1e67fc700 1 -- 172.21.15.40:0/660955984 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb1e805e850 con 0x7fb1f0108580 2024-06-06T03:39:32.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.925+0000 7fb1f766d700 1 -- 172.21.15.40:0/660955984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1f002dca0 con 0x7fb1f0108580 2024-06-06T03:39:32.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.925+0000 7fb1e67fc700 1 -- 172.21.15.40:0/660955984 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11933 (secure 0 0 0) 0x7fb1e8013070 con 0x7fb1f0108580 2024-06-06T03:39:32.926 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:39:32.926 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:38:35.323771+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24443,24425,24398,24403]},"id":1}]} 2024-06-06T03:39:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.928+0000 7fb1f766d700 1 -- 172.21.15.40:0/660955984 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fb1dc0a2eb0 msgr2=0x7fb1dc0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.928+0000 7fb1f766d700 1 --2- 172.21.15.40:0/660955984 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fb1dc0a2eb0 0x7fb1dc0a5370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009550 tx=0x7fb1ec0091b0 comp rx=0 tx=0).stop 2024-06-06T03:39:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.928+0000 7fb1f766d700 1 -- 172.21.15.40:0/660955984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1f0108580 msgr2=0x7fb1f01989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.928+0000 7fb1f766d700 1 --2- 172.21.15.40:0/660955984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1f0108580 0x7fb1f01989f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80059d0 tx=0x7fb1e8009f80 comp rx=0 tx=0).stop 2024-06-06T03:39:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.928+0000 7fb1f766d700 1 -- 172.21.15.40:0/660955984 shutdown_connections 2024-06-06T03:39:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.928+0000 7fb1f766d700 1 --2- 172.21.15.40:0/660955984 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fb1dc0a2eb0 0x7fb1dc0a5370 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.928+0000 7fb1f766d700 1 --2- 172.21.15.40:0/660955984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1f0108580 0x7fb1f01989f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.928+0000 7fb1f766d700 1 --2- 172.21.15.40:0/660955984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1f006ac60 0x7fb1f019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.928+0000 7fb1f766d700 1 --2- 172.21.15.40:0/660955984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1f006b620 0x7fb1f019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.928+0000 7fb1f766d700 1 -- 172.21.15.40:0/660955984 >> 172.21.15.40:0/660955984 conn(0x7fb1f0078430 msgr2=0x7fb1f00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.928+0000 7fb1f766d700 1 -- 172.21.15.40:0/660955984 shutdown_connections 2024-06-06T03:39:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:32.928+0000 7fb1f766d700 1 -- 172.21.15.40:0/660955984 wait complete. 2024-06-06T03:39:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 29 2024-06-06T03:39:33.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:33 smithi040 ceph-mon[32220]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:39:33.261 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:33 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/660955984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:39:33.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:33 smithi104 ceph-mon[36569]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:39:33.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:33 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/660955984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:39:33.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:33 smithi131 ceph-mon[36687]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:39:33.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:33 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/660955984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:39:33.699 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 154.4 --await-for 45.0' 2024-06-06T03:39:33.699 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 154.4 --await-for 45.0 2024-06-06T03:39:33.973 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:39:35.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:35 smithi040 ceph-mon[32220]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-06T03:39:35.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:35 smithi104 ceph-mon[36569]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-06T03:39:35.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:35 smithi131 ceph-mon[36687]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-06T03:39:35.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.773+0000 7f5b89698700 1 -- 172.21.15.40:0/3303139630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84107470 msgr2=0x7f5b84107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:35.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.773+0000 7f5b89698700 1 --2- 172.21.15.40:0/3303139630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84107470 0x7f5b84107850 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009990 tx=0x7f5b74009ca0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.773+0000 7f5b89698700 1 -- 172.21.15.40:0/3303139630 shutdown_connections 2024-06-06T03:39:35.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.773+0000 7f5b89698700 1 --2- 172.21.15.40:0/3303139630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b84101d80 0x7f5b8410e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.773+0000 7f5b89698700 1 --2- 172.21.15.40:0/3303139630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84107470 0x7f5b84107850 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.773+0000 7f5b89698700 1 --2- 172.21.15.40:0/3303139630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b841013c0 0x7f5b84101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.773+0000 7f5b89698700 1 -- 172.21.15.40:0/3303139630 >> 172.21.15.40:0/3303139630 conn(0x7f5b840783d0 msgr2=0x7f5b840787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:35.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.773+0000 7f5b89698700 1 -- 172.21.15.40:0/3303139630 shutdown_connections 2024-06-06T03:39:35.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.773+0000 7f5b89698700 1 -- 172.21.15.40:0/3303139630 wait complete. 2024-06-06T03:39:35.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.773+0000 7f5b89698700 1 Processor -- start 2024-06-06T03:39:35.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.774+0000 7f5b89698700 1 -- start start 2024-06-06T03:39:35.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.774+0000 7f5b89698700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b841013c0 0x7f5b84199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.774+0000 7f5b89698700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84101d80 0x7f5b84199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.774+0000 7f5b89698700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b84107470 0x7f5b841a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.774+0000 7f5b89698700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841116b0 con 0x7f5b84101d80 2024-06-06T03:39:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.774+0000 7f5b89698700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84111530 con 0x7f5b84107470 2024-06-06T03:39:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.774+0000 7f5b89698700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84111830 con 0x7f5b841013c0 2024-06-06T03:39:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.774+0000 7f5b7a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84101d80 0x7f5b84199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.774+0000 7f5b7a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84101d80 0x7f5b84199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46290/0 (socket says 172.21.15.40:46290) 2024-06-06T03:39:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.774+0000 7f5b7a7fc700 1 -- 172.21.15.40:0/722607353 learned_addr learned my addr 172.21.15.40:0/722607353 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.774+0000 7f5b82ffd700 1 --2- 172.21.15.40:0/722607353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b841013c0 0x7f5b84199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.774+0000 7f5b837fe700 1 --2- 172.21.15.40:0/722607353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b84107470 0x7f5b841a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.775+0000 7f5b7a7fc700 1 -- 172.21.15.40:0/722607353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b841013c0 msgr2=0x7f5b84199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.775+0000 7f5b7a7fc700 1 --2- 172.21.15.40:0/722607353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b841013c0 0x7f5b84199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.775+0000 7f5b7a7fc700 1 -- 172.21.15.40:0/722607353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b84107470 msgr2=0x7f5b841a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:35.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.775+0000 7f5b7a7fc700 1 --2- 172.21.15.40:0/722607353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b84107470 0x7f5b841a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.775+0000 7f5b7a7fc700 1 -- 172.21.15.40:0/722607353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b841a4f10 con 0x7f5b84101d80 2024-06-06T03:39:35.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.775+0000 7f5b7a7fc700 1 --2- 172.21.15.40:0/722607353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84101d80 0x7f5b84199e90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00d950 tx=0x7f5b6c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:35.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.775+0000 7f5b80ff9700 1 -- 172.21.15.40:0/722607353 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c016070 con 0x7f5b84101d80 2024-06-06T03:39:35.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.775+0000 7f5b89698700 1 -- 172.21.15.40:0/722607353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b841a5140 con 0x7f5b84101d80 2024-06-06T03:39:35.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.775+0000 7f5b80ff9700 1 -- 172.21.15.40:0/722607353 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b6c00fd70 con 0x7f5b84101d80 2024-06-06T03:39:35.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.775+0000 7f5b80ff9700 1 -- 172.21.15.40:0/722607353 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c00e610 con 0x7f5b84101d80 2024-06-06T03:39:35.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.775+0000 7f5b89698700 1 -- 172.21.15.40:0/722607353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b841a56a0 con 0x7f5b84101d80 2024-06-06T03:39:35.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.778+0000 7f5b80ff9700 1 -- 172.21.15.40:0/722607353 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f5b6c019070 con 0x7f5b84101d80 2024-06-06T03:39:35.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.779+0000 7f5b80ff9700 1 --2- 172.21.15.40:0/722607353 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f5b700a2fd0 0x7f5b700a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.779+0000 7f5b93f1bb80 1 Processor -- start 2024-06-06T03:39:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.779+0000 7f5b82ffd700 1 --2- 172.21.15.40:0/722607353 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f5b700a2fd0 0x7f5b700a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.779+0000 7f5b80ff9700 1 -- 172.21.15.40:0/722607353 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f5b6c0cf090 con 0x7f5b84101d80 2024-06-06T03:39:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.779+0000 7f5b93f1bb80 1 -- start start 2024-06-06T03:39:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b82ffd700 1 --2- 172.21.15.40:0/722607353 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f5b700a2fd0 0x7f5b700a5490 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400b5c0 tx=0x7f5b74005960 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b93f1bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92590 0x55bf55d92970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b93f1bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92eb0 0x55bf55d8c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b93f1bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d8cce0 0x55bf55d8d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b93f1bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bf55c7f0f0 con 0x55bf55d8cce0 2024-06-06T03:39:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b93f1bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bf55cdd9e0 con 0x55bf55d92eb0 2024-06-06T03:39:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b93f1bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bf55c455f0 con 0x55bf55d92590 2024-06-06T03:39:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d8cce0 0x55bf55d8d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d8cce0 0x55bf55d8d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46304/0 (socket says 172.21.15.40:46304) 2024-06-06T03:39:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b837fe700 1 -- 172.21.15.40:0/928417634 learned_addr learned my addr 172.21.15.40:0/928417634 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b82ffd700 1 --2- 172.21.15.40:0/928417634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92590 0x55bf55d92970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b7a7fc700 1 --2- 172.21.15.40:0/928417634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92eb0 0x55bf55d8c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b837fe700 1 -- 172.21.15.40:0/928417634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92590 msgr2=0x55bf55d92970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b837fe700 1 --2- 172.21.15.40:0/928417634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92590 0x55bf55d92970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b837fe700 1 -- 172.21.15.40:0/928417634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92eb0 msgr2=0x55bf55d8c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:35.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b837fe700 1 --2- 172.21.15.40:0/928417634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92eb0 0x55bf55d8c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b837fe700 1 -- 172.21.15.40:0/928417634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf55c2ca60 con 0x55bf55d8cce0 2024-06-06T03:39:35.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.780+0000 7f5b837fe700 1 --2- 172.21.15.40:0/928417634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d8cce0 0x55bf55d8d100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c003fa0 tx=0x7f5b7c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fa9cc56cad2b7f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:35.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.781+0000 7f5b797fa700 1 -- 172.21.15.40:0/928417634 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c004bb0 con 0x55bf55d8cce0 2024-06-06T03:39:35.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.781+0000 7f5b797fa700 1 -- 172.21.15.40:0/928417634 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b7c004d50 con 0x55bf55d8cce0 2024-06-06T03:39:35.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.781+0000 7f5b797fa700 1 -- 172.21.15.40:0/928417634 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c007dc0 con 0x55bf55d8cce0 2024-06-06T03:39:35.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.781+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/928417634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d8cce0 msgr2=0x55bf55d8d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:35.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.781+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/928417634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d8cce0 0x55bf55d8d100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c003fa0 tx=0x7f5b7c009ba0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/928417634 shutdown_connections 2024-06-06T03:39:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/928417634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92590 0x55bf55d92970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/928417634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d8cce0 0x55bf55d8d100 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/928417634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92eb0 0x55bf55d8c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/928417634 >> 172.21.15.40:0/928417634 conn(0x55bf55cb9210 msgr2=0x55bf55d87a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/928417634 shutdown_connections 2024-06-06T03:39:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/928417634 wait complete. 2024-06-06T03:39:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 -- start start 2024-06-06T03:39:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 Processor -- start 2024-06-06T03:39:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d8cce0 0x55bf55d98b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92590 0x55bf55d95e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92eb0 0x55bf55d96380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bf55c7e150 con 0x55bf55d8cce0 2024-06-06T03:39:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bf55cd99a0 con 0x55bf55d92590 2024-06-06T03:39:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.782+0000 7f5b93f1bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bf55d99890 con 0x55bf55d92eb0 2024-06-06T03:39:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b93f1bb80 10 client.0 _ll_register_callbacks cb 0x55bf55c73740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:39:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b82ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d8cce0 0x55bf55d98b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b82ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d8cce0 0x55bf55d98b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46318/0 (socket says 172.21.15.40:46318) 2024-06-06T03:39:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b82ffd700 1 -- 172.21.15.40:0/225374859 learned_addr learned my addr 172.21.15.40:0/225374859 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b7a7fc700 1 --2- 172.21.15.40:0/225374859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92590 0x55bf55d95e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b837fe700 1 --2- 172.21.15.40:0/225374859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92eb0 0x55bf55d96380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b82ffd700 1 -- 172.21.15.40:0/225374859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92eb0 msgr2=0x55bf55d96380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b82ffd700 1 --2- 172.21.15.40:0/225374859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92eb0 0x55bf55d96380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b82ffd700 1 -- 172.21.15.40:0/225374859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92590 msgr2=0x55bf55d95e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b82ffd700 1 --2- 172.21.15.40:0/225374859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92590 0x55bf55d95e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b82ffd700 1 -- 172.21.15.40:0/225374859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf55ca38e0 con 0x55bf55d8cce0 2024-06-06T03:39:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b82ffd700 1 --2- 172.21.15.40:0/225374859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d8cce0 0x55bf55d98b40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5b74003900 tx=0x7f5b7401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6528d6eca3cad5df server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b6a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:39:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b6a7fc700 1 -- 172.21.15.40:0/225374859 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74024410 con 0x55bf55d8cce0 2024-06-06T03:39:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b6a7fc700 1 -- 172.21.15.40:0/225374859 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f5b740245b0 con 0x55bf55d8cce0 2024-06-06T03:39:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.783+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/225374859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b7c004d50 con 0x55bf55d8cce0 2024-06-06T03:39:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.784+0000 7f5b6a7fc700 1 -- 172.21.15.40:0/225374859 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b740247f0 con 0x55bf55d8cce0 2024-06-06T03:39:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.784+0000 7f5b93f1bb80 10 client.15444 fetch_fsmap learned FSMap version 29 2024-06-06T03:39:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.784+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/225374859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bf55da3fa0 con 0x55bf55d8cce0 2024-06-06T03:39:35.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.784+0000 7f5b6a7fc700 1 -- 172.21.15.40:0/225374859 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b7c004d50 con 0x55bf55d8cce0 2024-06-06T03:39:35.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.784+0000 7f5b6a7fc700 1 -- 172.21.15.40:0/225374859 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5b74039a80 con 0x55bf55d8cce0 2024-06-06T03:39:35.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.784+0000 7f5b93f1bb80 10 client.15444 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T03:39:35.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.784+0000 7f5b93f1bb80 10 client.15444 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:39:35.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.784+0000 7f5b93f1bb80 20 client.15444 populate_metadata read hostname 'smithi040' 2024-06-06T03:39:35.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.784+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/225374859 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bf55da6530 0x55bf55da8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.784+0000 7f5b93f1bb80 4 client.15444 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:39:35.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.784+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/225374859 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf55ceaee0 con 0x55bf55da6530 2024-06-06T03:39:35.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.785+0000 7f5b7a7fc700 1 --2- 172.21.15.40:0/225374859 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bf55da6530 0x55bf55da8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.785+0000 7f5b6a7fc700 10 client.15444 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:39:35.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.785+0000 7f5b7a7fc700 1 --2- 172.21.15.40:0/225374859 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bf55da6530 0x55bf55da8990 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9034064e4e465da7 server_cookie=23fb77ea950e517e in_seq=0 out_seq=0 2024-06-06T03:39:35.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b6a7fc700 1 -- 172.21.15.40:0/225374859 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bf55ceaee0 con 0x55bf55da6530 2024-06-06T03:39:35.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b6a7fc700 10 client.15444 handle_command_reply: tid=0 2024-06-06T03:39:35.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b93f1bb80 1 client.15444 shutdown 2024-06-06T03:39:35.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/225374859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d8cce0 msgr2=0x55bf55d98b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:35.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/225374859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d8cce0 0x55bf55d98b40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5b74003900 tx=0x7f5b7401a050 comp rx=0 tx=0).stop 2024-06-06T03:39:35.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/225374859 shutdown_connections 2024-06-06T03:39:35.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/225374859 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bf55da6530 0x55bf55da8990 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b6a7fc700 0 client.15444 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:39:35.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/225374859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92eb0 0x55bf55d96380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/225374859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d8cce0 0x55bf55d98b40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/225374859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92590 0x55bf55d95e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/225374859 >> 172.21.15.40:0/225374859 conn(0x55bf55cb9210 msgr2=0x55bf55d86c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:35.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/225374859 shutdown_connections 2024-06-06T03:39:35.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/225374859 wait complete. 2024-06-06T03:39:35.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.786+0000 7f5b93f1bb80 20 client.15444 trim_cache size 0 max 16384 2024-06-06T03:39:35.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.810+0000 7f5b93f1bb80 1 Processor -- start 2024-06-06T03:39:35.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.810+0000 7f5b93f1bb80 1 -- start start 2024-06-06T03:39:35.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.810+0000 7f5b93f1bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d8cce0 0x55bf55d8c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.810+0000 7f5b93f1bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92590 0x55bf55d95e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.810+0000 7f5b93f1bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d92eb0 0x55bf55d96340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b93f1bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841aed70 con 0x55bf55d92eb0 2024-06-06T03:39:35.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b93f1bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841b08b0 con 0x55bf55d8cce0 2024-06-06T03:39:35.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b93f1bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841af440 con 0x55bf55d92590 2024-06-06T03:39:35.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d92eb0 0x55bf55d96340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d92eb0 0x55bf55d96340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46330/0 (socket says 172.21.15.40:46330) 2024-06-06T03:39:35.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b837fe700 1 -- 172.21.15.40:0/3040242534 learned_addr learned my addr 172.21.15.40:0/3040242534 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:35.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b82ffd700 1 --2- 172.21.15.40:0/3040242534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d8cce0 0x55bf55d8c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b7a7fc700 1 --2- 172.21.15.40:0/3040242534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92590 0x55bf55d95e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b837fe700 1 -- 172.21.15.40:0/3040242534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92590 msgr2=0x55bf55d95e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:35.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b837fe700 1 --2- 172.21.15.40:0/3040242534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92590 0x55bf55d95e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b837fe700 1 -- 172.21.15.40:0/3040242534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d8cce0 msgr2=0x55bf55d8c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:35.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b837fe700 1 --2- 172.21.15.40:0/3040242534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d8cce0 0x55bf55d8c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b837fe700 1 -- 172.21.15.40:0/3040242534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf55da9010 con 0x55bf55d92eb0 2024-06-06T03:39:35.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b82ffd700 1 --2- 172.21.15.40:0/3040242534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d8cce0 0x55bf55d8c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:39:35.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b837fe700 1 --2- 172.21.15.40:0/3040242534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d92eb0 0x55bf55d96340 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c007940 tx=0x7f5b7c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b44260fc12d8e3f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:35.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.811+0000 7f5b78ff9700 1 -- 172.21.15.40:0/3040242534 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c007a10 con 0x55bf55d92eb0 2024-06-06T03:39:35.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.812+0000 7f5b78ff9700 1 -- 172.21.15.40:0/3040242534 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b7c007bb0 con 0x55bf55d92eb0 2024-06-06T03:39:35.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.812+0000 7f5b78ff9700 1 -- 172.21.15.40:0/3040242534 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0052b0 con 0x55bf55d92eb0 2024-06-06T03:39:35.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.812+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/3040242534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d92eb0 msgr2=0x55bf55d96340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:35.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.812+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/3040242534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d92eb0 0x55bf55d96340 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c007940 tx=0x7f5b7c007800 comp rx=0 tx=0).stop 2024-06-06T03:39:35.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.812+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/3040242534 shutdown_connections 2024-06-06T03:39:35.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.812+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/3040242534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d92eb0 0x55bf55d96340 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.812+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/3040242534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d8cce0 0x55bf55d8c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.812+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/3040242534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d92590 0x55bf55d95e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.812+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/3040242534 >> 172.21.15.40:0/3040242534 conn(0x55bf55cb9210 msgr2=0x55bf55d956f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:35.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.812+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/3040242534 shutdown_connections 2024-06-06T03:39:35.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.812+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/3040242534 wait complete. 2024-06-06T03:39:35.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.813+0000 7f5b93f1bb80 1 -- start start 2024-06-06T03:39:35.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.813+0000 7f5b93f1bb80 1 Processor -- start 2024-06-06T03:39:35.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.813+0000 7f5b93f1bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d8cce0 0x55bf55d9f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.813+0000 7f5b93f1bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d92590 0x55bf55d9bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.813+0000 7f5b93f1bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92eb0 0x55bf55d9c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.813+0000 7f5b93f1bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841b01e0 con 0x55bf55d92590 2024-06-06T03:39:35.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.813+0000 7f5b93f1bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841afb10 con 0x55bf55d92eb0 2024-06-06T03:39:35.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.813+0000 7f5b93f1bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841acd90 con 0x55bf55d8cce0 2024-06-06T03:39:35.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.813+0000 7f5b93f1bb80 10 client.0 _ll_register_callbacks cb 0x55bf55d90a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:39:35.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.813+0000 7f5b7a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d92590 0x55bf55d9bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b7a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d92590 0x55bf55d9bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46340/0 (socket says 172.21.15.40:46340) 2024-06-06T03:39:35.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b7a7fc700 1 -- 172.21.15.40:0/3803373942 learned_addr learned my addr 172.21.15.40:0/3803373942 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:39:35.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b82ffd700 1 --2- 172.21.15.40:0/3803373942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d8cce0 0x55bf55d9f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b837fe700 1 --2- 172.21.15.40:0/3803373942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92eb0 0x55bf55d9c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b7a7fc700 1 -- 172.21.15.40:0/3803373942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d8cce0 msgr2=0x55bf55d9f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:35.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b7a7fc700 1 --2- 172.21.15.40:0/3803373942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d8cce0 0x55bf55d9f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b7a7fc700 1 -- 172.21.15.40:0/3803373942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92eb0 msgr2=0x55bf55d9c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:39:35.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b7a7fc700 1 --2- 172.21.15.40:0/3803373942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92eb0 0x55bf55d9c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b7a7fc700 1 -- 172.21.15.40:0/3803373942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf55d9c700 con 0x55bf55d92590 2024-06-06T03:39:35.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b7a7fc700 1 --2- 172.21.15.40:0/3803373942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d92590 0x55bf55d9bb80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0cce70 tx=0x7f5b6c097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fdcff046c8f5ea server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:39:35.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b6a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:39:35.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b6a7fc700 1 -- 172.21.15.40:0/3803373942 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c020040 con 0x55bf55d92590 2024-06-06T03:39:35.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b6a7fc700 1 -- 172.21.15.40:0/3803373942 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f5b6c010710 con 0x55bf55d92590 2024-06-06T03:39:35.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/3803373942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b7c007bb0 con 0x55bf55d92590 2024-06-06T03:39:35.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b6a7fc700 1 -- 172.21.15.40:0/3803373942 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b6c0a4060 con 0x55bf55d92590 2024-06-06T03:39:35.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b93f1bb80 10 client.15450 fetch_fsmap learned FSMap version 29 2024-06-06T03:39:35.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.814+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/3803373942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bf55e120c0 con 0x55bf55d92590 2024-06-06T03:39:35.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.815+0000 7f5b6a7fc700 1 -- 172.21.15.40:0/3803373942 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b7c007bb0 con 0x55bf55d92590 2024-06-06T03:39:35.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.815+0000 7f5b6a7fc700 1 -- 172.21.15.40:0/3803373942 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5b6c0cc9d0 con 0x55bf55d92590 2024-06-06T03:39:35.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.815+0000 7f5b93f1bb80 10 client.15450 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T03:39:35.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.815+0000 7f5b93f1bb80 10 client.15450 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:39:35.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.815+0000 7f5b93f1bb80 20 client.15450 populate_metadata read hostname 'smithi040' 2024-06-06T03:39:35.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.815+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/3803373942 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bf55e12870 0x55bf55e14c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:39:35.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.815+0000 7f5b93f1bb80 4 client.15450 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.4, "await_for": 45.0}] 2024-06-06T03:39:35.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.815+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/3803373942 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.4, "await_for": 45.0}) -- 0x55bf55ce5460 con 0x55bf55e12870 2024-06-06T03:39:35.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.815+0000 7f5b82ffd700 1 --2- 172.21.15.40:0/3803373942 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bf55e12870 0x55bf55e14c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:39:35.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.816+0000 7f5b82ffd700 1 --2- 172.21.15.40:0/3803373942 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bf55e12870 0x55bf55e14c90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92afca895b774a41 server_cookie=59439ca9e6b7a94d in_seq=0 out_seq=0 2024-06-06T03:39:35.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.816+0000 7f5b6a7fc700 10 client.15450 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:39:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b6a7fc700 1 -- 172.21.15.40:0/3803373942 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55bf55ce5460 con 0x55bf55e12870 2024-06-06T03:39:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b6a7fc700 10 client.15450 handle_command_reply: tid=0 2024-06-06T03:39:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b93f1bb80 1 client.15450 shutdown 2024-06-06T03:39:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/3803373942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d92590 msgr2=0x55bf55d9bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/3803373942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d92590 0x55bf55d9bb80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0cce70 tx=0x7f5b6c097040 comp rx=0 tx=0).stop 2024-06-06T03:39:35.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/3803373942 shutdown_connections 2024-06-06T03:39:35.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/3803373942 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55bf55e12870 0x55bf55e14c90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/3803373942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf55d8cce0 0x55bf55d9f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/3803373942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf55d92eb0 0x55bf55d9c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b93f1bb80 1 --2- 172.21.15.40:0/3803373942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bf55d92590 0x55bf55d9bb80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/3803373942 >> 172.21.15.40:0/3803373942 conn(0x55bf55cb9210 msgr2=0x55bf55d95f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:35.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b6a7fc700 0 client.15450 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:39:35.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/3803373942 shutdown_connections 2024-06-06T03:39:35.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b93f1bb80 1 -- 172.21.15.40:0/3803373942 wait complete. 2024-06-06T03:39:35.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.825+0000 7f5b93f1bb80 20 client.15450 trim_cache size 0 max 16384 2024-06-06T03:39:35.827 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:39:35.827 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 29, 2024-06-06T03:39:35.827 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:39:35.827 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 2, 2024-06-06T03:39:35.827 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": { 2024-06-06T03:39:35.827 INFO:teuthology.orchestra.run.smithi040.stdout: "899a3131": { 2024-06-06T03:39:35.827 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 2, 2024-06-06T03:39:35.827 INFO:teuthology.orchestra.run.smithi040.stdout: "age_ref": 0.0, 2024-06-06T03:39:35.827 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:39:35.827 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "QUIESCED", 2024-06-06T03:39:35.827 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:39:35.828 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:39:35.828 INFO:teuthology.orchestra.run.smithi040.stdout: "timeout": 90.0, 2024-06-06T03:39:35.828 INFO:teuthology.orchestra.run.smithi040.stdout: "expiration": 154.4, 2024-06-06T03:39:35.828 INFO:teuthology.orchestra.run.smithi040.stdout: "members": { 2024-06-06T03:39:35.828 INFO:teuthology.orchestra.run.smithi040.stdout: "file:/": { 2024-06-06T03:39:35.828 INFO:teuthology.orchestra.run.smithi040.stdout: "excluded": false, 2024-06-06T03:39:35.828 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:39:35.828 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "QUIESCED", 2024-06-06T03:39:35.828 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:39:35.828 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:39:35.828 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:39:35.828 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:39:35.828 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:39:35.828 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:39:35.829 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:39:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.826+0000 7f5b89698700 1 -- 172.21.15.40:0/722607353 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f5b700a2fd0 msgr2=0x7f5b700a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.826+0000 7f5b89698700 1 --2- 172.21.15.40:0/722607353 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f5b700a2fd0 0x7f5b700a5490 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400b5c0 tx=0x7f5b74005960 comp rx=0 tx=0).stop 2024-06-06T03:39:35.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.826+0000 7f5b89698700 1 -- 172.21.15.40:0/722607353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84101d80 msgr2=0x7f5b84199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:39:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.826+0000 7f5b89698700 1 --2- 172.21.15.40:0/722607353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84101d80 0x7f5b84199e90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00d950 tx=0x7f5b6c00dc60 comp rx=0 tx=0).stop 2024-06-06T03:39:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.826+0000 7f5b89698700 1 -- 172.21.15.40:0/722607353 shutdown_connections 2024-06-06T03:39:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.826+0000 7f5b89698700 1 --2- 172.21.15.40:0/722607353 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f5b700a2fd0 0x7f5b700a5490 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.826+0000 7f5b89698700 1 --2- 172.21.15.40:0/722607353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b841013c0 0x7f5b84199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.826+0000 7f5b89698700 1 --2- 172.21.15.40:0/722607353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b84107470 0x7f5b841a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.826+0000 7f5b89698700 1 --2- 172.21.15.40:0/722607353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84101d80 0x7f5b84199e90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:39:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.826+0000 7f5b89698700 1 -- 172.21.15.40:0/722607353 >> 172.21.15.40:0/722607353 conn(0x7f5b840783d0 msgr2=0x7f5b84100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:39:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.826+0000 7f5b89698700 1 -- 172.21.15.40:0/722607353 shutdown_connections 2024-06-06T03:39:35.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:39:35.826+0000 7f5b89698700 1 -- 172.21.15.40:0/722607353 wait complete. 2024-06-06T03:39:36.687 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '899a3131', quiesce took 6.1 seconds. Will release after: 28.299999999999997 2024-06-06T03:39:37.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:37 smithi040 ceph-mon[32220]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:39:37.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:37 smithi104 ceph-mon[36569]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:39:37.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:37 smithi131 ceph-mon[36687]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-06T03:39:39.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:39 smithi040 ceph-mon[32220]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:39:39.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:39 smithi104 ceph-mon[36569]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:39:39.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:39 smithi131 ceph-mon[36687]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:39:41.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:41 smithi040 ceph-mon[32220]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-06T03:39:41.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:41 smithi104 ceph-mon[36569]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-06T03:39:41.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:41 smithi131 ceph-mon[36687]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-06T03:39:43.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:43 smithi040 ceph-mon[32220]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-06T03:39:43.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:43 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:39:43.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:43 smithi104 ceph-mon[36569]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-06T03:39:43.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:43 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:39:43.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:43 smithi131 ceph-mon[36687]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-06T03:39:43.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:43 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:39:44.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:44 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:39:44.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:44 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:39:44.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:44 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:39:44.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:44 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:39:44.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:44 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:39:44.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:44 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:39:44.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:44 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:39:44.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:44 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:39:44.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:44 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:39:45.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:45 smithi040 ceph-mon[32220]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-06T03:39:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:45 smithi104 ceph-mon[36569]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-06T03:39:45.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:45 smithi131 ceph-mon[36687]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-06T03:39:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:47 smithi040 ceph-mon[32220]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-06T03:39:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:47 smithi104 ceph-mon[36569]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-06T03:39:47.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:47 smithi131 ceph-mon[36687]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-06T03:39:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:49 smithi040 ceph-mon[32220]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-06T03:39:49.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:49 smithi104 ceph-mon[36569]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-06T03:39:49.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:49 smithi131 ceph-mon[36687]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-06T03:39:51.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:51 smithi040 ceph-mon[32220]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1023 B/s wr, 13 op/s 2024-06-06T03:39:51.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:51 smithi104 ceph-mon[36569]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1023 B/s wr, 13 op/s 2024-06-06T03:39:51.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:51 smithi131 ceph-mon[36687]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1023 B/s wr, 13 op/s 2024-06-06T03:39:53.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:53 smithi104 ceph-mon[36569]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T03:39:53.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:53 smithi131 ceph-mon[36687]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T03:39:53.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:53 smithi040 ceph-mon[32220]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T03:39:55.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:55 smithi104 ceph-mon[36569]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-06T03:39:55.382 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:55 smithi040 ceph-mon[32220]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-06T03:39:55.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:55 smithi131 ceph-mon[36687]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-06T03:39:57.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:57 smithi104 ceph-mon[36569]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T03:39:57.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:57 smithi131 ceph-mon[36687]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T03:39:57.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:57 smithi040 ceph-mon[32220]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T03:39:59.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:39:59 smithi104 ceph-mon[36569]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T03:39:59.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:39:59 smithi131 ceph-mon[36687]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T03:39:59.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:39:59 smithi040 ceph-mon[32220]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T03:40:00.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:00 smithi131 ceph-mon[36687]: overall HEALTH_OK 2024-06-06T03:40:00.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:00 smithi040 ceph-mon[32220]: overall HEALTH_OK 2024-06-06T03:40:00.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:00 smithi104 ceph-mon[36569]: overall HEALTH_OK 2024-06-06T03:40:01.433 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:01 smithi131 ceph-mon[36687]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-06T03:40:01.491 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:01 smithi104 ceph-mon[36569]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-06T03:40:01.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:01 smithi040 ceph-mon[32220]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-06T03:40:03.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:03 smithi131 ceph-mon[36687]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:40:03.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:03 smithi040 ceph-mon[32220]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:40:03.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:03 smithi104 ceph-mon[36569]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T03:40:04.987 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:40:05.296 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:40:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:05 smithi040 ceph-mon[32220]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-06-06T03:40:05.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:05 smithi131 ceph-mon[36687]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-06-06T03:40:05.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:05 smithi104 ceph-mon[36569]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-06-06T03:40:07.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.293+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2410307693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e44106720 msgr2=0x7f0e4410e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:40:07.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.293+0000 7f0e4a3a8700 1 --2- 172.21.15.40:0/2410307693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e44106720 0x7f0e4410e2c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00b410 tx=0x7f0e3c00b720 comp rx=0 tx=0).stop 2024-06-06T03:40:07.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.293+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2410307693 shutdown_connections 2024-06-06T03:40:07.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.293+0000 7f0e4a3a8700 1 --2- 172.21.15.40:0/2410307693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e44106720 0x7f0e4410e2c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:07.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.293+0000 7f0e4a3a8700 1 --2- 172.21.15.40:0/2410307693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e441019d0 0x7f0e44101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:07.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.293+0000 7f0e4a3a8700 1 --2- 172.21.15.40:0/2410307693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e44102380 0x7f0e44105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:07.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.293+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2410307693 >> 172.21.15.40:0/2410307693 conn(0x7f0e44078340 msgr2=0x7f0e44078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:40:07.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.293+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2410307693 shutdown_connections 2024-06-06T03:40:07.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.293+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2410307693 wait complete. 2024-06-06T03:40:07.296 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:07 smithi040 ceph-mon[32220]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-06-06T03:40:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.295+0000 7f0e4a3a8700 1 Processor -- start 2024-06-06T03:40:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.295+0000 7f0e4a3a8700 1 -- start start 2024-06-06T03:40:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.296+0000 7f0e4a3a8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e441019d0 0x7f0e4419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.296+0000 7f0e4a3a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e44102380 0x7f0e4419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.296+0000 7f0e4a3a8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e44106720 0x7f0e44197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.296+0000 7f0e4a3a8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0e44111780 con 0x7f0e44102380 2024-06-06T03:40:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.297+0000 7f0e48945700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e44106720 0x7f0e44197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.297+0000 7f0e48945700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e44106720 0x7f0e44197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:42626/0 (socket says 172.21.15.40:42626) 2024-06-06T03:40:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.297+0000 7f0e48945700 1 -- 172.21.15.40:0/2541628990 learned_addr learned my addr 172.21.15.40:0/2541628990 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:40:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.297+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2541628990 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0e44111600 con 0x7f0e44106720 2024-06-06T03:40:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.297+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2541628990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0e44111900 con 0x7f0e441019d0 2024-06-06T03:40:07.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.298+0000 7f0e437fe700 1 --2- 172.21.15.40:0/2541628990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e44102380 0x7f0e4419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:07.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.298+0000 7f0e48945700 1 -- 172.21.15.40:0/2541628990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e441019d0 msgr2=0x7f0e4419d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:40:07.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.298+0000 7f0e48945700 1 --2- 172.21.15.40:0/2541628990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e441019d0 0x7f0e4419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:07.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.298+0000 7f0e48945700 1 -- 172.21.15.40:0/2541628990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e44102380 msgr2=0x7f0e4419dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:40:07.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.298+0000 7f0e48945700 1 --2- 172.21.15.40:0/2541628990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e44102380 0x7f0e4419dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.298+0000 7f0e48945700 1 -- 172.21.15.40:0/2541628990 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e44198160 con 0x7f0e44106720 2024-06-06T03:40:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.298+0000 7f0e48945700 1 --2- 172.21.15.40:0/2541628990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e44106720 0x7f0e44197a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c006080 tx=0x7f0e3c00cfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:40:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.299+0000 7f0e417fa700 1 -- 172.21.15.40:0/2541628990 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c024020 con 0x7f0e44106720 2024-06-06T03:40:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.299+0000 7f0e417fa700 1 -- 172.21.15.40:0/2541628990 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e3c01edb0 con 0x7f0e44106720 2024-06-06T03:40:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.299+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2541628990 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e441983f0 con 0x7f0e44106720 2024-06-06T03:40:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.299+0000 7f0e417fa700 1 -- 172.21.15.40:0/2541628990 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0057c0 con 0x7f0e44106720 2024-06-06T03:40:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.300+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2541628990 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4406ab60 con 0x7f0e44106720 2024-06-06T03:40:07.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.301+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2541628990 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e44102ad0 con 0x7f0e44106720 2024-06-06T03:40:07.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.303+0000 7f0e417fa700 1 -- 172.21.15.40:0/2541628990 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f0e3c027020 con 0x7f0e44106720 2024-06-06T03:40:07.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.304+0000 7f0e417fa700 1 --2- 172.21.15.40:0/2541628990 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f0e2c0a3000 0x7f0e2c0a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:07.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.304+0000 7f0e43fff700 1 --2- 172.21.15.40:0/2541628990 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f0e2c0a3000 0x7f0e2c0a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:07.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.304+0000 7f0e417fa700 1 -- 172.21.15.40:0/2541628990 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f0e3c0d4220 con 0x7f0e44106720 2024-06-06T03:40:07.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.305+0000 7f0e43fff700 1 --2- 172.21.15.40:0/2541628990 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f0e2c0a3000 0x7f0e2c0a54c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e34005950 tx=0x7f0e340058e0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:40:07.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.309+0000 7f0e417fa700 1 -- 172.21.15.40:0/2541628990 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0e3c019320 con 0x7f0e44106720 2024-06-06T03:40:07.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:07 smithi131 ceph-mon[36687]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-06-06T03:40:07.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:07 smithi104 ceph-mon[36569]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-06-06T03:40:07.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.683+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2541628990 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e44069180 con 0x7f0e44106720 2024-06-06T03:40:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:40:07.687 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:38:35.323771+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24443,24425,24398,24403]},"id":1}]} 2024-06-06T03:40:07.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.684+0000 7f0e417fa700 1 -- 172.21.15.40:0/2541628990 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11933 (secure 0 0 0) 0x7f0e3c0a0d10 con 0x7f0e44106720 2024-06-06T03:40:07.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.687+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2541628990 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f0e2c0a3000 msgr2=0x7f0e2c0a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:40:07.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.687+0000 7f0e4a3a8700 1 --2- 172.21.15.40:0/2541628990 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f0e2c0a3000 0x7f0e2c0a54c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e34005950 tx=0x7f0e340058e0 comp rx=0 tx=0).stop 2024-06-06T03:40:07.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.687+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2541628990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e44106720 msgr2=0x7f0e44197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:40:07.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.687+0000 7f0e4a3a8700 1 --2- 172.21.15.40:0/2541628990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e44106720 0x7f0e44197a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c006080 tx=0x7f0e3c00cfb0 comp rx=0 tx=0).stop 2024-06-06T03:40:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.688+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2541628990 shutdown_connections 2024-06-06T03:40:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.688+0000 7f0e4a3a8700 1 --2- 172.21.15.40:0/2541628990 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f0e2c0a3000 0x7f0e2c0a54c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.688+0000 7f0e4a3a8700 1 --2- 172.21.15.40:0/2541628990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e441019d0 0x7f0e4419d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.688+0000 7f0e4a3a8700 1 --2- 172.21.15.40:0/2541628990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e44106720 0x7f0e44197a00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.688+0000 7f0e4a3a8700 1 --2- 172.21.15.40:0/2541628990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e44102380 0x7f0e4419dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.688+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2541628990 >> 172.21.15.40:0/2541628990 conn(0x7f0e44078340 msgr2=0x7f0e440ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:40:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.688+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2541628990 shutdown_connections 2024-06-06T03:40:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:07.688+0000 7f0e4a3a8700 1 -- 172.21.15.40:0/2541628990 wait complete. 2024-06-06T03:40:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 29 2024-06-06T03:40:08.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:08 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2541628990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:40:08.392 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 899a3131 --release --await' 2024-06-06T03:40:08.393 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 899a3131 --release --await 2024-06-06T03:40:08.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:08 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2541628990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:40:08.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:08 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2541628990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:40:08.688 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:40:09.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:09 smithi131 ceph-mon[36687]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-06-06T03:40:09.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:09 smithi040 ceph-mon[32220]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-06-06T03:40:09.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:09 smithi104 ceph-mon[36569]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-06-06T03:40:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.437+0000 7fcb54e3f700 1 -- 172.21.15.40:0/3578660923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5006ab30 msgr2=0x7fcb5010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:40:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.437+0000 7fcb54e3f700 1 --2- 172.21.15.40:0/3578660923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5006ab30 0x7fcb5010f320 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800b0a0 tx=0x7fcb4800b3b0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.437+0000 7fcb54e3f700 1 -- 172.21.15.40:0/3578660923 shutdown_connections 2024-06-06T03:40:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.437+0000 7fcb54e3f700 1 --2- 172.21.15.40:0/3578660923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5006ab30 0x7fcb5010f320 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.437+0000 7fcb54e3f700 1 --2- 172.21.15.40:0/3578660923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb50108500 0x7fcb501088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.437+0000 7fcb54e3f700 1 --2- 172.21.15.40:0/3578660923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb5006a170 0x7fcb5006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.437+0000 7fcb54e3f700 1 -- 172.21.15.40:0/3578660923 >> 172.21.15.40:0/3578660923 conn(0x7fcb50078340 msgr2=0x7fcb50078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:40:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.437+0000 7fcb54e3f700 1 -- 172.21.15.40:0/3578660923 shutdown_connections 2024-06-06T03:40:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.437+0000 7fcb54e3f700 1 -- 172.21.15.40:0/3578660923 wait complete. 2024-06-06T03:40:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.438+0000 7fcb54e3f700 1 Processor -- start 2024-06-06T03:40:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.438+0000 7fcb54e3f700 1 -- start start 2024-06-06T03:40:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.438+0000 7fcb54e3f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5006a170 0x7fcb50101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.438+0000 7fcb54e3f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb5006ab30 0x7fcb501019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.438+0000 7fcb54e3f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb50108500 0x7fcb50101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.439+0000 7fcb54e3f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50112760 con 0x7fcb5006a170 2024-06-06T03:40:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.439+0000 7fcb54e3f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb501125e0 con 0x7fcb5006ab30 2024-06-06T03:40:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.439+0000 7fcb54e3f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcb501128e0 con 0x7fcb50108500 2024-06-06T03:40:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.439+0000 7fcb4ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb50108500 0x7fcb50101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.439+0000 7fcb4e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5006a170 0x7fcb50101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.439+0000 7fcb4e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5006a170 0x7fcb50101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34494/0 (socket says 172.21.15.40:34494) 2024-06-06T03:40:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.439+0000 7fcb4e59c700 1 -- 172.21.15.40:0/3089222776 learned_addr learned my addr 172.21.15.40:0/3089222776 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:40:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.439+0000 7fcb4dd9b700 1 --2- 172.21.15.40:0/3089222776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb5006ab30 0x7fcb501019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.440+0000 7fcb4ed9d700 1 -- 172.21.15.40:0/3089222776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb5006ab30 msgr2=0x7fcb501019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:40:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.440+0000 7fcb4ed9d700 1 --2- 172.21.15.40:0/3089222776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb5006ab30 0x7fcb501019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.440+0000 7fcb4ed9d700 1 -- 172.21.15.40:0/3089222776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5006a170 msgr2=0x7fcb50101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:40:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.440+0000 7fcb4ed9d700 1 --2- 172.21.15.40:0/3089222776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5006a170 0x7fcb50101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.440+0000 7fcb4ed9d700 1 -- 172.21.15.40:0/3089222776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb501ab4e0 con 0x7fcb50108500 2024-06-06T03:40:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.440+0000 7fcb4e59c700 1 --2- 172.21.15.40:0/3089222776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5006a170 0x7fcb50101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:40:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.440+0000 7fcb4ed9d700 1 --2- 172.21.15.40:0/3089222776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb50108500 0x7fcb50101f00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800d010 tx=0x7fcb48004dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:40:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.440+0000 7fcb3f7fe700 1 -- 172.21.15.40:0/3089222776 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48021070 con 0x7fcb50108500 2024-06-06T03:40:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.440+0000 7fcb54e3f700 1 -- 172.21.15.40:0/3089222776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb501ab770 con 0x7fcb50108500 2024-06-06T03:40:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.440+0000 7fcb3f7fe700 1 -- 172.21.15.40:0/3089222776 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb48007c40 con 0x7fcb50108500 2024-06-06T03:40:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.440+0000 7fcb3f7fe700 1 -- 172.21.15.40:0/3089222776 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb480052a0 con 0x7fcb50108500 2024-06-06T03:40:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.440+0000 7fcb54e3f700 1 -- 172.21.15.40:0/3089222776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb501abc20 con 0x7fcb50108500 2024-06-06T03:40:10.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.444+0000 7fcb5f6c2b80 1 Processor -- start 2024-06-06T03:40:10.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.444+0000 7fcb3f7fe700 1 -- 172.21.15.40:0/3089222776 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcb48024070 con 0x7fcb50108500 2024-06-06T03:40:10.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.445+0000 7fcb5f6c2b80 1 -- start start 2024-06-06T03:40:10.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.445+0000 7fcb5f6c2b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dd500 0x5627509dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.445+0000 7fcb5f6c2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dde20 0x5627509d7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.445+0000 7fcb4dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dd500 0x5627509dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:10.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.445+0000 7fcb5f6c2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509d7c50 0x5627509d8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.445+0000 7fcb5f6c2b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5627508ca060 con 0x5627509dd500 2024-06-06T03:40:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.445+0000 7fcb5f6c2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562750928950 con 0x5627509d7c50 2024-06-06T03:40:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.445+0000 7fcb5f6c2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562750890560 con 0x5627509dde20 2024-06-06T03:40:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.445+0000 7fcb4dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dd500 0x5627509dd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34506/0 (socket says 172.21.15.40:34506) 2024-06-06T03:40:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.445+0000 7fcb4dd9b700 1 -- 172.21.15.40:0/2391152820 learned_addr learned my addr 172.21.15.40:0/2391152820 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:40:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.446+0000 7fcb4e59c700 1 --2- 172.21.15.40:0/2391152820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dde20 0x5627509d7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.446+0000 7fcb3f7fe700 1 --2- 172.21.15.40:0/3089222776 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fcb380a2e90 0x7fcb380a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.446+0000 7fcb4dd9b700 1 -- 172.21.15.40:0/2391152820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dde20 msgr2=0x5627509d7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:40:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.446+0000 7fcb4dd9b700 1 --2- 172.21.15.40:0/2391152820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dde20 0x5627509d7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.446+0000 7fcb4dd9b700 1 -- 172.21.15.40:0/2391152820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509d7c50 msgr2=0x5627509d8070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T03:40:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.446+0000 7fcb4dd9b700 1 --2- 172.21.15.40:0/2391152820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509d7c50 0x5627509d8070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.446+0000 7fcb4dd9b700 1 -- 172.21.15.40:0/2391152820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627508779d0 con 0x5627509dd500 2024-06-06T03:40:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.446+0000 7fcb3f7fe700 1 -- 172.21.15.40:0/3089222776 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fcb4800f020 con 0x7fcb50108500 2024-06-06T03:40:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.446+0000 7fcb4dd9b700 1 --2- 172.21.15.40:0/2391152820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dd500 0x5627509dd8e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400abd0 tx=0x7fcb4400aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7771c6f63fb56985 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:40:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.447+0000 7fcb3cff9700 1 -- 172.21.15.40:0/2391152820 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400fde0 con 0x5627509dd500 2024-06-06T03:40:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.447+0000 7fcb3cff9700 1 -- 172.21.15.40:0/2391152820 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb44004500 con 0x5627509dd500 2024-06-06T03:40:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.447+0000 7fcb4ed9d700 1 --2- 172.21.15.40:0/3089222776 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fcb380a2e90 0x7fcb380a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.447+0000 7fcb3cff9700 1 -- 172.21.15.40:0/2391152820 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb440186a0 con 0x5627509dd500 2024-06-06T03:40:10.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.447+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/2391152820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dd500 msgr2=0x5627509dd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:40:10.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.447+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/2391152820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dd500 0x5627509dd8e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400abd0 tx=0x7fcb4400aee0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.448+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/2391152820 shutdown_connections 2024-06-06T03:40:10.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.448+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/2391152820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509d7c50 0x5627509d8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.448+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/2391152820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dd500 0x5627509dd8e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.448+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/2391152820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dde20 0x5627509d7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.448+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/2391152820 >> 172.21.15.40:0/2391152820 conn(0x562750904180 msgr2=0x5627509d2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:40:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.448+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/2391152820 shutdown_connections 2024-06-06T03:40:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.448+0000 7fcb4ed9d700 1 --2- 172.21.15.40:0/3089222776 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fcb380a2e90 0x7fcb380a5350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcb4809cb40 tx=0x7fcb48009040 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:40:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.448+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/2391152820 wait complete. 2024-06-06T03:40:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.448+0000 7fcb5f6c2b80 1 -- start start 2024-06-06T03:40:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.449+0000 7fcb5f6c2b80 1 Processor -- start 2024-06-06T03:40:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.449+0000 7fcb5f6c2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509d7c50 0x5627509e3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.449+0000 7fcb5f6c2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dd500 0x5627509e0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.449+0000 7fcb5f6c2b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dde20 0x5627509e12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.449+0000 7fcb4dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dde20 0x5627509e12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.449+0000 7fcb5f6c2b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5627508c90c0 con 0x5627509dde20 2024-06-06T03:40:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.449+0000 7fcb4dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dde20 0x5627509e12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34510/0 (socket says 172.21.15.40:34510) 2024-06-06T03:40:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.449+0000 7fcb4dd9b700 1 -- 172.21.15.40:0/944958007 learned_addr learned my addr 172.21.15.40:0/944958007 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:40:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.449+0000 7fcb5f6c2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562750924910 con 0x5627509d7c50 2024-06-06T03:40:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.449+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/944958007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5627509e48a0 con 0x5627509dd500 2024-06-06T03:40:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.449+0000 7fcb4e59c700 1 --2- 172.21.15.40:0/944958007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dd500 0x5627509e0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.450+0000 7fcb4dd9b700 1 -- 172.21.15.40:0/944958007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dd500 msgr2=0x5627509e0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:40:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.450+0000 7fcb4dd9b700 1 --2- 172.21.15.40:0/944958007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dd500 0x5627509e0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.450+0000 7fcb4dd9b700 1 -- 172.21.15.40:0/944958007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509d7c50 msgr2=0x5627509e3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:40:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.450+0000 7fcb4dd9b700 1 --2- 172.21.15.40:0/944958007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509d7c50 0x5627509e3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.450+0000 7fcb4dd9b700 1 -- 172.21.15.40:0/944958007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627508ee850 con 0x5627509dde20 2024-06-06T03:40:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.450+0000 7fcb4dd9b700 1 --2- 172.21.15.40:0/944958007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dde20 0x5627509e12a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcb44003d10 tx=0x7fcb4400fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b558999cc4f179 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:40:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.450+0000 7fcb35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:40:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.450+0000 7fcb5f6c2b80 10 client.0 _ll_register_callbacks cb 0x5627508be6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:40:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.450+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/944958007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb44004500 con 0x5627509dde20 2024-06-06T03:40:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.450+0000 7fcb35ffb700 1 -- 172.21.15.40:0/944958007 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4401c040 con 0x5627509dde20 2024-06-06T03:40:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.450+0000 7fcb35ffb700 1 -- 172.21.15.40:0/944958007 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fcb440042e0 con 0x5627509dde20 2024-06-06T03:40:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.450+0000 7fcb35ffb700 1 -- 172.21.15.40:0/944958007 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcb44004750 con 0x5627509dde20 2024-06-06T03:40:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.450+0000 7fcb5f6c2b80 10 client.15468 fetch_fsmap learned FSMap version 29 2024-06-06T03:40:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.451+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/944958007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5627509eec00 con 0x5627509dde20 2024-06-06T03:40:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.451+0000 7fcb35ffb700 1 -- 172.21.15.40:0/944958007 <== mon.0 v2:172.21.15.40:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcb44022070 con 0x5627509dde20 2024-06-06T03:40:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.451+0000 7fcb5f6c2b80 10 client.15468 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T03:40:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.451+0000 7fcb5f6c2b80 10 client.15468 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:40:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.451+0000 7fcb5f6c2b80 20 client.15468 populate_metadata read hostname 'smithi040' 2024-06-06T03:40:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.451+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/944958007 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x5627509f10d0 0x5627509f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.451+0000 7fcb5f6c2b80 4 client.15468 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:40:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.451+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/944958007 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562750935e50 con 0x5627509f10d0 2024-06-06T03:40:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.451+0000 7fcb4e59c700 1 --2- 172.21.15.40:0/944958007 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x5627509f10d0 0x5627509f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.452+0000 7fcb35ffb700 10 client.15468 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:40:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.452+0000 7fcb4e59c700 1 --2- 172.21.15.40:0/944958007 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x5627509f10d0 0x5627509f34c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=805ce82125a64bc0 server_cookie=99fa2a7d0610a89e in_seq=0 out_seq=0 2024-06-06T03:40:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb35ffb700 1 -- 172.21.15.40:0/944958007 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562750935e50 con 0x5627509f10d0 2024-06-06T03:40:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb35ffb700 10 client.15468 handle_command_reply: tid=0 2024-06-06T03:40:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb5f6c2b80 1 client.15468 shutdown 2024-06-06T03:40:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/944958007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dde20 msgr2=0x5627509e12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:40:10.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/944958007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dde20 0x5627509e12a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcb44003d10 tx=0x7fcb4400fef0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/944958007 shutdown_connections 2024-06-06T03:40:10.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/944958007 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x5627509f10d0 0x5627509f34c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/944958007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dde20 0x5627509e12a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/944958007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509d7c50 0x5627509e3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/944958007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dd500 0x5627509e0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb35ffb700 0 client.15468 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:40:10.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/944958007 >> 172.21.15.40:0/944958007 conn(0x562750904180 msgr2=0x5627509d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:40:10.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/944958007 shutdown_connections 2024-06-06T03:40:10.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/944958007 wait complete. 2024-06-06T03:40:10.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.453+0000 7fcb5f6c2b80 20 client.15468 trim_cache size 0 max 16384 2024-06-06T03:40:10.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.474+0000 7fcb5f6c2b80 1 Processor -- start 2024-06-06T03:40:10.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.474+0000 7fcb5f6c2b80 1 -- start start 2024-06-06T03:40:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb5f6c2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509d7c50 0x5627509d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb5f6c2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dd500 0x5627509d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb5f6c2b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dde20 0x5627509f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb5f6c2b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3000d450 con 0x5627509dde20 2024-06-06T03:40:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb5f6c2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3000ef90 con 0x5627509d7c50 2024-06-06T03:40:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb5f6c2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3000db20 con 0x5627509dd500 2024-06-06T03:40:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb4e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dd500 0x5627509d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb4e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dd500 0x5627509d2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.40:58730/0 (socket says 172.21.15.40:58730) 2024-06-06T03:40:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb4e59c700 1 -- 172.21.15.40:0/3306187587 learned_addr learned my addr 172.21.15.40:0/3306187587 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:40:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb4dd9b700 1 --2- 172.21.15.40:0/3306187587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509d7c50 0x5627509d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb4e59c700 1 -- 172.21.15.40:0/3306187587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509d7c50 msgr2=0x5627509d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:40:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb4e59c700 1 --2- 172.21.15.40:0/3306187587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509d7c50 0x5627509d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb4e59c700 1 -- 172.21.15.40:0/3306187587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dde20 msgr2=0x5627509f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:40:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb4e59c700 1 --2- 172.21.15.40:0/3306187587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dde20 0x5627509f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.475+0000 7fcb4e59c700 1 -- 172.21.15.40:0/3306187587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627509db810 con 0x5627509dd500 2024-06-06T03:40:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.476+0000 7fcb4e59c700 1 --2- 172.21.15.40:0/3306187587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dd500 0x5627509d2870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb40000c00 tx=0x7fcb40012990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232238ebab059de9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:40:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.476+0000 7fcb37fff700 1 -- 172.21.15.40:0/3306187587 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40004a30 con 0x5627509dd500 2024-06-06T03:40:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.476+0000 7fcb37fff700 1 -- 172.21.15.40:0/3306187587 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb40005500 con 0x5627509dd500 2024-06-06T03:40:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.476+0000 7fcb37fff700 1 -- 172.21.15.40:0/3306187587 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb400117c0 con 0x5627509dd500 2024-06-06T03:40:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.477+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/3306187587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dd500 msgr2=0x5627509d2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:40:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.477+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/3306187587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dd500 0x5627509d2870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb40000c00 tx=0x7fcb40012990 comp rx=0 tx=0).stop 2024-06-06T03:40:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.477+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/3306187587 shutdown_connections 2024-06-06T03:40:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.477+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/3306187587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509dde20 0x5627509f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.477+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/3306187587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509d7c50 0x5627509d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.477+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/3306187587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dd500 0x5627509d2870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.477+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/3306187587 >> 172.21.15.40:0/3306187587 conn(0x562750904180 msgr2=0x5627509e0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:40:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.477+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/3306187587 shutdown_connections 2024-06-06T03:40:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.477+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/3306187587 wait complete. 2024-06-06T03:40:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.477+0000 7fcb5f6c2b80 1 -- start start 2024-06-06T03:40:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.477+0000 7fcb5f6c2b80 1 Processor -- start 2024-06-06T03:40:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb5f6c2b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509d7c50 0x5627509ea280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb5f6c2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dde20 0x5627509e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb5f6c2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509e7120 0x5627509e75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb4dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509d7c50 0x5627509ea280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb4dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509d7c50 0x5627509ea280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34520/0 (socket says 172.21.15.40:34520) 2024-06-06T03:40:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb4dd9b700 1 -- 172.21.15.40:0/765151067 learned_addr learned my addr 172.21.15.40:0/765151067 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:40:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/765151067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3000e8c0 con 0x5627509d7c50 2024-06-06T03:40:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/765151067 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3000e1f0 con 0x5627509e7120 2024-06-06T03:40:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/765151067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3000b3f0 con 0x5627509dde20 2024-06-06T03:40:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb4e59c700 1 --2- 172.21.15.40:0/765151067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dde20 0x5627509e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb4dd9b700 1 -- 172.21.15.40:0/765151067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dde20 msgr2=0x5627509e6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:40:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb4dd9b700 1 --2- 172.21.15.40:0/765151067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dde20 0x5627509e6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb4dd9b700 1 -- 172.21.15.40:0/765151067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509e7120 msgr2=0x5627509e75a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T03:40:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb4dd9b700 1 --2- 172.21.15.40:0/765151067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509e7120 0x5627509e75a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.478+0000 7fcb4dd9b700 1 -- 172.21.15.40:0/765151067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562750a5bc60 con 0x5627509d7c50 2024-06-06T03:40:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.479+0000 7fcb4e59c700 1 --2- 172.21.15.40:0/765151067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dde20 0x5627509e6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:40:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.479+0000 7fcb35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:40:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.479+0000 7fcb5f6c2b80 10 client.0 _ll_register_callbacks cb 0x5627508be6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:40:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.479+0000 7fcb4dd9b700 1 --2- 172.21.15.40:0/765151067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509d7c50 0x5627509ea280 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcb44004940 tx=0x7fcb4400a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d5ce366654e92e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:40:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.479+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/765151067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb40005500 con 0x5627509d7c50 2024-06-06T03:40:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.479+0000 7fcb35ffb700 1 -- 172.21.15.40:0/765151067 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4403b020 con 0x5627509d7c50 2024-06-06T03:40:10.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.479+0000 7fcb35ffb700 1 -- 172.21.15.40:0/765151067 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fcb44037480 con 0x5627509d7c50 2024-06-06T03:40:10.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.479+0000 7fcb35ffb700 1 -- 172.21.15.40:0/765151067 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcb4401c040 con 0x5627509d7c50 2024-06-06T03:40:10.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.479+0000 7fcb5f6c2b80 10 client.15471 fetch_fsmap learned FSMap version 29 2024-06-06T03:40:10.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.479+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/765151067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({fsmap=29}) -- 0x562750a5cd00 con 0x5627509d7c50 2024-06-06T03:40:10.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.480+0000 7fcb35ffb700 1 -- 172.21.15.40:0/765151067 <== mon.0 v2:172.21.15.40:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcb44037720 con 0x5627509d7c50 2024-06-06T03:40:10.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.480+0000 7fcb5f6c2b80 10 client.15471 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T03:40:10.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.480+0000 7fcb5f6c2b80 10 client.15471 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:40:10.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.480+0000 7fcb5f6c2b80 20 client.15471 populate_metadata read hostname 'smithi040' 2024-06-06T03:40:10.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.480+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/765151067 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x562750a5d370 0x562750a5f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:40:10.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.480+0000 7fcb5f6c2b80 4 client.15471 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "899a3131", "release": true, "await": true}] 2024-06-06T03:40:10.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.480+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/765151067 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "set_id": "899a3131", "release": true, "await": true}) -- 0x5627509303d0 con 0x562750a5d370 2024-06-06T03:40:10.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.480+0000 7fcb4e59c700 1 --2- 172.21.15.40:0/765151067 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x562750a5d370 0x562750a5f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:40:10.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.481+0000 7fcb35ffb700 10 client.15471 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:40:10.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.481+0000 7fcb4e59c700 1 --2- 172.21.15.40:0/765151067 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x562750a5d370 0x562750a5f7d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=180e4b4e44ab4a85 server_cookie=1fd07062dbc387e7 in_seq=0 out_seq=0 2024-06-06T03:40:10.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb35ffb700 1 -- 172.21.15.40:0/765151067 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5627509303d0 con 0x562750a5d370 2024-06-06T03:40:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb35ffb700 10 client.15471 handle_command_reply: tid=0 2024-06-06T03:40:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb5f6c2b80 1 client.15471 shutdown 2024-06-06T03:40:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/765151067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509d7c50 msgr2=0x5627509ea280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:40:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/765151067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509d7c50 0x5627509ea280 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcb44004940 tx=0x7fcb4400a300 comp rx=0 tx=0).stop 2024-06-06T03:40:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/765151067 shutdown_connections 2024-06-06T03:40:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/765151067 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x562750a5d370 0x562750a5f7d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb35ffb700 0 client.15471 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:40:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/765151067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5627509e7120 0x5627509e75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/765151067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627509d7c50 0x5627509ea280 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb5f6c2b80 1 --2- 172.21.15.40:0/765151067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627509dde20 0x5627509e6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/765151067 >> 172.21.15.40:0/765151067 conn(0x562750904180 msgr2=0x5627509e2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:40:10.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/765151067 shutdown_connections 2024-06-06T03:40:10.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb5f6c2b80 1 -- 172.21.15.40:0/765151067 wait complete. 2024-06-06T03:40:10.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.483+0000 7fcb5f6c2b80 20 client.15471 trim_cache size 0 max 16384 2024-06-06T03:40:10.488 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:40:10.488 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 29, 2024-06-06T03:40:10.489 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:40:10.489 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 4, 2024-06-06T03:40:10.489 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": { 2024-06-06T03:40:10.489 INFO:teuthology.orchestra.run.smithi040.stdout: "899a3131": { 2024-06-06T03:40:10.489 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 4, 2024-06-06T03:40:10.489 INFO:teuthology.orchestra.run.smithi040.stdout: "age_ref": 0.0, 2024-06-06T03:40:10.489 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:40:10.489 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RELEASED", 2024-06-06T03:40:10.489 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:40:10.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:40:10.490 INFO:teuthology.orchestra.run.smithi040.stdout: "timeout": 90.0, 2024-06-06T03:40:10.490 INFO:teuthology.orchestra.run.smithi040.stdout: "expiration": 154.4, 2024-06-06T03:40:10.490 INFO:teuthology.orchestra.run.smithi040.stdout: "members": { 2024-06-06T03:40:10.490 INFO:teuthology.orchestra.run.smithi040.stdout: "file:/": { 2024-06-06T03:40:10.490 INFO:teuthology.orchestra.run.smithi040.stdout: "excluded": false, 2024-06-06T03:40:10.490 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:40:10.490 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RELEASED", 2024-06-06T03:40:10.490 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:40:10.490 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:40:10.490 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:40:10.490 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:40:10.490 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:40:10.491 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:40:10.491 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:40:10.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.484+0000 7fcb54e3f700 1 -- 172.21.15.40:0/3089222776 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fcb380a2e90 msgr2=0x7fcb380a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:40:10.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.484+0000 7fcb54e3f700 1 --2- 172.21.15.40:0/3089222776 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fcb380a2e90 0x7fcb380a5350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcb4809cb40 tx=0x7fcb48009040 comp rx=0 tx=0).stop 2024-06-06T03:40:10.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.484+0000 7fcb54e3f700 1 -- 172.21.15.40:0/3089222776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb50108500 msgr2=0x7fcb50101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:40:10.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.484+0000 7fcb54e3f700 1 --2- 172.21.15.40:0/3089222776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb50108500 0x7fcb50101f00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800d010 tx=0x7fcb48004dd0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.484+0000 7fcb54e3f700 1 -- 172.21.15.40:0/3089222776 shutdown_connections 2024-06-06T03:40:10.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.485+0000 7fcb54e3f700 1 --2- 172.21.15.40:0/3089222776 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fcb380a2e90 0x7fcb380a5350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.485+0000 7fcb54e3f700 1 --2- 172.21.15.40:0/3089222776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb50108500 0x7fcb50101f00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.485+0000 7fcb54e3f700 1 --2- 172.21.15.40:0/3089222776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5006a170 0x7fcb50101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.485+0000 7fcb54e3f700 1 --2- 172.21.15.40:0/3089222776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb5006ab30 0x7fcb501019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:40:10.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.485+0000 7fcb54e3f700 1 -- 172.21.15.40:0/3089222776 >> 172.21.15.40:0/3089222776 conn(0x7fcb50078340 msgr2=0x7fcb500ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:40:10.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.485+0000 7fcb54e3f700 1 -- 172.21.15.40:0/3089222776 shutdown_connections 2024-06-06T03:40:10.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:40:10.485+0000 7fcb54e3f700 1 -- 172.21.15.40:0/3089222776 wait complete. 2024-06-06T03:40:11.353 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '899a3131', total seconds elapsed: 40.8 2024-06-06T03:40:11.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:11 smithi131 ceph-mon[36687]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-06-06T03:40:11.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:11 smithi131 ceph-mon[36687]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-06-06T03:40:11.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:11 smithi040 ceph-mon[32220]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-06-06T03:40:11.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:11 smithi040 ceph-mon[32220]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-06-06T03:40:11.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:11 smithi104 ceph-mon[36569]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-06-06T03:40:11.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:11 smithi104 ceph-mon[36569]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-06-06T03:40:13.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:13 smithi131 ceph-mon[36687]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-06T03:40:13.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:13 smithi040 ceph-mon[32220]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-06T03:40:13.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:13 smithi104 ceph-mon[36569]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-06T03:40:15.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:15 smithi131 ceph-mon[36687]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-06-06T03:40:15.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:15 smithi040 ceph-mon[32220]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-06-06T03:40:15.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:15 smithi104 ceph-mon[36569]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-06-06T03:40:16.082 INFO:tasks.workunit.client.0.smithi040.stderr:Updating files: 97% (12194/12568) Updating files: 98% (12317/12568) Updating files: 99% (12443/12568) Updating files: 100% (12568/12568) Updating files: 100% (12568/12568), done. 2024-06-06T03:40:16.371 INFO:tasks.workunit.client.0.smithi040.stderr:Note: switching to '3d746ca5346f34eef7e826a1eb71e59e02b682ae'. 2024-06-06T03:40:16.371 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-06-06T03:40:16.371 INFO:tasks.workunit.client.0.smithi040.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-06T03:40:16.371 INFO:tasks.workunit.client.0.smithi040.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-06T03:40:16.371 INFO:tasks.workunit.client.0.smithi040.stderr:state without impacting any branches by switching back to a branch. 2024-06-06T03:40:16.371 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-06-06T03:40:16.371 INFO:tasks.workunit.client.0.smithi040.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-06T03:40:16.371 INFO:tasks.workunit.client.0.smithi040.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-06T03:40:16.371 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-06-06T03:40:16.372 INFO:tasks.workunit.client.0.smithi040.stderr: git switch -c 2024-06-06T03:40:16.372 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-06-06T03:40:16.372 INFO:tasks.workunit.client.0.smithi040.stderr:Or undo this operation with: 2024-06-06T03:40:16.372 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-06-06T03:40:16.372 INFO:tasks.workunit.client.0.smithi040.stderr: git switch - 2024-06-06T03:40:16.372 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-06-06T03:40:16.372 INFO:tasks.workunit.client.0.smithi040.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-06T03:40:16.372 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-06-06T03:40:16.372 INFO:tasks.workunit.client.0.smithi040.stderr:HEAD is now at 3d746ca5346 Merge pull request #57870 from zdover23/wip-doc-2024-06-05-backport-57867-to-squid 2024-06-06T03:40:16.376 DEBUG:teuthology.orchestra.run.smithi040:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-06T03:40:16.415 INFO:tasks.workunit.client.0.smithi040.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-06T03:40:16.417 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-06T03:40:16.417 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-06T03:40:17.141 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-06T03:40:17.195 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-06T03:40:17.236 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-06T03:40:17.238 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-06T03:40:17.238 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-06T03:40:17.282 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-06T03:40:17.283 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:40:17.283 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-06T03:40:17.342 INFO:tasks.workunit:Running workunits matching kernel_untar_build.sh on client.0... 2024-06-06T03:40:17.343 INFO:tasks.workunit:Running workunit kernel_untar_build.sh... 2024-06-06T03:40:17.343 DEBUG:teuthology.orchestra.run.smithi040:workunit test kernel_untar_build.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d746ca5346f34eef7e826a1eb71e59e02b682ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh 2024-06-06T03:40:17.405 INFO:tasks.workunit.client.0.smithi040.stderr:+ wget -O linux.tar.xz http://download.ceph.com/qa/linux-6.5.11.tar.xz 2024-06-06T03:40:17.424 INFO:tasks.workunit.client.0.smithi040.stderr:--2024-06-06 03:40:17-- http://download.ceph.com/qa/linux-6.5.11.tar.xz 2024-06-06T03:40:17.435 INFO:tasks.workunit.client.0.smithi040.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-06-06T03:40:17.471 INFO:tasks.workunit.client.0.smithi040.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-06-06T03:40:17.508 INFO:tasks.workunit.client.0.smithi040.stderr:HTTP request sent, awaiting response... 200 OK 2024-06-06T03:40:17.509 INFO:tasks.workunit.client.0.smithi040.stderr:Length: 139010916 (133M) [application/octet-stream] 2024-06-06T03:40:17.509 INFO:tasks.workunit.client.0.smithi040.stderr:Saving to: ‘linux.tar.xz’ 2024-06-06T03:40:17.511 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-06-06T03:40:17.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:17 smithi131 ceph-mon[36687]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-06T03:40:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:17 smithi040 ceph-mon[32220]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-06T03:40:17.582 INFO:tasks.workunit.client.0.smithi040.stderr: 0K .......... .......... .......... .......... .......... 0% 689K 3m17s 2024-06-06T03:40:17.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:17 smithi104 ceph-mon[36569]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-06T03:40:17.619 INFO:tasks.workunit.client.0.smithi040.stderr: 50K .......... .......... .......... .......... .......... 0% 1.37M 2m27s 2024-06-06T03:40:17.620 INFO:tasks.workunit.client.0.smithi040.stderr: 100K .......... .......... .......... .......... .......... 0% 123M 98s 2024-06-06T03:40:17.620 INFO:tasks.workunit.client.0.smithi040.stderr: 150K .......... .......... .......... .......... .......... 0% 89.0M 74s 2024-06-06T03:40:17.657 INFO:tasks.workunit.client.0.smithi040.stderr: 200K .......... .......... .......... .......... .......... 0% 1.39M 78s 2024-06-06T03:40:17.657 INFO:tasks.workunit.client.0.smithi040.stderr: 250K .......... .......... .......... .......... .......... 0% 129M 65s 2024-06-06T03:40:17.657 INFO:tasks.workunit.client.0.smithi040.stderr: 300K .......... .......... .......... .......... .......... 0% 110M 56s 2024-06-06T03:40:17.657 INFO:tasks.workunit.client.0.smithi040.stderr: 350K .......... .......... .......... .......... .......... 0% 110M 49s 2024-06-06T03:40:17.700 INFO:tasks.workunit.client.0.smithi040.stderr: 400K .......... .......... .......... .......... .......... 0% 1.41M 54s 2024-06-06T03:40:17.700 INFO:tasks.workunit.client.0.smithi040.stderr: 450K .......... .......... .......... .......... .......... 0% 163M 49s 2024-06-06T03:40:17.700 INFO:tasks.workunit.client.0.smithi040.stderr: 500K .......... .......... .......... .......... .......... 0% 138M 44s 2024-06-06T03:40:17.700 INFO:tasks.workunit.client.0.smithi040.stderr: 550K .......... .......... .......... .......... .......... 0% 97.9M 41s 2024-06-06T03:40:17.700 INFO:tasks.workunit.client.0.smithi040.stderr: 600K .......... .......... .......... .......... .......... 0% 153M 38s 2024-06-06T03:40:17.700 INFO:tasks.workunit.client.0.smithi040.stderr: 650K .......... .......... .......... .......... .......... 0% 91.5M 35s 2024-06-06T03:40:17.700 INFO:tasks.workunit.client.0.smithi040.stderr: 700K .......... .......... .......... .......... .......... 0% 112M 33s 2024-06-06T03:40:17.701 INFO:tasks.workunit.client.0.smithi040.stderr: 750K .......... .......... .......... .......... .......... 0% 64.4M 31s 2024-06-06T03:40:17.701 INFO:tasks.workunit.client.0.smithi040.stderr: 800K .......... .......... .......... .......... .......... 0% 50.4M 29s 2024-06-06T03:40:17.749 INFO:tasks.workunit.client.0.smithi040.stderr: 850K .......... .......... .......... .......... .......... 0% 1.48M 33s 2024-06-06T03:40:17.749 INFO:tasks.workunit.client.0.smithi040.stderr: 900K .......... .......... .......... .......... .......... 0% 75.8M 31s 2024-06-06T03:40:17.749 INFO:tasks.workunit.client.0.smithi040.stderr: 950K .......... .......... .......... .......... .......... 0% 81.5M 29s 2024-06-06T03:40:17.749 INFO:tasks.workunit.client.0.smithi040.stderr: 1000K .......... .......... .......... .......... .......... 0% 73.2M 28s 2024-06-06T03:40:17.749 INFO:tasks.workunit.client.0.smithi040.stderr: 1050K .......... .......... .......... .......... .......... 0% 75.4M 27s 2024-06-06T03:40:17.749 INFO:tasks.workunit.client.0.smithi040.stderr: 1100K .......... .......... .......... .......... .......... 0% 236M 26s 2024-06-06T03:40:17.750 INFO:tasks.workunit.client.0.smithi040.stderr: 1150K .......... .......... .......... .......... .......... 0% 132M 25s 2024-06-06T03:40:17.750 INFO:tasks.workunit.client.0.smithi040.stderr: 1200K .......... .......... .......... .......... .......... 0% 153M 24s 2024-06-06T03:40:17.750 INFO:tasks.workunit.client.0.smithi040.stderr: 1250K .......... .......... .......... .......... .......... 0% 30.0M 23s 2024-06-06T03:40:17.750 INFO:tasks.workunit.client.0.smithi040.stderr: 1300K .......... .......... .......... .......... .......... 0% 45.4M 22s 2024-06-06T03:40:17.750 INFO:tasks.workunit.client.0.smithi040.stderr: 1350K .......... .......... .......... .......... .......... 1% 28.6M 22s 2024-06-06T03:40:17.750 INFO:tasks.workunit.client.0.smithi040.stderr: 1400K .......... .......... .......... .......... .......... 1% 24.6M 21s 2024-06-06T03:40:17.750 INFO:tasks.workunit.client.0.smithi040.stderr: 1450K .......... .......... .......... .......... .......... 1% 23.8M 21s 2024-06-06T03:40:17.750 INFO:tasks.workunit.client.0.smithi040.stderr: 1500K .......... .......... .......... .......... .......... 1% 24.3M 20s 2024-06-06T03:40:17.750 INFO:tasks.workunit.client.0.smithi040.stderr: 1550K .......... .......... .......... .......... .......... 1% 60.8M 19s 2024-06-06T03:40:17.790 INFO:tasks.workunit.client.0.smithi040.stderr: 1600K .......... .......... .......... .......... .......... 1% 2.22M 21s 2024-06-06T03:40:17.790 INFO:tasks.workunit.client.0.smithi040.stderr: 1650K .......... .......... .......... .......... .......... 1% 58.7M 20s 2024-06-06T03:40:17.790 INFO:tasks.workunit.client.0.smithi040.stderr: 1700K .......... .......... .......... .......... .......... 1% 145M 20s 2024-06-06T03:40:17.790 INFO:tasks.workunit.client.0.smithi040.stderr: 1750K .......... .......... .......... .......... .......... 1% 89.8M 19s 2024-06-06T03:40:17.791 INFO:tasks.workunit.client.0.smithi040.stderr: 1800K .......... .......... .......... .......... .......... 1% 113M 19s 2024-06-06T03:40:17.791 INFO:tasks.workunit.client.0.smithi040.stderr: 1850K .......... .......... .......... .......... .......... 1% 99.3M 18s 2024-06-06T03:40:17.791 INFO:tasks.workunit.client.0.smithi040.stderr: 1900K .......... .......... .......... .......... .......... 1% 156M 18s 2024-06-06T03:40:17.791 INFO:tasks.workunit.client.0.smithi040.stderr: 1950K .......... .......... .......... .......... .......... 1% 90.5M 17s 2024-06-06T03:40:17.791 INFO:tasks.workunit.client.0.smithi040.stderr: 2000K .......... .......... .......... .......... .......... 1% 57.1M 17s 2024-06-06T03:40:17.791 INFO:tasks.workunit.client.0.smithi040.stderr: 2050K .......... .......... .......... .......... .......... 1% 47.2M 17s 2024-06-06T03:40:17.791 INFO:tasks.workunit.client.0.smithi040.stderr: 2100K .......... .......... .......... .......... .......... 1% 21.2M 16s 2024-06-06T03:40:17.791 INFO:tasks.workunit.client.0.smithi040.stderr: 2150K .......... .......... .......... .......... .......... 1% 28.3M 16s 2024-06-06T03:40:17.791 INFO:tasks.workunit.client.0.smithi040.stderr: 2200K .......... .......... .......... .......... .......... 1% 38.7M 16s 2024-06-06T03:40:17.791 INFO:tasks.workunit.client.0.smithi040.stderr: 2250K .......... .......... .......... .......... .......... 1% 25.8M 15s 2024-06-06T03:40:17.791 INFO:tasks.workunit.client.0.smithi040.stderr: 2300K .......... .......... .......... .......... .......... 1% 45.0M 15s 2024-06-06T03:40:17.791 INFO:tasks.workunit.client.0.smithi040.stderr: 2350K .......... .......... .......... .......... .......... 1% 25.9M 15s 2024-06-06T03:40:17.791 INFO:tasks.workunit.client.0.smithi040.stderr: 2400K .......... .......... .......... .......... .......... 1% 33.9M 15s 2024-06-06T03:40:17.791 INFO:tasks.workunit.client.0.smithi040.stderr: 2450K .......... .......... .......... .......... .......... 1% 22.9M 15s 2024-06-06T03:40:17.792 INFO:tasks.workunit.client.0.smithi040.stderr: 2500K .......... .......... .......... .......... .......... 1% 24.4M 14s 2024-06-06T03:40:17.792 INFO:tasks.workunit.client.0.smithi040.stderr: 2550K .......... .......... .......... .......... .......... 1% 62.0M 14s 2024-06-06T03:40:17.792 INFO:tasks.workunit.client.0.smithi040.stderr: 2600K .......... .......... .......... .......... .......... 1% 22.5M 14s 2024-06-06T03:40:17.792 INFO:tasks.workunit.client.0.smithi040.stderr: 2650K .......... .......... .......... .......... .......... 1% 21.0M 14s 2024-06-06T03:40:17.793 INFO:tasks.workunit.client.0.smithi040.stderr: 2700K .......... .......... .......... .......... .......... 2% 17.1M 14s 2024-06-06T03:40:17.794 INFO:tasks.workunit.client.0.smithi040.stderr: 2750K .......... .......... .......... .......... .......... 2% 23.3M 14s 2024-06-06T03:40:17.796 INFO:tasks.workunit.client.0.smithi040.stderr: 2800K .......... .......... .......... .......... .......... 2% 24.5M 13s 2024-06-06T03:40:17.798 INFO:tasks.workunit.client.0.smithi040.stderr: 2850K .......... .......... .......... .......... .......... 2% 25.3M 13s 2024-06-06T03:40:17.800 INFO:tasks.workunit.client.0.smithi040.stderr: 2900K .......... .......... .......... .......... .......... 2% 20.3M 13s 2024-06-06T03:40:17.804 INFO:tasks.workunit.client.0.smithi040.stderr: 2950K .......... .......... .......... .......... .......... 2% 12.9M 13s 2024-06-06T03:40:17.807 INFO:tasks.workunit.client.0.smithi040.stderr: 3000K .......... .......... .......... .......... .......... 2% 22.2M 13s 2024-06-06T03:40:17.808 INFO:tasks.workunit.client.0.smithi040.stderr: 3050K .......... .......... .......... .......... .......... 2% 20.2M 13s 2024-06-06T03:40:17.811 INFO:tasks.workunit.client.0.smithi040.stderr: 3100K .......... .......... .......... .......... .......... 2% 20.4M 13s 2024-06-06T03:40:17.813 INFO:tasks.workunit.client.0.smithi040.stderr: 3150K .......... .......... .......... .......... .......... 2% 23.0M 13s 2024-06-06T03:40:17.817 INFO:tasks.workunit.client.0.smithi040.stderr: 3200K .......... .......... .......... .......... .......... 2% 14.5M 13s 2024-06-06T03:40:17.820 INFO:tasks.workunit.client.0.smithi040.stderr: 3250K .......... .......... .......... .......... .......... 2% 16.1M 12s 2024-06-06T03:40:17.827 INFO:tasks.workunit.client.0.smithi040.stderr: 3300K .......... .......... .......... .......... .......... 2% 98.3M 12s 2024-06-06T03:40:17.827 INFO:tasks.workunit.client.0.smithi040.stderr: 3350K .......... .......... .......... .......... .......... 2% 21.7M 12s 2024-06-06T03:40:17.827 INFO:tasks.workunit.client.0.smithi040.stderr: 3400K .......... .......... .......... .......... .......... 2% 22.4M 12s 2024-06-06T03:40:17.827 INFO:tasks.workunit.client.0.smithi040.stderr: 3450K .......... .......... .......... .......... .......... 2% 24.4M 12s 2024-06-06T03:40:17.829 INFO:tasks.workunit.client.0.smithi040.stderr: 3500K .......... .......... .......... .......... .......... 2% 24.8M 12s 2024-06-06T03:40:17.831 INFO:tasks.workunit.client.0.smithi040.stderr: 3550K .......... .......... .......... .......... .......... 2% 26.2M 12s 2024-06-06T03:40:17.831 INFO:tasks.workunit.client.0.smithi040.stderr: 3600K .......... .......... .......... .......... .......... 2% 112M 12s 2024-06-06T03:40:17.833 INFO:tasks.workunit.client.0.smithi040.stderr: 3650K .......... .......... .......... .......... .......... 2% 24.3M 12s 2024-06-06T03:40:17.835 INFO:tasks.workunit.client.0.smithi040.stderr: 3700K .......... .......... .......... .......... .......... 2% 25.9M 11s 2024-06-06T03:40:17.838 INFO:tasks.workunit.client.0.smithi040.stderr: 3750K .......... .......... .......... .......... .......... 2% 24.1M 11s 2024-06-06T03:40:17.858 INFO:tasks.workunit.client.0.smithi040.stderr: 3800K .......... .......... .......... .......... .......... 2% 25.6M 11s 2024-06-06T03:40:17.858 INFO:tasks.workunit.client.0.smithi040.stderr: 3850K .......... .......... .......... .......... .......... 2% 111M 11s 2024-06-06T03:40:17.858 INFO:tasks.workunit.client.0.smithi040.stderr: 3900K .......... .......... .......... .......... .......... 2% 25.3M 11s 2024-06-06T03:40:17.858 INFO:tasks.workunit.client.0.smithi040.stderr: 3950K .......... .......... .......... .......... .......... 2% 25.4M 11s 2024-06-06T03:40:17.858 INFO:tasks.workunit.client.0.smithi040.stderr: 4000K .......... .......... .......... .......... .......... 2% 22.4M 11s 2024-06-06T03:40:17.858 INFO:tasks.workunit.client.0.smithi040.stderr: 4050K .......... .......... .......... .......... .......... 3% 32.6M 11s 2024-06-06T03:40:17.858 INFO:tasks.workunit.client.0.smithi040.stderr: 4100K .......... .......... .......... .......... .......... 3% 83.2M 11s 2024-06-06T03:40:17.858 INFO:tasks.workunit.client.0.smithi040.stderr: 4150K .......... .......... .......... .......... .......... 3% 29.5M 11s 2024-06-06T03:40:17.859 INFO:tasks.workunit.client.0.smithi040.stderr: 4200K .......... .......... .......... .......... .......... 3% 17.7M 11s 2024-06-06T03:40:17.859 INFO:tasks.workunit.client.0.smithi040.stderr: 4250K .......... .......... .......... .......... .......... 3% 23.9M 11s 2024-06-06T03:40:17.859 INFO:tasks.workunit.client.0.smithi040.stderr: 4300K .......... .......... .......... .......... .......... 3% 65.5M 10s 2024-06-06T03:40:17.859 INFO:tasks.workunit.client.0.smithi040.stderr: 4350K .......... .......... .......... .......... .......... 3% 26.9M 10s 2024-06-06T03:40:17.863 INFO:tasks.workunit.client.0.smithi040.stderr: 4400K .......... .......... .......... .......... .......... 3% 15.3M 10s 2024-06-06T03:40:17.863 INFO:tasks.workunit.client.0.smithi040.stderr: 4450K .......... .......... .......... .......... .......... 3% 24.0M 10s 2024-06-06T03:40:17.863 INFO:tasks.workunit.client.0.smithi040.stderr: 4500K .......... .......... .......... .......... .......... 3% 39.6M 10s 2024-06-06T03:40:17.866 INFO:tasks.workunit.client.0.smithi040.stderr: 4550K .......... .......... .......... .......... .......... 3% 16.6M 10s 2024-06-06T03:40:17.868 INFO:tasks.workunit.client.0.smithi040.stderr: 4600K .......... .......... .......... .......... .......... 3% 19.6M 10s 2024-06-06T03:40:17.871 INFO:tasks.workunit.client.0.smithi040.stderr: 4650K .......... .......... .......... .......... .......... 3% 19.0M 10s 2024-06-06T03:40:17.875 INFO:tasks.workunit.client.0.smithi040.stderr: 4700K .......... .......... .......... .......... .......... 3% 20.9M 10s 2024-06-06T03:40:17.875 INFO:tasks.workunit.client.0.smithi040.stderr: 4750K .......... .......... .......... .......... .......... 3% 20.7M 10s 2024-06-06T03:40:17.880 INFO:tasks.workunit.client.0.smithi040.stderr: 4800K .......... .......... .......... .......... .......... 3% 14.2M 10s 2024-06-06T03:40:17.880 INFO:tasks.workunit.client.0.smithi040.stderr: 4850K .......... .......... .......... .......... .......... 3% 48.8M 10s 2024-06-06T03:40:17.884 INFO:tasks.workunit.client.0.smithi040.stderr: 4900K .......... .......... .......... .......... .......... 3% 17.8M 10s 2024-06-06T03:40:17.887 INFO:tasks.workunit.client.0.smithi040.stderr: 4950K .......... .......... .......... .......... .......... 3% 19.2M 10s 2024-06-06T03:40:17.887 INFO:tasks.workunit.client.0.smithi040.stderr: 5000K .......... .......... .......... .......... .......... 3% 23.1M 10s 2024-06-06T03:40:17.889 INFO:tasks.workunit.client.0.smithi040.stderr: 5050K .......... .......... .......... .......... .......... 3% 19.0M 10s 2024-06-06T03:40:17.891 INFO:tasks.workunit.client.0.smithi040.stderr: 5100K .......... .......... .......... .......... .......... 3% 37.5M 10s 2024-06-06T03:40:17.914 INFO:tasks.workunit.client.0.smithi040.stderr: 5150K .......... .......... .......... .......... .......... 3% 19.6M 10s 2024-06-06T03:40:17.914 INFO:tasks.workunit.client.0.smithi040.stderr: 5200K .......... .......... .......... .......... .......... 3% 35.4M 10s 2024-06-06T03:40:17.914 INFO:tasks.workunit.client.0.smithi040.stderr: 5250K .......... .......... .......... .......... .......... 3% 22.9M 10s 2024-06-06T03:40:17.914 INFO:tasks.workunit.client.0.smithi040.stderr: 5300K .......... .......... .......... .......... .......... 3% 25.1M 10s 2024-06-06T03:40:17.914 INFO:tasks.workunit.client.0.smithi040.stderr: 5350K .......... .......... .......... .......... .......... 3% 58.1M 9s 2024-06-06T03:40:17.914 INFO:tasks.workunit.client.0.smithi040.stderr: 5400K .......... .......... .......... .......... .......... 4% 25.7M 9s 2024-06-06T03:40:17.915 INFO:tasks.workunit.client.0.smithi040.stderr: 5450K .......... .......... .......... .......... .......... 4% 29.9M 9s 2024-06-06T03:40:17.915 INFO:tasks.workunit.client.0.smithi040.stderr: 5500K .......... .......... .......... .......... .......... 4% 26.4M 9s 2024-06-06T03:40:17.915 INFO:tasks.workunit.client.0.smithi040.stderr: 5550K .......... .......... .......... .......... .......... 4% 37.6M 9s 2024-06-06T03:40:17.915 INFO:tasks.workunit.client.0.smithi040.stderr: 5600K .......... .......... .......... .......... .......... 4% 24.8M 9s 2024-06-06T03:40:17.915 INFO:tasks.workunit.client.0.smithi040.stderr: 5650K .......... .......... .......... .......... .......... 4% 33.6M 9s 2024-06-06T03:40:17.915 INFO:tasks.workunit.client.0.smithi040.stderr: 5700K .......... .......... .......... .......... .......... 4% 21.7M 9s 2024-06-06T03:40:17.915 INFO:tasks.workunit.client.0.smithi040.stderr: 5750K .......... .......... .......... .......... .......... 4% 45.8M 9s 2024-06-06T03:40:17.917 INFO:tasks.workunit.client.0.smithi040.stderr: 5800K .......... .......... .......... .......... .......... 4% 26.0M 9s 2024-06-06T03:40:17.917 INFO:tasks.workunit.client.0.smithi040.stderr: 5850K .......... .......... .......... .......... .......... 4% 38.5M 9s 2024-06-06T03:40:17.925 INFO:tasks.workunit.client.0.smithi040.stderr: 5900K .......... .......... .......... .......... .......... 4% 25.6M 9s 2024-06-06T03:40:17.925 INFO:tasks.workunit.client.0.smithi040.stderr: 5950K .......... .......... .......... .......... .......... 4% 36.9M 9s 2024-06-06T03:40:17.925 INFO:tasks.workunit.client.0.smithi040.stderr: 6000K .......... .......... .......... .......... .......... 4% 22.7M 9s 2024-06-06T03:40:17.925 INFO:tasks.workunit.client.0.smithi040.stderr: 6050K .......... .......... .......... .......... .......... 4% 21.7M 9s 2024-06-06T03:40:17.925 INFO:tasks.workunit.client.0.smithi040.stderr: 6100K .......... .......... .......... .......... .......... 4% 44.1M 9s 2024-06-06T03:40:17.933 INFO:tasks.workunit.client.0.smithi040.stderr: 6150K .......... .......... .......... .......... .......... 4% 14.4M 9s 2024-06-06T03:40:17.933 INFO:tasks.workunit.client.0.smithi040.stderr: 6200K .......... .......... .......... .......... .......... 4% 26.2M 9s 2024-06-06T03:40:17.933 INFO:tasks.workunit.client.0.smithi040.stderr: 6250K .......... .......... .......... .......... .......... 4% 36.3M 9s 2024-06-06T03:40:17.937 INFO:tasks.workunit.client.0.smithi040.stderr: 6300K .......... .......... .......... .......... .......... 4% 18.1M 9s 2024-06-06T03:40:17.937 INFO:tasks.workunit.client.0.smithi040.stderr: 6350K .......... .......... .......... .......... .......... 4% 35.2M 9s 2024-06-06T03:40:17.939 INFO:tasks.workunit.client.0.smithi040.stderr: 6400K .......... .......... .......... .......... .......... 4% 19.1M 9s 2024-06-06T03:40:17.941 INFO:tasks.workunit.client.0.smithi040.stderr: 6450K .......... .......... .......... .......... .......... 4% 17.1M 9s 2024-06-06T03:40:17.944 INFO:tasks.workunit.client.0.smithi040.stderr: 6500K .......... .......... .......... .......... .......... 4% 20.3M 9s 2024-06-06T03:40:17.948 INFO:tasks.workunit.client.0.smithi040.stderr: 6550K .......... .......... .......... .......... .......... 4% 20.4M 9s 2024-06-06T03:40:17.948 INFO:tasks.workunit.client.0.smithi040.stderr: 6600K .......... .......... .......... .......... .......... 4% 24.2M 9s 2024-06-06T03:40:17.953 INFO:tasks.workunit.client.0.smithi040.stderr: 6650K .......... .......... .......... .......... .......... 4% 14.0M 9s 2024-06-06T03:40:17.953 INFO:tasks.workunit.client.0.smithi040.stderr: 6700K .......... .......... .......... .......... .......... 4% 34.7M 8s 2024-06-06T03:40:17.955 INFO:tasks.workunit.client.0.smithi040.stderr: 6750K .......... .......... .......... .......... .......... 5% 18.8M 8s 2024-06-06T03:40:17.958 INFO:tasks.workunit.client.0.smithi040.stderr: 6800K .......... .......... .......... .......... .......... 5% 18.5M 8s 2024-06-06T03:40:17.960 INFO:tasks.workunit.client.0.smithi040.stderr: 6850K .......... .......... .......... .......... .......... 5% 25.7M 8s 2024-06-06T03:40:17.967 INFO:tasks.workunit.client.0.smithi040.stderr: 6900K .......... .......... .......... .......... .......... 5% 19.2M 8s 2024-06-06T03:40:17.968 INFO:tasks.workunit.client.0.smithi040.stderr: 6950K .......... .......... .......... .......... .......... 5% 25.8M 8s 2024-06-06T03:40:17.968 INFO:tasks.workunit.client.0.smithi040.stderr: 7000K .......... .......... .......... .......... .......... 5% 26.0M 8s 2024-06-06T03:40:17.994 INFO:tasks.workunit.client.0.smithi040.stderr: 7050K .......... .......... .......... .......... .......... 5% 21.6M 8s 2024-06-06T03:40:17.994 INFO:tasks.workunit.client.0.smithi040.stderr: 7100K .......... .......... .......... .......... .......... 5% 47.0M 8s 2024-06-06T03:40:17.994 INFO:tasks.workunit.client.0.smithi040.stderr: 7150K .......... .......... .......... .......... .......... 5% 33.2M 8s 2024-06-06T03:40:17.994 INFO:tasks.workunit.client.0.smithi040.stderr: 7200K .......... .......... .......... .......... .......... 5% 25.5M 8s 2024-06-06T03:40:17.994 INFO:tasks.workunit.client.0.smithi040.stderr: 7250K .......... .......... .......... .......... .......... 5% 22.6M 8s 2024-06-06T03:40:17.994 INFO:tasks.workunit.client.0.smithi040.stderr: 7300K .......... .......... .......... .......... .......... 5% 33.4M 8s 2024-06-06T03:40:17.994 INFO:tasks.workunit.client.0.smithi040.stderr: 7350K .......... .......... .......... .......... .......... 5% 47.8M 8s 2024-06-06T03:40:17.994 INFO:tasks.workunit.client.0.smithi040.stderr: 7400K .......... .......... .......... .......... .......... 5% 23.7M 8s 2024-06-06T03:40:17.994 INFO:tasks.workunit.client.0.smithi040.stderr: 7450K .......... .......... .......... .......... .......... 5% 35.6M 8s 2024-06-06T03:40:17.994 INFO:tasks.workunit.client.0.smithi040.stderr: 7500K .......... .......... .......... .......... .......... 5% 23.6M 8s 2024-06-06T03:40:17.995 INFO:tasks.workunit.client.0.smithi040.stderr: 7550K .......... .......... .......... .......... .......... 5% 26.6M 8s 2024-06-06T03:40:17.995 INFO:tasks.workunit.client.0.smithi040.stderr: 7600K .......... .......... .......... .......... .......... 5% 40.3M 8s 2024-06-06T03:40:17.995 INFO:tasks.workunit.client.0.smithi040.stderr: 7650K .......... .......... .......... .......... .......... 5% 26.6M 8s 2024-06-06T03:40:17.995 INFO:tasks.workunit.client.0.smithi040.stderr: 7700K .......... .......... .......... .......... .......... 5% 21.9M 8s 2024-06-06T03:40:17.995 INFO:tasks.workunit.client.0.smithi040.stderr: 7750K .......... .......... .......... .......... .......... 5% 40.4M 8s 2024-06-06T03:40:17.995 INFO:tasks.workunit.client.0.smithi040.stderr: 7800K .......... .......... .......... .......... .......... 5% 23.4M 8s 2024-06-06T03:40:18.002 INFO:tasks.workunit.client.0.smithi040.stderr: 7850K .......... .......... .......... .......... .......... 5% 29.7M 8s 2024-06-06T03:40:18.002 INFO:tasks.workunit.client.0.smithi040.stderr: 7900K .......... .......... .......... .......... .......... 5% 26.0M 8s 2024-06-06T03:40:18.002 INFO:tasks.workunit.client.0.smithi040.stderr: 7950K .......... .......... .......... .......... .......... 5% 17.8M 8s 2024-06-06T03:40:18.002 INFO:tasks.workunit.client.0.smithi040.stderr: 8000K .......... .......... .......... .......... .......... 5% 30.8M 8s 2024-06-06T03:40:18.010 INFO:tasks.workunit.client.0.smithi040.stderr: 8050K .......... .......... .......... .......... .......... 5% 27.1M 8s 2024-06-06T03:40:18.010 INFO:tasks.workunit.client.0.smithi040.stderr: 8100K .......... .......... .......... .......... .......... 6% 31.7M 8s 2024-06-06T03:40:18.010 INFO:tasks.workunit.client.0.smithi040.stderr: 8150K .......... .......... .......... .......... .......... 6% 21.2M 8s 2024-06-06T03:40:18.010 INFO:tasks.workunit.client.0.smithi040.stderr: 8200K .......... .......... .......... .......... .......... 6% 29.6M 8s 2024-06-06T03:40:18.012 INFO:tasks.workunit.client.0.smithi040.stderr: 8250K .......... .......... .......... .......... .......... 6% 16.7M 8s 2024-06-06T03:40:18.014 INFO:tasks.workunit.client.0.smithi040.stderr: 8300K .......... .......... .......... .......... .......... 6% 25.6M 8s 2024-06-06T03:40:18.016 INFO:tasks.workunit.client.0.smithi040.stderr: 8350K .......... .......... .......... .......... .......... 6% 27.4M 8s 2024-06-06T03:40:18.017 INFO:tasks.workunit.client.0.smithi040.stderr: 8400K .......... .......... .......... .......... .......... 6% 21.7M 8s 2024-06-06T03:40:18.022 INFO:tasks.workunit.client.0.smithi040.stderr: 8450K .......... .......... .......... .......... .......... 6% 14.8M 8s 2024-06-06T03:40:18.024 INFO:tasks.workunit.client.0.smithi040.stderr: 8500K .......... .......... .......... .......... .......... 6% 44.6M 8s 2024-06-06T03:40:18.026 INFO:tasks.workunit.client.0.smithi040.stderr: 8550K .......... .......... .......... .......... .......... 6% 13.4M 8s 2024-06-06T03:40:18.028 INFO:tasks.workunit.client.0.smithi040.stderr: 8600K .......... .......... .......... .......... .......... 6% 26.4M 8s 2024-06-06T03:40:18.030 INFO:tasks.workunit.client.0.smithi040.stderr: 8650K .......... .......... .......... .......... .......... 6% 18.8M 8s 2024-06-06T03:40:18.033 INFO:tasks.workunit.client.0.smithi040.stderr: 8700K .......... .......... .......... .......... .......... 6% 22.4M 8s 2024-06-06T03:40:18.034 INFO:tasks.workunit.client.0.smithi040.stderr: 8750K .......... .......... .......... .......... .......... 6% 24.6M 8s 2024-06-06T03:40:18.076 INFO:tasks.workunit.client.0.smithi040.stderr: 8800K .......... .......... .......... .......... .......... 6% 13.6M 8s 2024-06-06T03:40:18.076 INFO:tasks.workunit.client.0.smithi040.stderr: 8850K .......... .......... .......... .......... .......... 6% 27.3M 8s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 8900K .......... .......... .......... .......... .......... 6% 36.9M 8s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 8950K .......... .......... .......... .......... .......... 6% 29.7M 8s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 9000K .......... .......... .......... .......... .......... 6% 28.4M 8s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 9050K .......... .......... .......... .......... .......... 6% 26.2M 7s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 9100K .......... .......... .......... .......... .......... 6% 23.8M 7s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 9150K .......... .......... .......... .......... .......... 6% 42.9M 7s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 9200K .......... .......... .......... .......... .......... 6% 27.3M 7s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 9250K .......... .......... .......... .......... .......... 6% 33.1M 7s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 9300K .......... .......... .......... .......... .......... 6% 23.3M 7s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 9350K .......... .......... .......... .......... .......... 6% 24.0M 7s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 9400K .......... .......... .......... .......... .......... 6% 42.0M 7s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 9450K .......... .......... .......... .......... .......... 6% 25.0M 7s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 9500K .......... .......... .......... .......... .......... 7% 42.2M 7s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 9550K .......... .......... .......... .......... .......... 7% 24.5M 7s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 9600K .......... .......... .......... .......... .......... 7% 34.3M 7s 2024-06-06T03:40:18.077 INFO:tasks.workunit.client.0.smithi040.stderr: 9650K .......... .......... .......... .......... .......... 7% 20.1M 7s 2024-06-06T03:40:18.078 INFO:tasks.workunit.client.0.smithi040.stderr: 9700K .......... .......... .......... .......... .......... 7% 29.7M 7s 2024-06-06T03:40:18.078 INFO:tasks.workunit.client.0.smithi040.stderr: 9750K .......... .......... .......... .......... .......... 7% 26.8M 7s 2024-06-06T03:40:18.078 INFO:tasks.workunit.client.0.smithi040.stderr: 9800K .......... .......... .......... .......... .......... 7% 33.0M 7s 2024-06-06T03:40:18.078 INFO:tasks.workunit.client.0.smithi040.stderr: 9850K .......... .......... .......... .......... .......... 7% 24.2M 7s 2024-06-06T03:40:18.078 INFO:tasks.workunit.client.0.smithi040.stderr: 9900K .......... .......... .......... .......... .......... 7% 29.5M 7s 2024-06-06T03:40:18.089 INFO:tasks.workunit.client.0.smithi040.stderr: 9950K .......... .......... .......... .......... .......... 7% 15.8M 7s 2024-06-06T03:40:18.089 INFO:tasks.workunit.client.0.smithi040.stderr: 10000K .......... .......... .......... .......... .......... 7% 56.5M 7s 2024-06-06T03:40:18.089 INFO:tasks.workunit.client.0.smithi040.stderr: 10050K .......... .......... .......... .......... .......... 7% 20.6M 7s 2024-06-06T03:40:18.089 INFO:tasks.workunit.client.0.smithi040.stderr: 10100K .......... .......... .......... .......... .......... 7% 33.3M 7s 2024-06-06T03:40:18.089 INFO:tasks.workunit.client.0.smithi040.stderr: 10150K .......... .......... .......... .......... .......... 7% 22.2M 7s 2024-06-06T03:40:18.090 INFO:tasks.workunit.client.0.smithi040.stderr: 10200K .......... .......... .......... .......... .......... 7% 20.2M 7s 2024-06-06T03:40:18.093 INFO:tasks.workunit.client.0.smithi040.stderr: 10250K .......... .......... .......... .......... .......... 7% 19.5M 7s 2024-06-06T03:40:18.093 INFO:tasks.workunit.client.0.smithi040.stderr: 10300K .......... .......... .......... .......... .......... 7% 41.1M 7s 2024-06-06T03:40:18.094 INFO:tasks.workunit.client.0.smithi040.stderr: 10350K .......... .......... .......... .......... .......... 7% 18.2M 7s 2024-06-06T03:40:18.097 INFO:tasks.workunit.client.0.smithi040.stderr: 10400K .......... .......... .......... .......... .......... 7% 32.8M 7s 2024-06-06T03:40:18.099 INFO:tasks.workunit.client.0.smithi040.stderr: 10450K .......... .......... .......... .......... .......... 7% 15.9M 7s 2024-06-06T03:40:18.101 INFO:tasks.workunit.client.0.smithi040.stderr: 10500K .......... .......... .......... .......... .......... 7% 40.8M 7s 2024-06-06T03:40:18.106 INFO:tasks.workunit.client.0.smithi040.stderr: 10550K .......... .......... .......... .......... .......... 7% 19.5M 7s 2024-06-06T03:40:18.106 INFO:tasks.workunit.client.0.smithi040.stderr: 10600K .......... .......... .......... .......... .......... 7% 17.2M 7s 2024-06-06T03:40:18.107 INFO:tasks.workunit.client.0.smithi040.stderr: 10650K .......... .......... .......... .......... .......... 7% 33.7M 7s 2024-06-06T03:40:18.110 INFO:tasks.workunit.client.0.smithi040.stderr: 10700K .......... .......... .......... .......... .......... 7% 14.1M 7s 2024-06-06T03:40:18.113 INFO:tasks.workunit.client.0.smithi040.stderr: 10750K .......... .......... .......... .......... .......... 7% 27.7M 7s 2024-06-06T03:40:18.115 INFO:tasks.workunit.client.0.smithi040.stderr: 10800K .......... .......... .......... .......... .......... 7% 15.7M 7s 2024-06-06T03:40:18.117 INFO:tasks.workunit.client.0.smithi040.stderr: 10850K .......... .......... .......... .......... .......... 8% 25.3M 7s 2024-06-06T03:40:18.118 INFO:tasks.workunit.client.0.smithi040.stderr: 10900K .......... .......... .......... .......... .......... 8% 50.5M 7s 2024-06-06T03:40:18.123 INFO:tasks.workunit.client.0.smithi040.stderr: 10950K .......... .......... .......... .......... .......... 8% 24.5M 7s 2024-06-06T03:40:18.123 INFO:tasks.workunit.client.0.smithi040.stderr: 11000K .......... .......... .......... .......... .......... 8% 25.4M 7s 2024-06-06T03:40:18.123 INFO:tasks.workunit.client.0.smithi040.stderr: 11050K .......... .......... .......... .......... .......... 8% 42.3M 7s 2024-06-06T03:40:18.125 INFO:tasks.workunit.client.0.smithi040.stderr: 11100K .......... .......... .......... .......... .......... 8% 22.7M 7s 2024-06-06T03:40:18.127 INFO:tasks.workunit.client.0.smithi040.stderr: 11150K .......... .......... .......... .......... .......... 8% 41.4M 7s 2024-06-06T03:40:18.129 INFO:tasks.workunit.client.0.smithi040.stderr: 11200K .......... .......... .......... .......... .......... 8% 26.2M 7s 2024-06-06T03:40:18.133 INFO:tasks.workunit.client.0.smithi040.stderr: 11250K .......... .......... .......... .......... .......... 8% 27.6M 7s 2024-06-06T03:40:18.133 INFO:tasks.workunit.client.0.smithi040.stderr: 11300K .......... .......... .......... .......... .......... 8% 26.1M 7s 2024-06-06T03:40:18.147 INFO:tasks.workunit.client.0.smithi040.stderr: 11350K .......... .......... .......... .......... .......... 8% 9.95M 7s 2024-06-06T03:40:18.147 INFO:tasks.workunit.client.0.smithi040.stderr: 11400K .......... .......... .......... .......... .......... 8% 94.3M 7s 2024-06-06T03:40:18.147 INFO:tasks.workunit.client.0.smithi040.stderr: 11450K .......... .......... .......... .......... .......... 8% 88.3M 7s 2024-06-06T03:40:18.148 INFO:tasks.workunit.client.0.smithi040.stderr: 11500K .......... .......... .......... .......... .......... 8% 35.3M 7s 2024-06-06T03:40:18.148 INFO:tasks.workunit.client.0.smithi040.stderr: 11550K .......... .......... .......... .......... .......... 8% 30.8M 7s 2024-06-06T03:40:18.148 INFO:tasks.workunit.client.0.smithi040.stderr: 11600K .......... .......... .......... .......... .......... 8% 44.2M 7s 2024-06-06T03:40:18.148 INFO:tasks.workunit.client.0.smithi040.stderr: 11650K .......... .......... .......... .......... .......... 8% 32.8M 7s 2024-06-06T03:40:18.148 INFO:tasks.workunit.client.0.smithi040.stderr: 11700K .......... .......... .......... .......... .......... 8% 29.1M 7s 2024-06-06T03:40:18.148 INFO:tasks.workunit.client.0.smithi040.stderr: 11750K .......... .......... .......... .......... .......... 8% 20.2M 7s 2024-06-06T03:40:18.155 INFO:tasks.workunit.client.0.smithi040.stderr: 11800K .......... .......... .......... .......... .......... 8% 55.3M 7s 2024-06-06T03:40:18.155 INFO:tasks.workunit.client.0.smithi040.stderr: 11850K .......... .......... .......... .......... .......... 8% 20.2M 7s 2024-06-06T03:40:18.155 INFO:tasks.workunit.client.0.smithi040.stderr: 11900K .......... .......... .......... .......... .......... 8% 44.3M 7s 2024-06-06T03:40:18.155 INFO:tasks.workunit.client.0.smithi040.stderr: 11950K .......... .......... .......... .......... .......... 8% 27.8M 7s 2024-06-06T03:40:18.157 INFO:tasks.workunit.client.0.smithi040.stderr: 12000K .......... .......... .......... .......... .......... 8% 21.1M 7s 2024-06-06T03:40:18.157 INFO:tasks.workunit.client.0.smithi040.stderr: 12050K .......... .......... .......... .......... .......... 8% 49.2M 7s 2024-06-06T03:40:18.159 INFO:tasks.workunit.client.0.smithi040.stderr: 12100K .......... .......... .......... .......... .......... 8% 25.0M 7s 2024-06-06T03:40:18.160 INFO:tasks.workunit.client.0.smithi040.stderr: 12150K .......... .......... .......... .......... .......... 8% 35.6M 7s 2024-06-06T03:40:18.163 INFO:tasks.workunit.client.0.smithi040.stderr: 12200K .......... .......... .......... .......... .......... 9% 24.1M 7s 2024-06-06T03:40:18.164 INFO:tasks.workunit.client.0.smithi040.stderr: 12250K .......... .......... .......... .......... .......... 9% 29.8M 7s 2024-06-06T03:40:18.179 INFO:tasks.workunit.client.0.smithi040.stderr: 12300K .......... .......... .......... .......... .......... 9% 36.0M 7s 2024-06-06T03:40:18.179 INFO:tasks.workunit.client.0.smithi040.stderr: 12350K .......... .......... .......... .......... .......... 9% 22.9M 7s 2024-06-06T03:40:18.179 INFO:tasks.workunit.client.0.smithi040.stderr: 12400K .......... .......... .......... .......... .......... 9% 43.0M 7s 2024-06-06T03:40:18.179 INFO:tasks.workunit.client.0.smithi040.stderr: 12450K .......... .......... .......... .......... .......... 9% 25.1M 7s 2024-06-06T03:40:18.179 INFO:tasks.workunit.client.0.smithi040.stderr: 12500K .......... .......... .......... .......... .......... 9% 25.9M 7s 2024-06-06T03:40:18.179 INFO:tasks.workunit.client.0.smithi040.stderr: 12550K .......... .......... .......... .......... .......... 9% 35.1M 7s 2024-06-06T03:40:18.179 INFO:tasks.workunit.client.0.smithi040.stderr: 12600K .......... .......... .......... .......... .......... 9% 24.1M 6s 2024-06-06T03:40:18.179 INFO:tasks.workunit.client.0.smithi040.stderr: 12650K .......... .......... .......... .......... .......... 9% 44.0M 6s 2024-06-06T03:40:18.179 INFO:tasks.workunit.client.0.smithi040.stderr: 12700K .......... .......... .......... .......... .......... 9% 24.6M 6s 2024-06-06T03:40:18.192 INFO:tasks.workunit.client.0.smithi040.stderr: 12750K .......... .......... .......... .......... .......... 9% 25.3M 6s 2024-06-06T03:40:18.192 INFO:tasks.workunit.client.0.smithi040.stderr: 12800K .......... .......... .......... .......... .......... 9% 29.6M 6s 2024-06-06T03:40:18.192 INFO:tasks.workunit.client.0.smithi040.stderr: 12850K .......... .......... .......... .......... .......... 9% 38.9M 6s 2024-06-06T03:40:18.193 INFO:tasks.workunit.client.0.smithi040.stderr: 12900K .......... .......... .......... .......... .......... 9% 34.1M 6s 2024-06-06T03:40:18.193 INFO:tasks.workunit.client.0.smithi040.stderr: 12950K .......... .......... .......... .......... .......... 9% 21.4M 6s 2024-06-06T03:40:18.193 INFO:tasks.workunit.client.0.smithi040.stderr: 13000K .......... .......... .......... .......... .......... 9% 19.6M 6s 2024-06-06T03:40:18.193 INFO:tasks.workunit.client.0.smithi040.stderr: 13050K .......... .......... .......... .......... .......... 9% 39.9M 6s 2024-06-06T03:40:18.194 INFO:tasks.workunit.client.0.smithi040.stderr: 13100K .......... .......... .......... .......... .......... 9% 21.8M 6s 2024-06-06T03:40:18.196 INFO:tasks.workunit.client.0.smithi040.stderr: 13150K .......... .......... .......... .......... .......... 9% 35.0M 6s 2024-06-06T03:40:18.197 INFO:tasks.workunit.client.0.smithi040.stderr: 13200K .......... .......... .......... .......... .......... 9% 24.6M 6s 2024-06-06T03:40:18.200 INFO:tasks.workunit.client.0.smithi040.stderr: 13250K .......... .......... .......... .......... .......... 9% 22.6M 6s 2024-06-06T03:40:18.202 INFO:tasks.workunit.client.0.smithi040.stderr: 13300K .......... .......... .......... .......... .......... 9% 26.5M 6s 2024-06-06T03:40:18.202 INFO:tasks.workunit.client.0.smithi040.stderr: 13350K .......... .......... .......... .......... .......... 9% 49.8M 6s 2024-06-06T03:40:18.204 INFO:tasks.workunit.client.0.smithi040.stderr: 13400K .......... .......... .......... .......... .......... 9% 36.3M 6s 2024-06-06T03:40:18.205 INFO:tasks.workunit.client.0.smithi040.stderr: 13450K .......... .......... .......... .......... .......... 9% 22.9M 6s 2024-06-06T03:40:18.208 INFO:tasks.workunit.client.0.smithi040.stderr: 13500K .......... .......... .......... .......... .......... 9% 27.7M 6s 2024-06-06T03:40:18.244 INFO:tasks.workunit.client.0.smithi040.stderr: 13550K .......... .......... .......... .......... .......... 10% 27.7M 6s 2024-06-06T03:40:18.244 INFO:tasks.workunit.client.0.smithi040.stderr: 13600K .......... .......... .......... .......... .......... 10% 34.8M 6s 2024-06-06T03:40:18.244 INFO:tasks.workunit.client.0.smithi040.stderr: 13650K .......... .......... .......... .......... .......... 10% 36.3M 6s 2024-06-06T03:40:18.244 INFO:tasks.workunit.client.0.smithi040.stderr: 13700K .......... .......... .......... .......... .......... 10% 24.6M 6s 2024-06-06T03:40:18.244 INFO:tasks.workunit.client.0.smithi040.stderr: 13750K .......... .......... .......... .......... .......... 10% 24.7M 6s 2024-06-06T03:40:18.244 INFO:tasks.workunit.client.0.smithi040.stderr: 13800K .......... .......... .......... .......... .......... 10% 35.7M 6s 2024-06-06T03:40:18.244 INFO:tasks.workunit.client.0.smithi040.stderr: 13850K .......... .......... .......... .......... .......... 10% 25.6M 6s 2024-06-06T03:40:18.244 INFO:tasks.workunit.client.0.smithi040.stderr: 13900K .......... .......... .......... .......... .......... 10% 36.1M 6s 2024-06-06T03:40:18.244 INFO:tasks.workunit.client.0.smithi040.stderr: 13950K .......... .......... .......... .......... .......... 10% 32.0M 6s 2024-06-06T03:40:18.244 INFO:tasks.workunit.client.0.smithi040.stderr: 14000K .......... .......... .......... .......... .......... 10% 22.8M 6s 2024-06-06T03:40:18.245 INFO:tasks.workunit.client.0.smithi040.stderr: 14050K .......... .......... .......... .......... .......... 10% 36.3M 6s 2024-06-06T03:40:18.245 INFO:tasks.workunit.client.0.smithi040.stderr: 14100K .......... .......... .......... .......... .......... 10% 20.5M 6s 2024-06-06T03:40:18.245 INFO:tasks.workunit.client.0.smithi040.stderr: 14150K .......... .......... .......... .......... .......... 10% 27.2M 6s 2024-06-06T03:40:18.245 INFO:tasks.workunit.client.0.smithi040.stderr: 14200K .......... .......... .......... .......... .......... 10% 46.1M 6s 2024-06-06T03:40:18.245 INFO:tasks.workunit.client.0.smithi040.stderr: 14250K .......... .......... .......... .......... .......... 10% 28.5M 6s 2024-06-06T03:40:18.245 INFO:tasks.workunit.client.0.smithi040.stderr: 14300K .......... .......... .......... .......... .......... 10% 35.6M 6s 2024-06-06T03:40:18.245 INFO:tasks.workunit.client.0.smithi040.stderr: 14350K .......... .......... .......... .......... .......... 10% 20.9M 6s 2024-06-06T03:40:18.245 INFO:tasks.workunit.client.0.smithi040.stderr: 14400K .......... .......... .......... .......... .......... 10% 30.7M 6s 2024-06-06T03:40:18.245 INFO:tasks.workunit.client.0.smithi040.stderr: 14450K .......... .......... .......... .......... .......... 10% 38.1M 6s 2024-06-06T03:40:18.245 INFO:tasks.workunit.client.0.smithi040.stderr: 14500K .......... .......... .......... .......... .......... 10% 22.3M 6s 2024-06-06T03:40:18.245 INFO:tasks.workunit.client.0.smithi040.stderr: 14550K .......... .......... .......... .......... .......... 10% 36.0M 6s 2024-06-06T03:40:18.245 INFO:tasks.workunit.client.0.smithi040.stderr: 14600K .......... .......... .......... .......... .......... 10% 31.4M 6s 2024-06-06T03:40:18.267 INFO:tasks.workunit.client.0.smithi040.stderr: 14650K .......... .......... .......... .......... .......... 10% 25.4M 6s 2024-06-06T03:40:18.267 INFO:tasks.workunit.client.0.smithi040.stderr: 14700K .......... .......... .......... .......... .......... 10% 35.1M 6s 2024-06-06T03:40:18.267 INFO:tasks.workunit.client.0.smithi040.stderr: 14750K .......... .......... .......... .......... .......... 10% 30.6M 6s 2024-06-06T03:40:18.267 INFO:tasks.workunit.client.0.smithi040.stderr: 14800K .......... .......... .......... .......... .......... 10% 25.0M 6s 2024-06-06T03:40:18.267 INFO:tasks.workunit.client.0.smithi040.stderr: 14850K .......... .......... .......... .......... .......... 10% 26.3M 6s 2024-06-06T03:40:18.267 INFO:tasks.workunit.client.0.smithi040.stderr: 14900K .......... .......... .......... .......... .......... 11% 26.3M 6s 2024-06-06T03:40:18.267 INFO:tasks.workunit.client.0.smithi040.stderr: 14950K .......... .......... .......... .......... .......... 11% 39.7M 6s 2024-06-06T03:40:18.267 INFO:tasks.workunit.client.0.smithi040.stderr: 15000K .......... .......... .......... .......... .......... 11% 24.7M 6s 2024-06-06T03:40:18.267 INFO:tasks.workunit.client.0.smithi040.stderr: 15050K .......... .......... .......... .......... .......... 11% 22.9M 6s 2024-06-06T03:40:18.267 INFO:tasks.workunit.client.0.smithi040.stderr: 15100K .......... .......... .......... .......... .......... 11% 47.0M 6s 2024-06-06T03:40:18.267 INFO:tasks.workunit.client.0.smithi040.stderr: 15150K .......... .......... .......... .......... .......... 11% 23.9M 6s 2024-06-06T03:40:18.268 INFO:tasks.workunit.client.0.smithi040.stderr: 15200K .......... .......... .......... .......... .......... 11% 40.9M 6s 2024-06-06T03:40:18.277 INFO:tasks.workunit.client.0.smithi040.stderr: 15250K .......... .......... .......... .......... .......... 11% 3.81M 6s 2024-06-06T03:40:18.284 INFO:tasks.workunit.client.0.smithi040.stderr: 15300K .......... .......... .......... .......... .......... 11% 38.1M 6s 2024-06-06T03:40:18.284 INFO:tasks.workunit.client.0.smithi040.stderr: 15350K .......... .......... .......... .......... .......... 11% 25.6M 6s 2024-06-06T03:40:18.284 INFO:tasks.workunit.client.0.smithi040.stderr: 15400K .......... .......... .......... .......... .......... 11% 22.7M 6s 2024-06-06T03:40:18.284 INFO:tasks.workunit.client.0.smithi040.stderr: 15450K .......... .......... .......... .......... .......... 11% 54.0M 6s 2024-06-06T03:40:18.293 INFO:tasks.workunit.client.0.smithi040.stderr: 15500K .......... .......... .......... .......... .......... 11% 26.1M 6s 2024-06-06T03:40:18.293 INFO:tasks.workunit.client.0.smithi040.stderr: 15550K .......... .......... .......... .......... .......... 11% 26.4M 6s 2024-06-06T03:40:18.293 INFO:tasks.workunit.client.0.smithi040.stderr: 15600K .......... .......... .......... .......... .......... 11% 27.3M 6s 2024-06-06T03:40:18.293 INFO:tasks.workunit.client.0.smithi040.stderr: 15650K .......... .......... .......... .......... .......... 11% 22.5M 6s 2024-06-06T03:40:18.293 INFO:tasks.workunit.client.0.smithi040.stderr: 15700K .......... .......... .......... .......... .......... 11% 46.4M 6s 2024-06-06T03:40:18.296 INFO:tasks.workunit.client.0.smithi040.stderr: 15750K .......... .......... .......... .......... .......... 11% 23.4M 6s 2024-06-06T03:40:18.296 INFO:tasks.workunit.client.0.smithi040.stderr: 15800K .......... .......... .......... .......... .......... 11% 34.4M 6s 2024-06-06T03:40:18.297 INFO:tasks.workunit.client.0.smithi040.stderr: 15850K .......... .......... .......... .......... .......... 11% 22.2M 6s 2024-06-06T03:40:18.300 INFO:tasks.workunit.client.0.smithi040.stderr: 15900K .......... .......... .......... .......... .......... 11% 28.8M 6s 2024-06-06T03:40:18.301 INFO:tasks.workunit.client.0.smithi040.stderr: 15950K .......... .......... .......... .......... .......... 11% 42.8M 6s 2024-06-06T03:40:18.302 INFO:tasks.workunit.client.0.smithi040.stderr: 16000K .......... .......... .......... .......... .......... 11% 23.4M 6s 2024-06-06T03:40:18.304 INFO:tasks.workunit.client.0.smithi040.stderr: 16050K .......... .......... .......... .......... .......... 11% 24.1M 6s 2024-06-06T03:40:18.306 INFO:tasks.workunit.client.0.smithi040.stderr: 16100K .......... .......... .......... .......... .......... 11% 39.9M 6s 2024-06-06T03:40:18.309 INFO:tasks.workunit.client.0.smithi040.stderr: 16150K .......... .......... .......... .......... .......... 11% 29.3M 6s 2024-06-06T03:40:18.309 INFO:tasks.workunit.client.0.smithi040.stderr: 16200K .......... .......... .......... .......... .......... 11% 33.3M 6s 2024-06-06T03:40:18.421 INFO:tasks.workunit.client.0.smithi040.stderr: 16250K .......... .......... .......... .......... .......... 12% 486K 7s 2024-06-06T03:40:18.421 INFO:tasks.workunit.client.0.smithi040.stderr: 16300K .......... .......... .......... .......... .......... 12% 539M 7s 2024-06-06T03:40:18.422 INFO:tasks.workunit.client.0.smithi040.stderr: 16350K .......... .......... .......... .......... .......... 12% 538M 7s 2024-06-06T03:40:18.422 INFO:tasks.workunit.client.0.smithi040.stderr: 16400K .......... .......... .......... .......... .......... 12% 399M 7s 2024-06-06T03:40:18.422 INFO:tasks.workunit.client.0.smithi040.stderr: 16450K .......... .......... .......... .......... .......... 12% 571M 7s 2024-06-06T03:40:18.422 INFO:tasks.workunit.client.0.smithi040.stderr: 16500K .......... .......... .......... .......... .......... 12% 581M 7s 2024-06-06T03:40:18.422 INFO:tasks.workunit.client.0.smithi040.stderr: 16550K .......... .......... .......... .......... .......... 12% 577M 6s 2024-06-06T03:40:18.422 INFO:tasks.workunit.client.0.smithi040.stderr: 16600K .......... .......... .......... .......... .......... 12% 478M 6s 2024-06-06T03:40:18.422 INFO:tasks.workunit.client.0.smithi040.stderr: 16650K .......... .......... .......... .......... .......... 12% 579M 6s 2024-06-06T03:40:18.422 INFO:tasks.workunit.client.0.smithi040.stderr: 16700K .......... .......... .......... .......... .......... 12% 587M 6s 2024-06-06T03:40:18.422 INFO:tasks.workunit.client.0.smithi040.stderr: 16750K .......... .......... .......... .......... .......... 12% 584M 6s 2024-06-06T03:40:18.422 INFO:tasks.workunit.client.0.smithi040.stderr: 16800K .......... .......... .......... .......... .......... 12% 481M 6s 2024-06-06T03:40:18.422 INFO:tasks.workunit.client.0.smithi040.stderr: 16850K .......... .......... .......... .......... .......... 12% 547M 6s 2024-06-06T03:40:18.422 INFO:tasks.workunit.client.0.smithi040.stderr: 16900K .......... .......... .......... .......... .......... 12% 432M 6s 2024-06-06T03:40:18.422 INFO:tasks.workunit.client.0.smithi040.stderr: 16950K .......... .......... .......... .......... .......... 12% 573M 6s 2024-06-06T03:40:18.433 INFO:tasks.workunit.client.0.smithi040.stderr: 17000K .......... .......... .......... .......... .......... 12% 2.94M 6s 2024-06-06T03:40:18.434 INFO:tasks.workunit.client.0.smithi040.stderr: 17050K .......... .......... .......... .......... .......... 12% 513M 6s 2024-06-06T03:40:18.434 INFO:tasks.workunit.client.0.smithi040.stderr: 17100K .......... .......... .......... .......... .......... 12% 575M 6s 2024-06-06T03:40:18.434 INFO:tasks.workunit.client.0.smithi040.stderr: 17150K .......... .......... .......... .......... .......... 12% 554M 6s 2024-06-06T03:40:18.434 INFO:tasks.workunit.client.0.smithi040.stderr: 17200K .......... .......... .......... .......... .......... 12% 404M 6s 2024-06-06T03:40:18.434 INFO:tasks.workunit.client.0.smithi040.stderr: 17250K .......... .......... .......... .......... .......... 12% 537M 6s 2024-06-06T03:40:18.469 INFO:tasks.workunit.client.0.smithi040.stderr: 17300K .......... .......... .......... .......... .......... 12% 8.33M 6s 2024-06-06T03:40:18.469 INFO:tasks.workunit.client.0.smithi040.stderr: 17350K .......... .......... .......... .......... .......... 12% 144M 6s 2024-06-06T03:40:18.470 INFO:tasks.workunit.client.0.smithi040.stderr: 17400K .......... .......... .......... .......... .......... 12% 461M 6s 2024-06-06T03:40:18.470 INFO:tasks.workunit.client.0.smithi040.stderr: 17450K .......... .......... .......... .......... .......... 12% 565M 6s 2024-06-06T03:40:18.470 INFO:tasks.workunit.client.0.smithi040.stderr: 17500K .......... .......... .......... .......... .......... 12% 571M 6s 2024-06-06T03:40:18.470 INFO:tasks.workunit.client.0.smithi040.stderr: 17550K .......... .......... .......... .......... .......... 12% 511M 6s 2024-06-06T03:40:18.470 INFO:tasks.workunit.client.0.smithi040.stderr: 17600K .......... .......... .......... .......... .......... 13% 425M 6s 2024-06-06T03:40:18.470 INFO:tasks.workunit.client.0.smithi040.stderr: 17650K .......... .......... .......... .......... .......... 13% 509M 6s 2024-06-06T03:40:18.470 INFO:tasks.workunit.client.0.smithi040.stderr: 17700K .......... .......... .......... .......... .......... 13% 566M 6s 2024-06-06T03:40:18.470 INFO:tasks.workunit.client.0.smithi040.stderr: 17750K .......... .......... .......... .......... .......... 13% 572M 6s 2024-06-06T03:40:18.470 INFO:tasks.workunit.client.0.smithi040.stderr: 17800K .......... .......... .......... .......... .......... 13% 496M 6s 2024-06-06T03:40:18.470 INFO:tasks.workunit.client.0.smithi040.stderr: 17850K .......... .......... .......... .......... .......... 13% 580M 6s 2024-06-06T03:40:18.470 INFO:tasks.workunit.client.0.smithi040.stderr: 17900K .......... .......... .......... .......... .......... 13% 519M 6s 2024-06-06T03:40:18.470 INFO:tasks.workunit.client.0.smithi040.stderr: 17950K .......... .......... .......... .......... .......... 13% 501M 6s 2024-06-06T03:40:18.470 INFO:tasks.workunit.client.0.smithi040.stderr: 18000K .......... .......... .......... .......... .......... 13% 491M 6s 2024-06-06T03:40:18.471 INFO:tasks.workunit.client.0.smithi040.stderr: 18050K .......... .......... .......... .......... .......... 13% 578M 6s 2024-06-06T03:40:18.471 INFO:tasks.workunit.client.0.smithi040.stderr: 18100K .......... .......... .......... .......... .......... 13% 513M 6s 2024-06-06T03:40:18.471 INFO:tasks.workunit.client.0.smithi040.stderr: 18150K .......... .......... .......... .......... .......... 13% 544M 6s 2024-06-06T03:40:18.471 INFO:tasks.workunit.client.0.smithi040.stderr: 18200K .......... .......... .......... .......... .......... 13% 491M 6s 2024-06-06T03:40:18.471 INFO:tasks.workunit.client.0.smithi040.stderr: 18250K .......... .......... .......... .......... .......... 13% 540M 6s 2024-06-06T03:40:18.471 INFO:tasks.workunit.client.0.smithi040.stderr: 18300K .......... .......... .......... .......... .......... 13% 569M 6s 2024-06-06T03:40:18.471 INFO:tasks.workunit.client.0.smithi040.stderr: 18350K .......... .......... .......... .......... .......... 13% 575M 6s 2024-06-06T03:40:18.471 INFO:tasks.workunit.client.0.smithi040.stderr: 18400K .......... .......... .......... .......... .......... 13% 478M 6s 2024-06-06T03:40:18.471 INFO:tasks.workunit.client.0.smithi040.stderr: 18450K .......... .......... .......... .......... .......... 13% 567M 6s 2024-06-06T03:40:18.471 INFO:tasks.workunit.client.0.smithi040.stderr: 18500K .......... .......... .......... .......... .......... 13% 577M 6s 2024-06-06T03:40:18.471 INFO:tasks.workunit.client.0.smithi040.stderr: 18550K .......... .......... .......... .......... .......... 13% 578M 6s 2024-06-06T03:40:18.471 INFO:tasks.workunit.client.0.smithi040.stderr: 18600K .......... .......... .......... .......... .......... 13% 498M 6s 2024-06-06T03:40:18.471 INFO:tasks.workunit.client.0.smithi040.stderr: 18650K .......... .......... .......... .......... .......... 13% 525M 6s 2024-06-06T03:40:18.471 INFO:tasks.workunit.client.0.smithi040.stderr: 18700K .......... .......... .......... .......... .......... 13% 559M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 18750K .......... .......... .......... .......... .......... 13% 571M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 18800K .......... .......... .......... .......... .......... 13% 493M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 18850K .......... .......... .......... .......... .......... 13% 3.31M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 18900K .......... .......... .......... .......... .......... 13% 517M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 18950K .......... .......... .......... .......... .......... 13% 565M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 19000K .......... .......... .......... .......... .......... 14% 502M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 19050K .......... .......... .......... .......... .......... 14% 567M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 19100K .......... .......... .......... .......... .......... 14% 594M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 19150K .......... .......... .......... .......... .......... 14% 566M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 19200K .......... .......... .......... .......... .......... 14% 506M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 19250K .......... .......... .......... .......... .......... 14% 594M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 19300K .......... .......... .......... .......... .......... 14% 589M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 19350K .......... .......... .......... .......... .......... 14% 583M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 19400K .......... .......... .......... .......... .......... 14% 464M 6s 2024-06-06T03:40:18.472 INFO:tasks.workunit.client.0.smithi040.stderr: 19450K .......... .......... .......... .......... .......... 14% 586M 6s 2024-06-06T03:40:18.473 INFO:tasks.workunit.client.0.smithi040.stderr: 19500K .......... .......... .......... .......... .......... 14% 595M 6s 2024-06-06T03:40:18.473 INFO:tasks.workunit.client.0.smithi040.stderr: 19550K .......... .......... .......... .......... .......... 14% 593M 6s 2024-06-06T03:40:18.473 INFO:tasks.workunit.client.0.smithi040.stderr: 19600K .......... .......... .......... .......... .......... 14% 503M 6s 2024-06-06T03:40:18.473 INFO:tasks.workunit.client.0.smithi040.stderr: 19650K .......... .......... .......... .......... .......... 14% 565M 6s 2024-06-06T03:40:18.473 INFO:tasks.workunit.client.0.smithi040.stderr: 19700K .......... .......... .......... .......... .......... 14% 591M 6s 2024-06-06T03:40:18.473 INFO:tasks.workunit.client.0.smithi040.stderr: 19750K .......... .......... .......... .......... .......... 14% 596M 6s 2024-06-06T03:40:18.473 INFO:tasks.workunit.client.0.smithi040.stderr: 19800K .......... .......... .......... .......... .......... 14% 524M 6s 2024-06-06T03:40:18.473 INFO:tasks.workunit.client.0.smithi040.stderr: 19850K .......... .......... .......... .......... .......... 14% 603M 6s 2024-06-06T03:40:18.473 INFO:tasks.workunit.client.0.smithi040.stderr: 19900K .......... .......... .......... .......... .......... 14% 3.78M 6s 2024-06-06T03:40:18.522 INFO:tasks.workunit.client.0.smithi040.stderr: 19950K .......... .......... .......... .......... .......... 14% 13.7M 6s 2024-06-06T03:40:18.522 INFO:tasks.workunit.client.0.smithi040.stderr: 20000K .......... .......... .......... .......... .......... 14% 106M 6s 2024-06-06T03:40:18.522 INFO:tasks.workunit.client.0.smithi040.stderr: 20050K .......... .......... .......... .......... .......... 14% 110M 6s 2024-06-06T03:40:18.522 INFO:tasks.workunit.client.0.smithi040.stderr: 20100K .......... .......... .......... .......... .......... 14% 130M 6s 2024-06-06T03:40:18.522 INFO:tasks.workunit.client.0.smithi040.stderr: 20150K .......... .......... .......... .......... .......... 14% 84.0M 6s 2024-06-06T03:40:18.522 INFO:tasks.workunit.client.0.smithi040.stderr: 20200K .......... .......... .......... .......... .......... 14% 115M 6s 2024-06-06T03:40:18.525 INFO:tasks.workunit.client.0.smithi040.stderr: 20250K .......... .......... .......... .......... .......... 14% 131M 5s 2024-06-06T03:40:18.525 INFO:tasks.workunit.client.0.smithi040.stderr: 20300K .......... .......... .......... .......... .......... 14% 62.4M 5s 2024-06-06T03:40:18.526 INFO:tasks.workunit.client.0.smithi040.stderr: 20350K .......... .......... .......... .......... .......... 15% 34.2M 5s 2024-06-06T03:40:18.526 INFO:tasks.workunit.client.0.smithi040.stderr: 20400K .......... .......... .......... .......... .......... 15% 22.1M 5s 2024-06-06T03:40:18.526 INFO:tasks.workunit.client.0.smithi040.stderr: 20450K .......... .......... .......... .......... .......... 15% 4.79M 6s 2024-06-06T03:40:18.526 INFO:tasks.workunit.client.0.smithi040.stderr: 20500K .......... .......... .......... .......... .......... 15% 130M 5s 2024-06-06T03:40:18.526 INFO:tasks.workunit.client.0.smithi040.stderr: 20550K .......... .......... .......... .......... .......... 15% 56.5M 5s 2024-06-06T03:40:18.526 INFO:tasks.workunit.client.0.smithi040.stderr: 20600K .......... .......... .......... .......... .......... 15% 145M 5s 2024-06-06T03:40:18.526 INFO:tasks.workunit.client.0.smithi040.stderr: 20650K .......... .......... .......... .......... .......... 15% 71.2M 5s 2024-06-06T03:40:18.526 INFO:tasks.workunit.client.0.smithi040.stderr: 20700K .......... .......... .......... .......... .......... 15% 62.3M 5s 2024-06-06T03:40:18.526 INFO:tasks.workunit.client.0.smithi040.stderr: 20750K .......... .......... .......... .......... .......... 15% 130M 5s 2024-06-06T03:40:18.526 INFO:tasks.workunit.client.0.smithi040.stderr: 20800K .......... .......... .......... .......... .......... 15% 56.1M 5s 2024-06-06T03:40:18.526 INFO:tasks.workunit.client.0.smithi040.stderr: 20850K .......... .......... .......... .......... .......... 15% 67.2M 5s 2024-06-06T03:40:18.526 INFO:tasks.workunit.client.0.smithi040.stderr: 20900K .......... .......... .......... .......... .......... 15% 26.8M 5s 2024-06-06T03:40:18.527 INFO:tasks.workunit.client.0.smithi040.stderr: 20950K .......... .......... .......... .......... .......... 15% 35.3M 5s 2024-06-06T03:40:18.527 INFO:tasks.workunit.client.0.smithi040.stderr: 21000K .......... .......... .......... .......... .......... 15% 24.0M 5s 2024-06-06T03:40:18.527 INFO:tasks.workunit.client.0.smithi040.stderr: 21050K .......... .......... .......... .......... .......... 15% 26.4M 5s 2024-06-06T03:40:18.527 INFO:tasks.workunit.client.0.smithi040.stderr: 21100K .......... .......... .......... .......... .......... 15% 44.0M 5s 2024-06-06T03:40:18.527 INFO:tasks.workunit.client.0.smithi040.stderr: 21150K .......... .......... .......... .......... .......... 15% 26.4M 5s 2024-06-06T03:40:18.527 INFO:tasks.workunit.client.0.smithi040.stderr: 21200K .......... .......... .......... .......... .......... 15% 21.2M 5s 2024-06-06T03:40:18.527 INFO:tasks.workunit.client.0.smithi040.stderr: 21250K .......... .......... .......... .......... .......... 15% 39.1M 5s 2024-06-06T03:40:18.527 INFO:tasks.workunit.client.0.smithi040.stderr: 21300K .......... .......... .......... .......... .......... 15% 26.5M 5s 2024-06-06T03:40:18.527 INFO:tasks.workunit.client.0.smithi040.stderr: 21350K .......... .......... .......... .......... .......... 15% 4.07M 5s 2024-06-06T03:40:18.567 INFO:tasks.workunit.client.0.smithi040.stderr: 21400K .......... .......... .......... .......... .......... 15% 22.3M 5s 2024-06-06T03:40:18.567 INFO:tasks.workunit.client.0.smithi040.stderr: 21450K .......... .......... .......... .......... .......... 15% 29.0M 5s 2024-06-06T03:40:18.567 INFO:tasks.workunit.client.0.smithi040.stderr: 21500K .......... .......... .......... .......... .......... 15% 24.1M 5s 2024-06-06T03:40:18.567 INFO:tasks.workunit.client.0.smithi040.stderr: 21550K .......... .......... .......... .......... .......... 15% 24.7M 5s 2024-06-06T03:40:18.567 INFO:tasks.workunit.client.0.smithi040.stderr: 21600K .......... .......... .......... .......... .......... 15% 88.1M 5s 2024-06-06T03:40:18.567 INFO:tasks.workunit.client.0.smithi040.stderr: 21650K .......... .......... .......... .......... .......... 15% 23.1M 5s 2024-06-06T03:40:18.567 INFO:tasks.workunit.client.0.smithi040.stderr: 21700K .......... .......... .......... .......... .......... 16% 23.1M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 21750K .......... .......... .......... .......... .......... 16% 25.5M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 21800K .......... .......... .......... .......... .......... 16% 40.3M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 21850K .......... .......... .......... .......... .......... 16% 38.8M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 21900K .......... .......... .......... .......... .......... 16% 25.9M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 21950K .......... .......... .......... .......... .......... 16% 24.8M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 22000K .......... .......... .......... .......... .......... 16% 28.2M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 22050K .......... .......... .......... .......... .......... 16% 22.3M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 22100K .......... .......... .......... .......... .......... 16% 66.3M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 22150K .......... .......... .......... .......... .......... 16% 23.9M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 22200K .......... .......... .......... .......... .......... 16% 30.9M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 22250K .......... .......... .......... .......... .......... 16% 24.6M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 22300K .......... .......... .......... .......... .......... 16% 22.0M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 22350K .......... .......... .......... .......... .......... 16% 53.0M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 22400K .......... .......... .......... .......... .......... 16% 27.3M 5s 2024-06-06T03:40:18.568 INFO:tasks.workunit.client.0.smithi040.stderr: 22450K .......... .......... .......... .......... .......... 16% 34.8M 5s 2024-06-06T03:40:18.569 INFO:tasks.workunit.client.0.smithi040.stderr: 22500K .......... .......... .......... .......... .......... 16% 13.8M 5s 2024-06-06T03:40:18.569 INFO:tasks.workunit.client.0.smithi040.stderr: 22550K .......... .......... .......... .......... .......... 16% 13.5M 5s 2024-06-06T03:40:18.570 INFO:tasks.workunit.client.0.smithi040.stderr: 22600K .......... .......... .......... .......... .......... 16% 21.2M 5s 2024-06-06T03:40:18.573 INFO:tasks.workunit.client.0.smithi040.stderr: 22650K .......... .......... .......... .......... .......... 16% 12.0M 5s 2024-06-06T03:40:18.577 INFO:tasks.workunit.client.0.smithi040.stderr: 22700K .......... .......... .......... .......... .......... 16% 11.8M 5s 2024-06-06T03:40:18.579 INFO:tasks.workunit.client.0.smithi040.stderr: 22750K .......... .......... .......... .......... .......... 16% 29.5M 5s 2024-06-06T03:40:18.613 INFO:tasks.workunit.client.0.smithi040.stderr: 22800K .......... .......... .......... .......... .......... 16% 13.6M 5s 2024-06-06T03:40:18.613 INFO:tasks.workunit.client.0.smithi040.stderr: 22850K .......... .......... .......... .......... .......... 16% 41.1M 5s 2024-06-06T03:40:18.613 INFO:tasks.workunit.client.0.smithi040.stderr: 22900K .......... .......... .......... .......... .......... 16% 25.5M 5s 2024-06-06T03:40:18.613 INFO:tasks.workunit.client.0.smithi040.stderr: 22950K .......... .......... .......... .......... .......... 16% 34.2M 5s 2024-06-06T03:40:18.614 INFO:tasks.workunit.client.0.smithi040.stderr: 23000K .......... .......... .......... .......... .......... 16% 24.4M 5s 2024-06-06T03:40:18.614 INFO:tasks.workunit.client.0.smithi040.stderr: 23050K .......... .......... .......... .......... .......... 17% 25.2M 5s 2024-06-06T03:40:18.614 INFO:tasks.workunit.client.0.smithi040.stderr: 23100K .......... .......... .......... .......... .......... 17% 43.9M 5s 2024-06-06T03:40:18.614 INFO:tasks.workunit.client.0.smithi040.stderr: 23150K .......... .......... .......... .......... .......... 17% 25.0M 5s 2024-06-06T03:40:18.614 INFO:tasks.workunit.client.0.smithi040.stderr: 23200K .......... .......... .......... .......... .......... 17% 36.9M 5s 2024-06-06T03:40:18.614 INFO:tasks.workunit.client.0.smithi040.stderr: 23250K .......... .......... .......... .......... .......... 17% 25.4M 5s 2024-06-06T03:40:18.614 INFO:tasks.workunit.client.0.smithi040.stderr: 23300K .......... .......... .......... .......... .......... 17% 25.5M 5s 2024-06-06T03:40:18.614 INFO:tasks.workunit.client.0.smithi040.stderr: 23350K .......... .......... .......... .......... .......... 17% 41.0M 5s 2024-06-06T03:40:18.614 INFO:tasks.workunit.client.0.smithi040.stderr: 23400K .......... .......... .......... .......... .......... 17% 25.1M 5s 2024-06-06T03:40:18.614 INFO:tasks.workunit.client.0.smithi040.stderr: 23450K .......... .......... .......... .......... .......... 17% 33.5M 5s 2024-06-06T03:40:18.614 INFO:tasks.workunit.client.0.smithi040.stderr: 23500K .......... .......... .......... .......... .......... 17% 25.6M 5s 2024-06-06T03:40:18.614 INFO:tasks.workunit.client.0.smithi040.stderr: 23550K .......... .......... .......... .......... .......... 17% 24.2M 5s 2024-06-06T03:40:18.614 INFO:tasks.workunit.client.0.smithi040.stderr: 23600K .......... .......... .......... .......... .......... 17% 21.5M 5s 2024-06-06T03:40:18.614 INFO:tasks.workunit.client.0.smithi040.stderr: 23650K .......... .......... .......... .......... .......... 17% 58.4M 5s 2024-06-06T03:40:18.615 INFO:tasks.workunit.client.0.smithi040.stderr: 23700K .......... .......... .......... .......... .......... 17% 15.6M 5s 2024-06-06T03:40:18.616 INFO:tasks.workunit.client.0.smithi040.stderr: 23750K .......... .......... .......... .......... .......... 17% 28.7M 5s 2024-06-06T03:40:18.619 INFO:tasks.workunit.client.0.smithi040.stderr: 23800K .......... .......... .......... .......... .......... 17% 14.1M 5s 2024-06-06T03:40:18.621 INFO:tasks.workunit.client.0.smithi040.stderr: 23850K .......... .......... .......... .......... .......... 17% 20.9M 5s 2024-06-06T03:40:18.623 INFO:tasks.workunit.client.0.smithi040.stderr: 23900K .......... .......... .......... .......... .......... 17% 23.3M 5s 2024-06-06T03:40:18.627 INFO:tasks.workunit.client.0.smithi040.stderr: 23950K .......... .......... .......... .......... .......... 17% 12.3M 5s 2024-06-06T03:40:18.630 INFO:tasks.workunit.client.0.smithi040.stderr: 24000K .......... .......... .......... .......... .......... 17% 14.3M 5s 2024-06-06T03:40:18.633 INFO:tasks.workunit.client.0.smithi040.stderr: 24050K .......... .......... .......... .......... .......... 17% 19.0M 5s 2024-06-06T03:40:18.636 INFO:tasks.workunit.client.0.smithi040.stderr: 24100K .......... .......... .......... .......... .......... 17% 13.8M 5s 2024-06-06T03:40:18.644 INFO:tasks.workunit.client.0.smithi040.stderr: 24150K .......... .......... .......... .......... .......... 17% 35.6M 5s 2024-06-06T03:40:18.645 INFO:tasks.workunit.client.0.smithi040.stderr: 24200K .......... .......... .......... .......... .......... 17% 25.7M 5s 2024-06-06T03:40:18.645 INFO:tasks.workunit.client.0.smithi040.stderr: 24250K .......... .......... .......... .......... .......... 17% 25.6M 5s 2024-06-06T03:40:18.645 INFO:tasks.workunit.client.0.smithi040.stderr: 24300K .......... .......... .......... .......... .......... 17% 29.0M 5s 2024-06-06T03:40:18.651 INFO:tasks.workunit.client.0.smithi040.stderr: 24350K .......... .......... .......... .......... .......... 17% 26.6M 5s 2024-06-06T03:40:18.651 INFO:tasks.workunit.client.0.smithi040.stderr: 24400K .......... .......... .......... .......... .......... 18% 41.7M 5s 2024-06-06T03:40:18.651 INFO:tasks.workunit.client.0.smithi040.stderr: 24450K .......... .......... .......... .......... .......... 18% 25.5M 5s 2024-06-06T03:40:18.651 INFO:tasks.workunit.client.0.smithi040.stderr: 24500K .......... .......... .......... .......... .......... 18% 33.6M 5s 2024-06-06T03:40:18.654 INFO:tasks.workunit.client.0.smithi040.stderr: 24550K .......... .......... .......... .......... .......... 18% 25.5M 5s 2024-06-06T03:40:18.654 INFO:tasks.workunit.client.0.smithi040.stderr: 24600K .......... .......... .......... .......... .......... 18% 23.7M 5s 2024-06-06T03:40:18.661 INFO:tasks.workunit.client.0.smithi040.stderr: 24650K .......... .......... .......... .......... .......... 18% 46.2M 5s 2024-06-06T03:40:18.661 INFO:tasks.workunit.client.0.smithi040.stderr: 24700K .......... .......... .......... .......... .......... 18% 22.9M 5s 2024-06-06T03:40:18.661 INFO:tasks.workunit.client.0.smithi040.stderr: 24750K .......... .......... .......... .......... .......... 18% 37.8M 5s 2024-06-06T03:40:18.661 INFO:tasks.workunit.client.0.smithi040.stderr: 24800K .......... .......... .......... .......... .......... 18% 22.2M 5s 2024-06-06T03:40:18.663 INFO:tasks.workunit.client.0.smithi040.stderr: 24850K .......... .......... .......... .......... .......... 18% 25.7M 5s 2024-06-06T03:40:18.669 INFO:tasks.workunit.client.0.smithi040.stderr: 24900K .......... .......... .......... .......... .......... 18% 30.8M 5s 2024-06-06T03:40:18.669 INFO:tasks.workunit.client.0.smithi040.stderr: 24950K .......... .......... .......... .......... .......... 18% 16.9M 5s 2024-06-06T03:40:18.669 INFO:tasks.workunit.client.0.smithi040.stderr: 25000K .......... .......... .......... .......... .......... 18% 36.3M 5s 2024-06-06T03:40:18.670 INFO:tasks.workunit.client.0.smithi040.stderr: 25050K .......... .......... .......... .......... .......... 18% 22.9M 5s 2024-06-06T03:40:18.675 INFO:tasks.workunit.client.0.smithi040.stderr: 25100K .......... .......... .......... .......... .......... 18% 14.7M 5s 2024-06-06T03:40:18.675 INFO:tasks.workunit.client.0.smithi040.stderr: 25150K .......... .......... .......... .......... .......... 18% 40.0M 5s 2024-06-06T03:40:18.678 INFO:tasks.workunit.client.0.smithi040.stderr: 25200K .......... .......... .......... .......... .......... 18% 15.1M 5s 2024-06-06T03:40:18.682 INFO:tasks.workunit.client.0.smithi040.stderr: 25250K .......... .......... .......... .......... .......... 18% 14.5M 5s 2024-06-06T03:40:18.684 INFO:tasks.workunit.client.0.smithi040.stderr: 25300K .......... .......... .......... .......... .......... 18% 22.2M 5s 2024-06-06T03:40:18.688 INFO:tasks.workunit.client.0.smithi040.stderr: 25350K .......... .......... .......... .......... .......... 18% 11.8M 5s 2024-06-06T03:40:18.689 INFO:tasks.workunit.client.0.smithi040.stderr: 25400K .......... .......... .......... .......... .......... 18% 35.4M 5s 2024-06-06T03:40:18.695 INFO:tasks.workunit.client.0.smithi040.stderr: 25450K .......... .......... .......... .......... .......... 18% 13.0M 5s 2024-06-06T03:40:18.695 INFO:tasks.workunit.client.0.smithi040.stderr: 25500K .......... .......... .......... .......... .......... 18% 24.3M 5s 2024-06-06T03:40:18.725 INFO:tasks.workunit.client.0.smithi040.stderr: 25550K .......... .......... .......... .......... .......... 18% 29.8M 5s 2024-06-06T03:40:18.726 INFO:tasks.workunit.client.0.smithi040.stderr: 25600K .......... .......... .......... .......... .......... 18% 23.3M 5s 2024-06-06T03:40:18.726 INFO:tasks.workunit.client.0.smithi040.stderr: 25650K .......... .......... .......... .......... .......... 18% 42.8M 5s 2024-06-06T03:40:18.726 INFO:tasks.workunit.client.0.smithi040.stderr: 25700K .......... .......... .......... .......... .......... 18% 25.5M 5s 2024-06-06T03:40:18.726 INFO:tasks.workunit.client.0.smithi040.stderr: 25750K .......... .......... .......... .......... .......... 19% 26.0M 5s 2024-06-06T03:40:18.726 INFO:tasks.workunit.client.0.smithi040.stderr: 25800K .......... .......... .......... .......... .......... 19% 29.3M 5s 2024-06-06T03:40:18.726 INFO:tasks.workunit.client.0.smithi040.stderr: 25850K .......... .......... .......... .......... .......... 19% 30.0M 5s 2024-06-06T03:40:18.726 INFO:tasks.workunit.client.0.smithi040.stderr: 25900K .......... .......... .......... .......... .......... 19% 35.9M 5s 2024-06-06T03:40:18.726 INFO:tasks.workunit.client.0.smithi040.stderr: 25950K .......... .......... .......... .......... .......... 19% 22.0M 5s 2024-06-06T03:40:18.726 INFO:tasks.workunit.client.0.smithi040.stderr: 26000K .......... .......... .......... .......... .......... 19% 26.8M 5s 2024-06-06T03:40:18.726 INFO:tasks.workunit.client.0.smithi040.stderr: 26050K .......... .......... .......... .......... .......... 19% 30.6M 5s 2024-06-06T03:40:18.726 INFO:tasks.workunit.client.0.smithi040.stderr: 26100K .......... .......... .......... .......... .......... 19% 23.2M 5s 2024-06-06T03:40:18.726 INFO:tasks.workunit.client.0.smithi040.stderr: 26150K .......... .......... .......... .......... .......... 19% 40.8M 5s 2024-06-06T03:40:18.726 INFO:tasks.workunit.client.0.smithi040.stderr: 26200K .......... .......... .......... .......... .......... 19% 15.2M 5s 2024-06-06T03:40:18.726 INFO:tasks.workunit.client.0.smithi040.stderr: 26250K .......... .......... .......... .......... .......... 19% 31.1M 5s 2024-06-06T03:40:18.727 INFO:tasks.workunit.client.0.smithi040.stderr: 26300K .......... .......... .......... .......... .......... 19% 22.0M 5s 2024-06-06T03:40:18.731 INFO:tasks.workunit.client.0.smithi040.stderr: 26350K .......... .......... .......... .......... .......... 19% 19.3M 5s 2024-06-06T03:40:18.731 INFO:tasks.workunit.client.0.smithi040.stderr: 26400K .......... .......... .......... .......... .......... 19% 26.5M 5s 2024-06-06T03:40:18.731 INFO:tasks.workunit.client.0.smithi040.stderr: 26450K .......... .......... .......... .......... .......... 19% 19.2M 5s 2024-06-06T03:40:18.735 INFO:tasks.workunit.client.0.smithi040.stderr: 26500K .......... .......... .......... .......... .......... 19% 24.9M 5s 2024-06-06T03:40:18.735 INFO:tasks.workunit.client.0.smithi040.stderr: 26550K .......... .......... .......... .......... .......... 19% 23.1M 5s 2024-06-06T03:40:18.738 INFO:tasks.workunit.client.0.smithi040.stderr: 26600K .......... .......... .......... .......... .......... 19% 13.8M 5s 2024-06-06T03:40:18.741 INFO:tasks.workunit.client.0.smithi040.stderr: 26650K .......... .......... .......... .......... .......... 19% 30.3M 5s 2024-06-06T03:40:18.745 INFO:tasks.workunit.client.0.smithi040.stderr: 26700K .......... .......... .......... .......... .......... 19% 12.4M 5s 2024-06-06T03:40:18.748 INFO:tasks.workunit.client.0.smithi040.stderr: 26750K .......... .......... .......... .......... .......... 19% 14.5M 5s 2024-06-06T03:40:18.751 INFO:tasks.workunit.client.0.smithi040.stderr: 26800K .......... .......... .......... .......... .......... 19% 19.1M 5s 2024-06-06T03:40:18.760 INFO:tasks.workunit.client.0.smithi040.stderr: 26850K .......... .......... .......... .......... .......... 19% 33.3M 5s 2024-06-06T03:40:18.760 INFO:tasks.workunit.client.0.smithi040.stderr: 26900K .......... .......... .......... .......... .......... 19% 38.5M 5s 2024-06-06T03:40:18.760 INFO:tasks.workunit.client.0.smithi040.stderr: 26950K .......... .......... .......... .......... .......... 19% 27.6M 5s 2024-06-06T03:40:18.760 INFO:tasks.workunit.client.0.smithi040.stderr: 27000K .......... .......... .......... .......... .......... 19% 25.9M 5s 2024-06-06T03:40:18.760 INFO:tasks.workunit.client.0.smithi040.stderr: 27050K .......... .......... .......... .......... .......... 19% 24.5M 5s 2024-06-06T03:40:18.761 INFO:tasks.workunit.client.0.smithi040.stderr: 27100K .......... .......... .......... .......... .......... 19% 35.0M 5s 2024-06-06T03:40:18.762 INFO:tasks.workunit.client.0.smithi040.stderr: 27150K .......... .......... .......... .......... .......... 20% 29.9M 5s 2024-06-06T03:40:18.764 INFO:tasks.workunit.client.0.smithi040.stderr: 27200K .......... .......... .......... .......... .......... 20% 19.2M 5s 2024-06-06T03:40:18.766 INFO:tasks.workunit.client.0.smithi040.stderr: 27250K .......... .......... .......... .......... .......... 20% 27.2M 5s 2024-06-06T03:40:18.767 INFO:tasks.workunit.client.0.smithi040.stderr: 27300K .......... .......... .......... .......... .......... 20% 30.2M 5s 2024-06-06T03:40:18.770 INFO:tasks.workunit.client.0.smithi040.stderr: 27350K .......... .......... .......... .......... .......... 20% 24.5M 5s 2024-06-06T03:40:18.772 INFO:tasks.workunit.client.0.smithi040.stderr: 27400K .......... .......... .......... .......... .......... 20% 36.9M 5s 2024-06-06T03:40:18.775 INFO:tasks.workunit.client.0.smithi040.stderr: 27450K .......... .......... .......... .......... .......... 20% 17.2M 5s 2024-06-06T03:40:18.776 INFO:tasks.workunit.client.0.smithi040.stderr: 27500K .......... .......... .......... .......... .......... 20% 24.9M 5s 2024-06-06T03:40:18.777 INFO:tasks.workunit.client.0.smithi040.stderr: 27550K .......... .......... .......... .......... .......... 20% 37.7M 5s 2024-06-06T03:40:18.779 INFO:tasks.workunit.client.0.smithi040.stderr: 27600K .......... .......... .......... .......... .......... 20% 23.3M 5s 2024-06-06T03:40:18.781 INFO:tasks.workunit.client.0.smithi040.stderr: 27650K .......... .......... .......... .......... .......... 20% 22.1M 5s 2024-06-06T03:40:18.784 INFO:tasks.workunit.client.0.smithi040.stderr: 27700K .......... .......... .......... .......... .......... 20% 21.2M 5s 2024-06-06T03:40:18.787 INFO:tasks.workunit.client.0.smithi040.stderr: 27750K .......... .......... .......... .......... .......... 20% 19.2M 5s 2024-06-06T03:40:18.788 INFO:tasks.workunit.client.0.smithi040.stderr: 27800K .......... .......... .......... .......... .......... 20% 24.6M 5s 2024-06-06T03:40:18.791 INFO:tasks.workunit.client.0.smithi040.stderr: 27850K .......... .......... .......... .......... .......... 20% 17.6M 5s 2024-06-06T03:40:18.794 INFO:tasks.workunit.client.0.smithi040.stderr: 27900K .......... .......... .......... .......... .......... 20% 25.5M 5s 2024-06-06T03:40:18.796 INFO:tasks.workunit.client.0.smithi040.stderr: 27950K .......... .......... .......... .......... .......... 20% 19.0M 5s 2024-06-06T03:40:18.799 INFO:tasks.workunit.client.0.smithi040.stderr: 28000K .......... .......... .......... .......... .......... 20% 14.9M 5s 2024-06-06T03:40:18.802 INFO:tasks.workunit.client.0.smithi040.stderr: 28050K .......... .......... .......... .......... .......... 20% 20.1M 5s 2024-06-06T03:40:18.804 INFO:tasks.workunit.client.0.smithi040.stderr: 28100K .......... .......... .......... .......... .......... 20% 13.5M 5s 2024-06-06T03:40:18.810 INFO:tasks.workunit.client.0.smithi040.stderr: 28150K .......... .......... .......... .......... .......... 20% 27.4M 5s 2024-06-06T03:40:18.811 INFO:tasks.workunit.client.0.smithi040.stderr: 28200K .......... .......... .......... .......... .......... 20% 24.7M 5s 2024-06-06T03:40:18.811 INFO:tasks.workunit.client.0.smithi040.stderr: 28250K .......... .......... .......... .......... .......... 20% 30.0M 5s 2024-06-06T03:40:18.815 INFO:tasks.workunit.client.0.smithi040.stderr: 28300K .......... .......... .......... .......... .......... 20% 36.2M 5s 2024-06-06T03:40:18.815 INFO:tasks.workunit.client.0.smithi040.stderr: 28350K .......... .......... .......... .......... .......... 20% 24.4M 5s 2024-06-06T03:40:18.815 INFO:tasks.workunit.client.0.smithi040.stderr: 28400K .......... .......... .......... .......... .......... 20% 42.5M 5s 2024-06-06T03:40:18.820 INFO:tasks.workunit.client.0.smithi040.stderr: 28450K .......... .......... .......... .......... .......... 20% 20.0M 5s 2024-06-06T03:40:18.820 INFO:tasks.workunit.client.0.smithi040.stderr: 28500K .......... .......... .......... .......... .......... 21% 27.3M 5s 2024-06-06T03:40:18.820 INFO:tasks.workunit.client.0.smithi040.stderr: 28550K .......... .......... .......... .......... .......... 21% 32.0M 5s 2024-06-06T03:40:18.823 INFO:tasks.workunit.client.0.smithi040.stderr: 28600K .......... .......... .......... .......... .......... 21% 23.3M 5s 2024-06-06T03:40:18.836 INFO:tasks.workunit.client.0.smithi040.stderr: 28650K .......... .......... .......... .......... .......... 21% 36.7M 5s 2024-06-06T03:40:18.837 INFO:tasks.workunit.client.0.smithi040.stderr: 28700K .......... .......... .......... .......... .......... 21% 16.9M 5s 2024-06-06T03:40:18.837 INFO:tasks.workunit.client.0.smithi040.stderr: 28750K .......... .......... .......... .......... .......... 21% 25.0M 5s 2024-06-06T03:40:18.837 INFO:tasks.workunit.client.0.smithi040.stderr: 28800K .......... .......... .......... .......... .......... 21% 37.4M 5s 2024-06-06T03:40:18.837 INFO:tasks.workunit.client.0.smithi040.stderr: 28850K .......... .......... .......... .......... .......... 21% 25.4M 5s 2024-06-06T03:40:18.837 INFO:tasks.workunit.client.0.smithi040.stderr: 28900K .......... .......... .......... .......... .......... 21% 22.0M 5s 2024-06-06T03:40:18.837 INFO:tasks.workunit.client.0.smithi040.stderr: 28950K .......... .......... .......... .......... .......... 21% 20.7M 5s 2024-06-06T03:40:18.842 INFO:tasks.workunit.client.0.smithi040.stderr: 29000K .......... .......... .......... .......... .......... 21% 27.0M 5s 2024-06-06T03:40:18.842 INFO:tasks.workunit.client.0.smithi040.stderr: 29050K .......... .......... .......... .......... .......... 21% 22.6M 5s 2024-06-06T03:40:18.844 INFO:tasks.workunit.client.0.smithi040.stderr: 29100K .......... .......... .......... .......... .......... 21% 16.6M 5s 2024-06-06T03:40:18.846 INFO:tasks.workunit.client.0.smithi040.stderr: 29150K .......... .......... .......... .......... .......... 21% 19.0M 5s 2024-06-06T03:40:18.848 INFO:tasks.workunit.client.0.smithi040.stderr: 29200K .......... .......... .......... .......... .......... 21% 30.2M 5s 2024-06-06T03:40:18.850 INFO:tasks.workunit.client.0.smithi040.stderr: 29250K .......... .......... .......... .......... .......... 21% 19.6M 5s 2024-06-06T03:40:18.852 INFO:tasks.workunit.client.0.smithi040.stderr: 29300K .......... .......... .......... .......... .......... 21% 26.5M 5s 2024-06-06T03:40:18.856 INFO:tasks.workunit.client.0.smithi040.stderr: 29350K .......... .......... .......... .......... .......... 21% 17.6M 5s 2024-06-06T03:40:18.860 INFO:tasks.workunit.client.0.smithi040.stderr: 29400K .......... .......... .......... .......... .......... 21% 17.0M 5s 2024-06-06T03:40:18.861 INFO:tasks.workunit.client.0.smithi040.stderr: 29450K .......... .......... .......... .......... .......... 21% 15.3M 5s 2024-06-06T03:40:18.863 INFO:tasks.workunit.client.0.smithi040.stderr: 29500K .......... .......... .......... .......... .......... 21% 20.9M 5s 2024-06-06T03:40:18.866 INFO:tasks.workunit.client.0.smithi040.stderr: 29550K .......... .......... .......... .......... .......... 21% 33.8M 5s 2024-06-06T03:40:18.868 INFO:tasks.workunit.client.0.smithi040.stderr: 29600K .......... .......... .......... .......... .......... 21% 30.3M 5s 2024-06-06T03:40:18.868 INFO:tasks.workunit.client.0.smithi040.stderr: 29650K .......... .......... .......... .......... .......... 21% 21.1M 5s 2024-06-06T03:40:18.877 INFO:tasks.workunit.client.0.smithi040.stderr: 29700K .......... .......... .......... .......... .......... 21% 27.0M 5s 2024-06-06T03:40:18.877 INFO:tasks.workunit.client.0.smithi040.stderr: 29750K .......... .......... .......... .......... .......... 21% 26.9M 5s 2024-06-06T03:40:18.877 INFO:tasks.workunit.client.0.smithi040.stderr: 29800K .......... .......... .......... .......... .......... 21% 23.1M 5s 2024-06-06T03:40:18.877 INFO:tasks.workunit.client.0.smithi040.stderr: 29850K .......... .......... .......... .......... .......... 22% 34.3M 5s 2024-06-06T03:40:18.880 INFO:tasks.workunit.client.0.smithi040.stderr: 29900K .......... .......... .......... .......... .......... 22% 23.1M 5s 2024-06-06T03:40:18.880 INFO:tasks.workunit.client.0.smithi040.stderr: 29950K .......... .......... .......... .......... .......... 22% 47.2M 5s 2024-06-06T03:40:18.887 INFO:tasks.workunit.client.0.smithi040.stderr: 30000K .......... .......... .......... .......... .......... 22% 17.9M 5s 2024-06-06T03:40:18.887 INFO:tasks.workunit.client.0.smithi040.stderr: 30050K .......... .......... .......... .......... .......... 22% 33.0M 5s 2024-06-06T03:40:18.887 INFO:tasks.workunit.client.0.smithi040.stderr: 30100K .......... .......... .......... .......... .......... 22% 29.8M 5s 2024-06-06T03:40:18.887 INFO:tasks.workunit.client.0.smithi040.stderr: 30150K .......... .......... .......... .......... .......... 22% 23.6M 5s 2024-06-06T03:40:18.889 INFO:tasks.workunit.client.0.smithi040.stderr: 30200K .......... .......... .......... .......... .......... 22% 24.3M 5s 2024-06-06T03:40:18.892 INFO:tasks.workunit.client.0.smithi040.stderr: 30250K .......... .......... .......... .......... .......... 22% 17.8M 5s 2024-06-06T03:40:18.895 INFO:tasks.workunit.client.0.smithi040.stderr: 30300K .......... .......... .......... .......... .......... 22% 27.3M 5s 2024-06-06T03:40:18.895 INFO:tasks.workunit.client.0.smithi040.stderr: 30350K .......... .......... .......... .......... .......... 22% 27.3M 5s 2024-06-06T03:40:18.898 INFO:tasks.workunit.client.0.smithi040.stderr: 30400K .......... .......... .......... .......... .......... 22% 18.9M 5s 2024-06-06T03:40:18.902 INFO:tasks.workunit.client.0.smithi040.stderr: 30450K .......... .......... .......... .......... .......... 22% 30.7M 5s 2024-06-06T03:40:18.903 INFO:tasks.workunit.client.0.smithi040.stderr: 30500K .......... .......... .......... .......... .......... 22% 18.8M 5s 2024-06-06T03:40:18.905 INFO:tasks.workunit.client.0.smithi040.stderr: 30550K .......... .......... .......... .......... .......... 22% 16.9M 5s 2024-06-06T03:40:18.908 INFO:tasks.workunit.client.0.smithi040.stderr: 30600K .......... .......... .......... .......... .......... 22% 22.9M 5s 2024-06-06T03:40:18.910 INFO:tasks.workunit.client.0.smithi040.stderr: 30650K .......... .......... .......... .......... .......... 22% 15.6M 5s 2024-06-06T03:40:18.911 INFO:tasks.workunit.client.0.smithi040.stderr: 30700K .......... .......... .......... .......... .......... 22% 42.7M 5s 2024-06-06T03:40:18.916 INFO:tasks.workunit.client.0.smithi040.stderr: 30750K .......... .......... .......... .......... .......... 22% 13.5M 5s 2024-06-06T03:40:18.918 INFO:tasks.workunit.client.0.smithi040.stderr: 30800K .......... .......... .......... .......... .......... 22% 15.0M 5s 2024-06-06T03:40:18.919 INFO:tasks.workunit.client.0.smithi040.stderr: 30850K .......... .......... .......... .......... .......... 22% 36.4M 5s 2024-06-06T03:40:18.923 INFO:tasks.workunit.client.0.smithi040.stderr: 30900K .......... .......... .......... .......... .......... 22% 20.5M 5s 2024-06-06T03:40:18.923 INFO:tasks.workunit.client.0.smithi040.stderr: 30950K .......... .......... .......... .......... .......... 22% 34.5M 5s 2024-06-06T03:40:18.925 INFO:tasks.workunit.client.0.smithi040.stderr: 31000K .......... .......... .......... .......... .......... 22% 19.4M 5s 2024-06-06T03:40:18.927 INFO:tasks.workunit.client.0.smithi040.stderr: 31050K .......... .......... .......... .......... .......... 22% 27.1M 5s 2024-06-06T03:40:18.928 INFO:tasks.workunit.client.0.smithi040.stderr: 31100K .......... .......... .......... .......... .......... 22% 43.0M 5s 2024-06-06T03:40:18.931 INFO:tasks.workunit.client.0.smithi040.stderr: 31150K .......... .......... .......... .......... .......... 22% 22.6M 5s 2024-06-06T03:40:18.932 INFO:tasks.workunit.client.0.smithi040.stderr: 31200K .......... .......... .......... .......... .......... 23% 35.6M 5s 2024-06-06T03:40:18.935 INFO:tasks.workunit.client.0.smithi040.stderr: 31250K .......... .......... .......... .......... .......... 23% 18.5M 5s 2024-06-06T03:40:18.937 INFO:tasks.workunit.client.0.smithi040.stderr: 31300K .......... .......... .......... .......... .......... 23% 22.9M 5s 2024-06-06T03:40:18.938 INFO:tasks.workunit.client.0.smithi040.stderr: 31350K .......... .......... .......... .......... .......... 23% 27.0M 5s 2024-06-06T03:40:18.940 INFO:tasks.workunit.client.0.smithi040.stderr: 31400K .......... .......... .......... .......... .......... 23% 25.8M 5s 2024-06-06T03:40:18.941 INFO:tasks.workunit.client.0.smithi040.stderr: 31450K .......... .......... .......... .......... .......... 23% 41.9M 5s 2024-06-06T03:40:18.944 INFO:tasks.workunit.client.0.smithi040.stderr: 31500K .......... .......... .......... .......... .......... 23% 24.1M 5s 2024-06-06T03:40:18.947 INFO:tasks.workunit.client.0.smithi040.stderr: 31550K .......... .......... .......... .......... .......... 23% 15.8M 5s 2024-06-06T03:40:18.948 INFO:tasks.workunit.client.0.smithi040.stderr: 31600K .......... .......... .......... .......... .......... 23% 25.9M 5s 2024-06-06T03:40:18.950 INFO:tasks.workunit.client.0.smithi040.stderr: 31650K .......... .......... .......... .......... .......... 23% 29.4M 5s 2024-06-06T03:40:18.951 INFO:tasks.workunit.client.0.smithi040.stderr: 31700K .......... .......... .......... .......... .......... 23% 37.5M 5s 2024-06-06T03:40:18.954 INFO:tasks.workunit.client.0.smithi040.stderr: 31750K .......... .......... .......... .......... .......... 23% 18.8M 5s 2024-06-06T03:40:18.957 INFO:tasks.workunit.client.0.smithi040.stderr: 31800K .......... .......... .......... .......... .......... 23% 17.8M 5s 2024-06-06T03:40:18.960 INFO:tasks.workunit.client.0.smithi040.stderr: 31850K .......... .......... .......... .......... .......... 23% 19.1M 5s 2024-06-06T03:40:18.961 INFO:tasks.workunit.client.0.smithi040.stderr: 31900K .......... .......... .......... .......... .......... 23% 27.0M 5s 2024-06-06T03:40:18.963 INFO:tasks.workunit.client.0.smithi040.stderr: 31950K .......... .......... .......... .......... .......... 23% 21.8M 5s 2024-06-06T03:40:18.966 INFO:tasks.workunit.client.0.smithi040.stderr: 32000K .......... .......... .......... .......... .......... 23% 20.2M 5s 2024-06-06T03:40:18.968 INFO:tasks.workunit.client.0.smithi040.stderr: 32050K .......... .......... .......... .......... .......... 23% 22.3M 5s 2024-06-06T03:40:18.971 INFO:tasks.workunit.client.0.smithi040.stderr: 32100K .......... .......... .......... .......... .......... 23% 15.8M 5s 2024-06-06T03:40:18.973 INFO:tasks.workunit.client.0.smithi040.stderr: 32150K .......... .......... .......... .......... .......... 23% 31.6M 5s 2024-06-06T03:40:18.976 INFO:tasks.workunit.client.0.smithi040.stderr: 32200K .......... .......... .......... .......... .......... 23% 15.7M 5s 2024-06-06T03:40:18.979 INFO:tasks.workunit.client.0.smithi040.stderr: 32250K .......... .......... .......... .......... .......... 23% 19.1M 5s 2024-06-06T03:40:18.980 INFO:tasks.workunit.client.0.smithi040.stderr: 32300K .......... .......... .......... .......... .......... 23% 30.4M 5s 2024-06-06T03:40:18.982 INFO:tasks.workunit.client.0.smithi040.stderr: 32350K .......... .......... .......... .......... .......... 23% 22.8M 5s 2024-06-06T03:40:18.984 INFO:tasks.workunit.client.0.smithi040.stderr: 32400K .......... .......... .......... .......... .......... 23% 24.5M 5s 2024-06-06T03:40:18.986 INFO:tasks.workunit.client.0.smithi040.stderr: 32450K .......... .......... .......... .......... .......... 23% 34.1M 5s 2024-06-06T03:40:18.987 INFO:tasks.workunit.client.0.smithi040.stderr: 32500K .......... .......... .......... .......... .......... 23% 21.8M 5s 2024-06-06T03:40:18.991 INFO:tasks.workunit.client.0.smithi040.stderr: 32550K .......... .......... .......... .......... .......... 24% 21.0M 5s 2024-06-06T03:40:18.992 INFO:tasks.workunit.client.0.smithi040.stderr: 32600K .......... .......... .......... .......... .......... 24% 26.0M 5s 2024-06-06T03:40:18.994 INFO:tasks.workunit.client.0.smithi040.stderr: 32650K .......... .......... .......... .......... .......... 24% 25.8M 5s 2024-06-06T03:40:18.995 INFO:tasks.workunit.client.0.smithi040.stderr: 32700K .......... .......... .......... .......... .......... 24% 40.0M 5s 2024-06-06T03:40:18.997 INFO:tasks.workunit.client.0.smithi040.stderr: 32750K .......... .......... .......... .......... .......... 24% 26.1M 5s 2024-06-06T03:40:18.999 INFO:tasks.workunit.client.0.smithi040.stderr: 32800K .......... .......... .......... .......... .......... 24% 22.0M 5s 2024-06-06T03:40:19.001 INFO:tasks.workunit.client.0.smithi040.stderr: 32850K .......... .......... .......... .......... .......... 24% 27.2M 5s 2024-06-06T03:40:19.003 INFO:tasks.workunit.client.0.smithi040.stderr: 32900K .......... .......... .......... .......... .......... 24% 19.5M 5s 2024-06-06T03:40:19.005 INFO:tasks.workunit.client.0.smithi040.stderr: 32950K .......... .......... .......... .......... .......... 24% 37.0M 5s 2024-06-06T03:40:19.007 INFO:tasks.workunit.client.0.smithi040.stderr: 33000K .......... .......... .......... .......... .......... 24% 22.6M 5s 2024-06-06T03:40:19.009 INFO:tasks.workunit.client.0.smithi040.stderr: 33050K .......... .......... .......... .......... .......... 24% 22.4M 5s 2024-06-06T03:40:19.011 INFO:tasks.workunit.client.0.smithi040.stderr: 33100K .......... .......... .......... .......... .......... 24% 25.5M 5s 2024-06-06T03:40:19.015 INFO:tasks.workunit.client.0.smithi040.stderr: 33150K .......... .......... .......... .......... .......... 24% 13.9M 5s 2024-06-06T03:40:19.016 INFO:tasks.workunit.client.0.smithi040.stderr: 33200K .......... .......... .......... .......... .......... 24% 48.0M 5s 2024-06-06T03:40:19.020 INFO:tasks.workunit.client.0.smithi040.stderr: 33250K .......... .......... .......... .......... .......... 24% 21.3M 5s 2024-06-06T03:40:19.023 INFO:tasks.workunit.client.0.smithi040.stderr: 33300K .......... .......... .......... .......... .......... 24% 16.1M 5s 2024-06-06T03:40:19.023 INFO:tasks.workunit.client.0.smithi040.stderr: 33350K .......... .......... .......... .......... .......... 24% 30.0M 5s 2024-06-06T03:40:19.027 INFO:tasks.workunit.client.0.smithi040.stderr: 33400K .......... .......... .......... .......... .......... 24% 16.6M 5s 2024-06-06T03:40:19.027 INFO:tasks.workunit.client.0.smithi040.stderr: 33450K .......... .......... .......... .......... .......... 24% 25.3M 5s 2024-06-06T03:40:19.030 INFO:tasks.workunit.client.0.smithi040.stderr: 33500K .......... .......... .......... .......... .......... 24% 20.9M 5s 2024-06-06T03:40:19.033 INFO:tasks.workunit.client.0.smithi040.stderr: 33550K .......... .......... .......... .......... .......... 24% 16.2M 5s 2024-06-06T03:40:19.035 INFO:tasks.workunit.client.0.smithi040.stderr: 33600K .......... .......... .......... .......... .......... 24% 24.9M 5s 2024-06-06T03:40:19.040 INFO:tasks.workunit.client.0.smithi040.stderr: 33650K .......... .......... .......... .......... .......... 24% 19.0M 5s 2024-06-06T03:40:19.040 INFO:tasks.workunit.client.0.smithi040.stderr: 33700K .......... .......... .......... .......... .......... 24% 37.4M 5s 2024-06-06T03:40:19.044 INFO:tasks.workunit.client.0.smithi040.stderr: 33750K .......... .......... .......... .......... .......... 24% 16.9M 5s 2024-06-06T03:40:19.044 INFO:tasks.workunit.client.0.smithi040.stderr: 33800K .......... .......... .......... .......... .......... 24% 33.0M 5s 2024-06-06T03:40:19.046 INFO:tasks.workunit.client.0.smithi040.stderr: 33850K .......... .......... .......... .......... .......... 24% 32.1M 5s 2024-06-06T03:40:19.047 INFO:tasks.workunit.client.0.smithi040.stderr: 33900K .......... .......... .......... .......... .......... 25% 20.4M 5s 2024-06-06T03:40:19.050 INFO:tasks.workunit.client.0.smithi040.stderr: 33950K .......... .......... .......... .......... .......... 25% 29.3M 5s 2024-06-06T03:40:19.061 INFO:tasks.workunit.client.0.smithi040.stderr: 34000K .......... .......... .......... .......... .......... 25% 21.5M 5s 2024-06-06T03:40:19.061 INFO:tasks.workunit.client.0.smithi040.stderr: 34050K .......... .......... .......... .......... .......... 25% 25.0M 5s 2024-06-06T03:40:19.061 INFO:tasks.workunit.client.0.smithi040.stderr: 34100K .......... .......... .......... .......... .......... 25% 41.5M 5s 2024-06-06T03:40:19.061 INFO:tasks.workunit.client.0.smithi040.stderr: 34150K .......... .......... .......... .......... .......... 25% 21.5M 5s 2024-06-06T03:40:19.061 INFO:tasks.workunit.client.0.smithi040.stderr: 34200K .......... .......... .......... .......... .......... 25% 17.7M 5s 2024-06-06T03:40:19.061 INFO:tasks.workunit.client.0.smithi040.stderr: 34250K .......... .......... .......... .......... .......... 25% 36.6M 5s 2024-06-06T03:40:19.069 INFO:tasks.workunit.client.0.smithi040.stderr: 34300K .......... .......... .......... .......... .......... 25% 21.2M 5s 2024-06-06T03:40:19.069 INFO:tasks.workunit.client.0.smithi040.stderr: 34350K .......... .......... .......... .......... .......... 25% 39.5M 5s 2024-06-06T03:40:19.069 INFO:tasks.workunit.client.0.smithi040.stderr: 34400K .......... .......... .......... .......... .......... 25% 25.5M 5s 2024-06-06T03:40:19.069 INFO:tasks.workunit.client.0.smithi040.stderr: 34450K .......... .......... .......... .......... .......... 25% 15.8M 5s 2024-06-06T03:40:19.071 INFO:tasks.workunit.client.0.smithi040.stderr: 34500K .......... .......... .......... .......... .......... 25% 30.9M 5s 2024-06-06T03:40:19.077 INFO:tasks.workunit.client.0.smithi040.stderr: 34550K .......... .......... .......... .......... .......... 25% 23.7M 5s 2024-06-06T03:40:19.077 INFO:tasks.workunit.client.0.smithi040.stderr: 34600K .......... .......... .......... .......... .......... 25% 22.8M 5s 2024-06-06T03:40:19.077 INFO:tasks.workunit.client.0.smithi040.stderr: 34650K .......... .......... .......... .......... .......... 25% 22.3M 5s 2024-06-06T03:40:19.080 INFO:tasks.workunit.client.0.smithi040.stderr: 34700K .......... .......... .......... .......... .......... 25% 17.7M 5s 2024-06-06T03:40:19.082 INFO:tasks.workunit.client.0.smithi040.stderr: 34750K .......... .......... .......... .......... .......... 25% 19.6M 5s 2024-06-06T03:40:19.084 INFO:tasks.workunit.client.0.smithi040.stderr: 34800K .......... .......... .......... .......... .......... 25% 33.4M 5s 2024-06-06T03:40:19.087 INFO:tasks.workunit.client.0.smithi040.stderr: 34850K .......... .......... .......... .......... .......... 25% 18.8M 5s 2024-06-06T03:40:19.090 INFO:tasks.workunit.client.0.smithi040.stderr: 34900K .......... .......... .......... .......... .......... 25% 17.5M 5s 2024-06-06T03:40:19.092 INFO:tasks.workunit.client.0.smithi040.stderr: 34950K .......... .......... .......... .......... .......... 25% 18.1M 5s 2024-06-06T03:40:19.094 INFO:tasks.workunit.client.0.smithi040.stderr: 35000K .......... .......... .......... .......... .......... 25% 25.3M 5s 2024-06-06T03:40:19.096 INFO:tasks.workunit.client.0.smithi040.stderr: 35050K .......... .......... .......... .......... .......... 25% 26.8M 5s 2024-06-06T03:40:19.098 INFO:tasks.workunit.client.0.smithi040.stderr: 35100K .......... .......... .......... .......... .......... 25% 21.2M 5s 2024-06-06T03:40:19.102 INFO:tasks.workunit.client.0.smithi040.stderr: 35150K .......... .......... .......... .......... .......... 25% 18.8M 5s 2024-06-06T03:40:19.102 INFO:tasks.workunit.client.0.smithi040.stderr: 35200K .......... .......... .......... .......... .......... 25% 47.2M 5s 2024-06-06T03:40:19.104 INFO:tasks.workunit.client.0.smithi040.stderr: 35250K .......... .......... .......... .......... .......... 26% 20.2M 5s 2024-06-06T03:40:19.116 INFO:tasks.workunit.client.0.smithi040.stderr: 35300K .......... .......... .......... .......... .......... 26% 21.3M 5s 2024-06-06T03:40:19.116 INFO:tasks.workunit.client.0.smithi040.stderr: 35350K .......... .......... .......... .......... .......... 26% 27.3M 5s 2024-06-06T03:40:19.116 INFO:tasks.workunit.client.0.smithi040.stderr: 35400K .......... .......... .......... .......... .......... 26% 32.8M 5s 2024-06-06T03:40:19.116 INFO:tasks.workunit.client.0.smithi040.stderr: 35450K .......... .......... .......... .......... .......... 26% 24.2M 5s 2024-06-06T03:40:19.116 INFO:tasks.workunit.client.0.smithi040.stderr: 35500K .......... .......... .......... .......... .......... 26% 36.7M 5s 2024-06-06T03:40:19.117 INFO:tasks.workunit.client.0.smithi040.stderr: 35550K .......... .......... .......... .......... .......... 26% 19.8M 5s 2024-06-06T03:40:19.119 INFO:tasks.workunit.client.0.smithi040.stderr: 35600K .......... .......... .......... .......... .......... 26% 22.1M 5s 2024-06-06T03:40:19.119 INFO:tasks.workunit.client.0.smithi040.stderr: 35650K .......... .......... .......... .......... .......... 26% 23.7M 5s 2024-06-06T03:40:19.124 INFO:tasks.workunit.client.0.smithi040.stderr: 35700K .......... .......... .......... .......... .......... 26% 25.3M 5s 2024-06-06T03:40:19.124 INFO:tasks.workunit.client.0.smithi040.stderr: 35750K .......... .......... .......... .......... .......... 26% 40.4M 5s 2024-06-06T03:40:19.125 INFO:tasks.workunit.client.0.smithi040.stderr: 35800K .......... .......... .......... .......... .......... 26% 18.8M 5s 2024-06-06T03:40:19.129 INFO:tasks.workunit.client.0.smithi040.stderr: 35850K .......... .......... .......... .......... .......... 26% 19.3M 5s 2024-06-06T03:40:19.132 INFO:tasks.workunit.client.0.smithi040.stderr: 35900K .......... .......... .......... .......... .......... 26% 21.3M 5s 2024-06-06T03:40:19.132 INFO:tasks.workunit.client.0.smithi040.stderr: 35950K .......... .......... .......... .......... .......... 26% 23.1M 4s 2024-06-06T03:40:19.139 INFO:tasks.workunit.client.0.smithi040.stderr: 36000K .......... .......... .......... .......... .......... 26% 10.6M 5s 2024-06-06T03:40:19.139 INFO:tasks.workunit.client.0.smithi040.stderr: 36050K .......... .......... .......... .......... .......... 26% 61.5M 4s 2024-06-06T03:40:19.139 INFO:tasks.workunit.client.0.smithi040.stderr: 36100K .......... .......... .......... .......... .......... 26% 36.7M 4s 2024-06-06T03:40:19.141 INFO:tasks.workunit.client.0.smithi040.stderr: 36150K .......... .......... .......... .......... .......... 26% 20.5M 4s 2024-06-06T03:40:19.151 INFO:tasks.workunit.client.0.smithi040.stderr: 36200K .......... .......... .......... .......... .......... 26% 26.0M 4s 2024-06-06T03:40:19.151 INFO:tasks.workunit.client.0.smithi040.stderr: 36250K .......... .......... .......... .......... .......... 26% 24.4M 4s 2024-06-06T03:40:19.151 INFO:tasks.workunit.client.0.smithi040.stderr: 36300K .......... .......... .......... .......... .......... 26% 16.2M 4s 2024-06-06T03:40:19.151 INFO:tasks.workunit.client.0.smithi040.stderr: 36350K .......... .......... .......... .......... .......... 26% 21.1M 4s 2024-06-06T03:40:19.153 INFO:tasks.workunit.client.0.smithi040.stderr: 36400K .......... .......... .......... .......... .......... 26% 23.5M 4s 2024-06-06T03:40:19.157 INFO:tasks.workunit.client.0.smithi040.stderr: 36450K .......... .......... .......... .......... .......... 26% 19.5M 4s 2024-06-06T03:40:19.157 INFO:tasks.workunit.client.0.smithi040.stderr: 36500K .......... .......... .......... .......... .......... 26% 31.6M 4s 2024-06-06T03:40:19.159 INFO:tasks.workunit.client.0.smithi040.stderr: 36550K .......... .......... .......... .......... .......... 26% 21.1M 4s 2024-06-06T03:40:19.161 INFO:tasks.workunit.client.0.smithi040.stderr: 36600K .......... .......... .......... .......... .......... 26% 26.0M 4s 2024-06-06T03:40:19.165 INFO:tasks.workunit.client.0.smithi040.stderr: 36650K .......... .......... .......... .......... .......... 27% 23.9M 4s 2024-06-06T03:40:19.165 INFO:tasks.workunit.client.0.smithi040.stderr: 36700K .......... .......... .......... .......... .......... 27% 19.4M 4s 2024-06-06T03:40:19.169 INFO:tasks.workunit.client.0.smithi040.stderr: 36750K .......... .......... .......... .......... .......... 27% 52.3M 4s 2024-06-06T03:40:19.169 INFO:tasks.workunit.client.0.smithi040.stderr: 36800K .......... .......... .......... .......... .......... 27% 25.2M 4s 2024-06-06T03:40:19.170 INFO:tasks.workunit.client.0.smithi040.stderr: 36850K .......... .......... .......... .......... .......... 27% 26.0M 4s 2024-06-06T03:40:19.172 INFO:tasks.workunit.client.0.smithi040.stderr: 36900K .......... .......... .......... .......... .......... 27% 22.8M 4s 2024-06-06T03:40:19.176 INFO:tasks.workunit.client.0.smithi040.stderr: 36950K .......... .......... .......... .......... .......... 27% 21.5M 4s 2024-06-06T03:40:19.177 INFO:tasks.workunit.client.0.smithi040.stderr: 37000K .......... .......... .......... .......... .......... 27% 38.3M 4s 2024-06-06T03:40:19.178 INFO:tasks.workunit.client.0.smithi040.stderr: 37050K .......... .......... .......... .......... .......... 27% 20.5M 4s 2024-06-06T03:40:19.180 INFO:tasks.workunit.client.0.smithi040.stderr: 37100K .......... .......... .......... .......... .......... 27% 24.1M 4s 2024-06-06T03:40:19.182 INFO:tasks.workunit.client.0.smithi040.stderr: 37150K .......... .......... .......... .......... .......... 27% 33.9M 4s 2024-06-06T03:40:19.185 INFO:tasks.workunit.client.0.smithi040.stderr: 37200K .......... .......... .......... .......... .......... 27% 15.8M 4s 2024-06-06T03:40:19.187 INFO:tasks.workunit.client.0.smithi040.stderr: 37250K .......... .......... .......... .......... .......... 27% 25.1M 4s 2024-06-06T03:40:19.189 INFO:tasks.workunit.client.0.smithi040.stderr: 37300K .......... .......... .......... .......... .......... 27% 19.1M 4s 2024-06-06T03:40:19.192 INFO:tasks.workunit.client.0.smithi040.stderr: 37350K .......... .......... .......... .......... .......... 27% 24.5M 4s 2024-06-06T03:40:19.196 INFO:tasks.workunit.client.0.smithi040.stderr: 37400K .......... .......... .......... .......... .......... 27% 29.3M 4s 2024-06-06T03:40:19.196 INFO:tasks.workunit.client.0.smithi040.stderr: 37450K .......... .......... .......... .......... .......... 27% 23.2M 4s 2024-06-06T03:40:19.198 INFO:tasks.workunit.client.0.smithi040.stderr: 37500K .......... .......... .......... .......... .......... 27% 22.7M 4s 2024-06-06T03:40:19.200 INFO:tasks.workunit.client.0.smithi040.stderr: 37550K .......... .......... .......... .......... .......... 27% 17.8M 4s 2024-06-06T03:40:19.204 INFO:tasks.workunit.client.0.smithi040.stderr: 37600K .......... .......... .......... .......... .......... 27% 13.3M 4s 2024-06-06T03:40:19.205 INFO:tasks.workunit.client.0.smithi040.stderr: 37650K .......... .......... .......... .......... .......... 27% 43.0M 4s 2024-06-06T03:40:19.208 INFO:tasks.workunit.client.0.smithi040.stderr: 37700K .......... .......... .......... .......... .......... 27% 23.5M 4s 2024-06-06T03:40:19.209 INFO:tasks.workunit.client.0.smithi040.stderr: 37750K .......... .......... .......... .......... .......... 27% 24.2M 4s 2024-06-06T03:40:19.212 INFO:tasks.workunit.client.0.smithi040.stderr: 37800K .......... .......... .......... .......... .......... 27% 19.4M 4s 2024-06-06T03:40:19.213 INFO:tasks.workunit.client.0.smithi040.stderr: 37850K .......... .......... .......... .......... .......... 27% 25.8M 4s 2024-06-06T03:40:19.215 INFO:tasks.workunit.client.0.smithi040.stderr: 37900K .......... .......... .......... .......... .......... 27% 22.8M 4s 2024-06-06T03:40:19.218 INFO:tasks.workunit.client.0.smithi040.stderr: 37950K .......... .......... .......... .......... .......... 27% 18.6M 4s 2024-06-06T03:40:19.220 INFO:tasks.workunit.client.0.smithi040.stderr: 38000K .......... .......... .......... .......... .......... 28% 32.1M 4s 2024-06-06T03:40:19.222 INFO:tasks.workunit.client.0.smithi040.stderr: 38050K .......... .......... .......... .......... .......... 28% 22.8M 4s 2024-06-06T03:40:19.225 INFO:tasks.workunit.client.0.smithi040.stderr: 38100K .......... .......... .......... .......... .......... 28% 22.4M 4s 2024-06-06T03:40:19.228 INFO:tasks.workunit.client.0.smithi040.stderr: 38150K .......... .......... .......... .......... .......... 28% 28.4M 4s 2024-06-06T03:40:19.228 INFO:tasks.workunit.client.0.smithi040.stderr: 38200K .......... .......... .......... .......... .......... 28% 25.6M 4s 2024-06-06T03:40:19.239 INFO:tasks.workunit.client.0.smithi040.stderr: 38250K .......... .......... .......... .......... .......... 28% 25.2M 4s 2024-06-06T03:40:19.239 INFO:tasks.workunit.client.0.smithi040.stderr: 38300K .......... .......... .......... .......... .......... 28% 23.7M 4s 2024-06-06T03:40:19.239 INFO:tasks.workunit.client.0.smithi040.stderr: 38350K .......... .......... .......... .......... .......... 28% 24.5M 4s 2024-06-06T03:40:19.239 INFO:tasks.workunit.client.0.smithi040.stderr: 38400K .......... .......... .......... .......... .......... 28% 26.4M 4s 2024-06-06T03:40:19.239 INFO:tasks.workunit.client.0.smithi040.stderr: 38450K .......... .......... .......... .......... .......... 28% 27.7M 4s 2024-06-06T03:40:19.239 INFO:tasks.workunit.client.0.smithi040.stderr: 38500K .......... .......... .......... .......... .......... 28% 35.8M 4s 2024-06-06T03:40:19.243 INFO:tasks.workunit.client.0.smithi040.stderr: 38550K .......... .......... .......... .......... .......... 28% 22.4M 4s 2024-06-06T03:40:19.243 INFO:tasks.workunit.client.0.smithi040.stderr: 38600K .......... .......... .......... .......... .......... 28% 19.2M 4s 2024-06-06T03:40:19.246 INFO:tasks.workunit.client.0.smithi040.stderr: 38650K .......... .......... .......... .......... .......... 28% 18.3M 4s 2024-06-06T03:40:19.248 INFO:tasks.workunit.client.0.smithi040.stderr: 38700K .......... .......... .......... .......... .......... 28% 22.9M 4s 2024-06-06T03:40:19.250 INFO:tasks.workunit.client.0.smithi040.stderr: 38750K .......... .......... .......... .......... .......... 28% 46.3M 4s 2024-06-06T03:40:19.251 INFO:tasks.workunit.client.0.smithi040.stderr: 38800K .......... .......... .......... .......... .......... 28% 19.0M 4s 2024-06-06T03:40:19.253 INFO:tasks.workunit.client.0.smithi040.stderr: 38850K .......... .......... .......... .......... .......... 28% 30.4M 4s 2024-06-06T03:40:19.273 INFO:tasks.workunit.client.0.smithi040.stderr: 38900K .......... .......... .......... .......... .......... 28% 25.6M 4s 2024-06-06T03:40:19.273 INFO:tasks.workunit.client.0.smithi040.stderr: 38950K .......... .......... .......... .......... .......... 28% 40.0M 4s 2024-06-06T03:40:19.274 INFO:tasks.workunit.client.0.smithi040.stderr: 39000K .......... .......... .......... .......... .......... 28% 30.6M 4s 2024-06-06T03:40:19.274 INFO:tasks.workunit.client.0.smithi040.stderr: 39050K .......... .......... .......... .......... .......... 28% 27.5M 4s 2024-06-06T03:40:19.274 INFO:tasks.workunit.client.0.smithi040.stderr: 39100K .......... .......... .......... .......... .......... 28% 32.6M 4s 2024-06-06T03:40:19.274 INFO:tasks.workunit.client.0.smithi040.stderr: 39150K .......... .......... .......... .......... .......... 28% 19.7M 4s 2024-06-06T03:40:19.274 INFO:tasks.workunit.client.0.smithi040.stderr: 39200K .......... .......... .......... .......... .......... 28% 34.8M 4s 2024-06-06T03:40:19.274 INFO:tasks.workunit.client.0.smithi040.stderr: 39250K .......... .......... .......... .......... .......... 28% 39.8M 4s 2024-06-06T03:40:19.274 INFO:tasks.workunit.client.0.smithi040.stderr: 39300K .......... .......... .......... .......... .......... 28% 26.3M 4s 2024-06-06T03:40:19.274 INFO:tasks.workunit.client.0.smithi040.stderr: 39350K .......... .......... .......... .......... .......... 29% 27.0M 4s 2024-06-06T03:40:19.274 INFO:tasks.workunit.client.0.smithi040.stderr: 39400K .......... .......... .......... .......... .......... 29% 29.0M 4s 2024-06-06T03:40:19.274 INFO:tasks.workunit.client.0.smithi040.stderr: 39450K .......... .......... .......... .......... .......... 29% 25.4M 4s 2024-06-06T03:40:19.306 INFO:tasks.workunit.client.0.smithi040.stderr: 39500K .......... .......... .......... .......... .......... 29% 28.0M 4s 2024-06-06T03:40:19.306 INFO:tasks.workunit.client.0.smithi040.stderr: 39550K .......... .......... .......... .......... .......... 29% 40.5M 4s 2024-06-06T03:40:19.306 INFO:tasks.workunit.client.0.smithi040.stderr: 39600K .......... .......... .......... .......... .......... 29% 23.8M 4s 2024-06-06T03:40:19.306 INFO:tasks.workunit.client.0.smithi040.stderr: 39650K .......... .......... .......... .......... .......... 29% 41.4M 4s 2024-06-06T03:40:19.306 INFO:tasks.workunit.client.0.smithi040.stderr: 39700K .......... .......... .......... .......... .......... 29% 24.2M 4s 2024-06-06T03:40:19.306 INFO:tasks.workunit.client.0.smithi040.stderr: 39750K .......... .......... .......... .......... .......... 29% 24.2M 4s 2024-06-06T03:40:19.306 INFO:tasks.workunit.client.0.smithi040.stderr: 39800K .......... .......... .......... .......... .......... 29% 42.1M 4s 2024-06-06T03:40:19.306 INFO:tasks.workunit.client.0.smithi040.stderr: 39850K .......... .......... .......... .......... .......... 29% 28.7M 4s 2024-06-06T03:40:19.306 INFO:tasks.workunit.client.0.smithi040.stderr: 39900K .......... .......... .......... .......... .......... 29% 32.1M 4s 2024-06-06T03:40:19.307 INFO:tasks.workunit.client.0.smithi040.stderr: 39950K .......... .......... .......... .......... .......... 29% 24.8M 4s 2024-06-06T03:40:19.307 INFO:tasks.workunit.client.0.smithi040.stderr: 40000K .......... .......... .......... .......... .......... 29% 22.4M 4s 2024-06-06T03:40:19.307 INFO:tasks.workunit.client.0.smithi040.stderr: 40050K .......... .......... .......... .......... .......... 29% 48.6M 4s 2024-06-06T03:40:19.307 INFO:tasks.workunit.client.0.smithi040.stderr: 40100K .......... .......... .......... .......... .......... 29% 25.5M 4s 2024-06-06T03:40:19.307 INFO:tasks.workunit.client.0.smithi040.stderr: 40150K .......... .......... .......... .......... .......... 29% 33.4M 4s 2024-06-06T03:40:19.307 INFO:tasks.workunit.client.0.smithi040.stderr: 40200K .......... .......... .......... .......... .......... 29% 26.1M 4s 2024-06-06T03:40:19.307 INFO:tasks.workunit.client.0.smithi040.stderr: 40250K .......... .......... .......... .......... .......... 29% 23.6M 4s 2024-06-06T03:40:19.307 INFO:tasks.workunit.client.0.smithi040.stderr: 40300K .......... .......... .......... .......... .......... 29% 46.8M 4s 2024-06-06T03:40:19.309 INFO:tasks.workunit.client.0.smithi040.stderr: 40350K .......... .......... .......... .......... .......... 29% 6.47M 4s 2024-06-06T03:40:19.314 INFO:tasks.workunit.client.0.smithi040.stderr: 40400K .......... .......... .......... .......... .......... 29% 31.3M 4s 2024-06-06T03:40:19.315 INFO:tasks.workunit.client.0.smithi040.stderr: 40450K .......... .......... .......... .......... .......... 29% 33.2M 4s 2024-06-06T03:40:19.315 INFO:tasks.workunit.client.0.smithi040.stderr: 40500K .......... .......... .......... .......... .......... 29% 24.2M 4s 2024-06-06T03:40:19.317 INFO:tasks.workunit.client.0.smithi040.stderr: 40550K .......... .......... .......... .......... .......... 29% 38.9M 4s 2024-06-06T03:40:19.328 INFO:tasks.workunit.client.0.smithi040.stderr: 40600K .......... .......... .......... .......... .......... 29% 26.6M 4s 2024-06-06T03:40:19.328 INFO:tasks.workunit.client.0.smithi040.stderr: 40650K .......... .......... .......... .......... .......... 29% 32.7M 4s 2024-06-06T03:40:19.328 INFO:tasks.workunit.client.0.smithi040.stderr: 40700K .......... .......... .......... .......... .......... 30% 26.8M 4s 2024-06-06T03:40:19.328 INFO:tasks.workunit.client.0.smithi040.stderr: 40750K .......... .......... .......... .......... .......... 30% 26.6M 4s 2024-06-06T03:40:19.328 INFO:tasks.workunit.client.0.smithi040.stderr: 40800K .......... .......... .......... .......... .......... 30% 38.2M 4s 2024-06-06T03:40:19.328 INFO:tasks.workunit.client.0.smithi040.stderr: 40850K .......... .......... .......... .......... .......... 30% 23.7M 4s 2024-06-06T03:40:19.328 INFO:tasks.workunit.client.0.smithi040.stderr: 40900K .......... .......... .......... .......... .......... 30% 27.8M 4s 2024-06-06T03:40:19.330 INFO:tasks.workunit.client.0.smithi040.stderr: 40950K .......... .......... .......... .......... .......... 30% 28.7M 4s 2024-06-06T03:40:19.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:19 smithi040 ceph-mon[32220]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-06T03:40:19.370 INFO:tasks.workunit.client.0.smithi040.stderr: 41000K .......... .......... .......... .......... .......... 30% 23.4M 4s 2024-06-06T03:40:19.370 INFO:tasks.workunit.client.0.smithi040.stderr: 41050K .......... .......... .......... .......... .......... 30% 46.8M 4s 2024-06-06T03:40:19.370 INFO:tasks.workunit.client.0.smithi040.stderr: 41100K .......... .......... .......... .......... .......... 30% 29.1M 4s 2024-06-06T03:40:19.370 INFO:tasks.workunit.client.0.smithi040.stderr: 41150K .......... .......... .......... .......... .......... 30% 32.6M 4s 2024-06-06T03:40:19.370 INFO:tasks.workunit.client.0.smithi040.stderr: 41200K .......... .......... .......... .......... .......... 30% 24.3M 4s 2024-06-06T03:40:19.370 INFO:tasks.workunit.client.0.smithi040.stderr: 41250K .......... .......... .......... .......... .......... 30% 25.4M 4s 2024-06-06T03:40:19.370 INFO:tasks.workunit.client.0.smithi040.stderr: 41300K .......... .......... .......... .......... .......... 30% 41.3M 4s 2024-06-06T03:40:19.371 INFO:tasks.workunit.client.0.smithi040.stderr: 41350K .......... .......... .......... .......... .......... 30% 31.3M 4s 2024-06-06T03:40:19.371 INFO:tasks.workunit.client.0.smithi040.stderr: 41400K .......... .......... .......... .......... .......... 30% 29.7M 4s 2024-06-06T03:40:19.371 INFO:tasks.workunit.client.0.smithi040.stderr: 41450K .......... .......... .......... .......... .......... 30% 22.1M 4s 2024-06-06T03:40:19.371 INFO:tasks.workunit.client.0.smithi040.stderr: 41500K .......... .......... .......... .......... .......... 30% 26.9M 4s 2024-06-06T03:40:19.371 INFO:tasks.workunit.client.0.smithi040.stderr: 41550K .......... .......... .......... .......... .......... 30% 47.2M 4s 2024-06-06T03:40:19.371 INFO:tasks.workunit.client.0.smithi040.stderr: 41600K .......... .......... .......... .......... .......... 30% 27.6M 4s 2024-06-06T03:40:19.371 INFO:tasks.workunit.client.0.smithi040.stderr: 41650K .......... .......... .......... .......... .......... 30% 28.7M 4s 2024-06-06T03:40:19.371 INFO:tasks.workunit.client.0.smithi040.stderr: 41700K .......... .......... .......... .......... .......... 30% 26.2M 4s 2024-06-06T03:40:19.371 INFO:tasks.workunit.client.0.smithi040.stderr: 41750K .......... .......... .......... .......... .......... 30% 26.2M 4s 2024-06-06T03:40:19.371 INFO:tasks.workunit.client.0.smithi040.stderr: 41800K .......... .......... .......... .......... .......... 30% 43.0M 4s 2024-06-06T03:40:19.371 INFO:tasks.workunit.client.0.smithi040.stderr: 41850K .......... .......... .......... .......... .......... 30% 28.1M 4s 2024-06-06T03:40:19.371 INFO:tasks.workunit.client.0.smithi040.stderr: 41900K .......... .......... .......... .......... .......... 30% 29.7M 4s 2024-06-06T03:40:19.371 INFO:tasks.workunit.client.0.smithi040.stderr: 41950K .......... .......... .......... .......... .......... 30% 23.6M 4s 2024-06-06T03:40:19.371 INFO:tasks.workunit.client.0.smithi040.stderr: 42000K .......... .......... .......... .......... .......... 30% 34.0M 4s 2024-06-06T03:40:19.372 INFO:tasks.workunit.client.0.smithi040.stderr: 42050K .......... .......... .......... .......... .......... 31% 25.9M 4s 2024-06-06T03:40:19.372 INFO:tasks.workunit.client.0.smithi040.stderr: 42100K .......... .......... .......... .......... .......... 31% 33.6M 4s 2024-06-06T03:40:19.374 INFO:tasks.workunit.client.0.smithi040.stderr: 42150K .......... .......... .......... .......... .......... 31% 8.72M 4s 2024-06-06T03:40:19.384 INFO:tasks.workunit.client.0.smithi040.stderr: 42200K .......... .......... .......... .......... .......... 31% 5.13M 4s 2024-06-06T03:40:19.395 INFO:tasks.workunit.client.0.smithi040.stderr: 42250K .......... .......... .......... .......... .......... 31% 25.0M 4s 2024-06-06T03:40:19.395 INFO:tasks.workunit.client.0.smithi040.stderr: 42300K .......... .......... .......... .......... .......... 31% 41.9M 4s 2024-06-06T03:40:19.395 INFO:tasks.workunit.client.0.smithi040.stderr: 42350K .......... .......... .......... .......... .......... 31% 29.6M 4s 2024-06-06T03:40:19.396 INFO:tasks.workunit.client.0.smithi040.stderr: 42400K .......... .......... .......... .......... .......... 31% 22.1M 4s 2024-06-06T03:40:19.396 INFO:tasks.workunit.client.0.smithi040.stderr: 42450K .......... .......... .......... .......... .......... 31% 35.4M 4s 2024-06-06T03:40:19.396 INFO:tasks.workunit.client.0.smithi040.stderr: 42500K .......... .......... .......... .......... .......... 31% 23.1M 4s 2024-06-06T03:40:19.396 INFO:tasks.workunit.client.0.smithi040.stderr: 42550K .......... .......... .......... .......... .......... 31% 43.9M 4s 2024-06-06T03:40:19.422 INFO:tasks.workunit.client.0.smithi040.stderr: 42600K .......... .......... .......... .......... .......... 31% 28.0M 4s 2024-06-06T03:40:19.423 INFO:tasks.workunit.client.0.smithi040.stderr: 42650K .......... .......... .......... .......... .......... 31% 24.4M 4s 2024-06-06T03:40:19.423 INFO:tasks.workunit.client.0.smithi040.stderr: 42700K .......... .......... .......... .......... .......... 31% 31.3M 4s 2024-06-06T03:40:19.423 INFO:tasks.workunit.client.0.smithi040.stderr: 42750K .......... .......... .......... .......... .......... 31% 29.8M 4s 2024-06-06T03:40:19.423 INFO:tasks.workunit.client.0.smithi040.stderr: 42800K .......... .......... .......... .......... .......... 31% 35.9M 4s 2024-06-06T03:40:19.423 INFO:tasks.workunit.client.0.smithi040.stderr: 42850K .......... .......... .......... .......... .......... 31% 27.1M 4s 2024-06-06T03:40:19.423 INFO:tasks.workunit.client.0.smithi040.stderr: 42900K .......... .......... .......... .......... .......... 31% 24.4M 4s 2024-06-06T03:40:19.423 INFO:tasks.workunit.client.0.smithi040.stderr: 42950K .......... .......... .......... .......... .......... 31% 34.5M 4s 2024-06-06T03:40:19.423 INFO:tasks.workunit.client.0.smithi040.stderr: 43000K .......... .......... .......... .......... .......... 31% 26.5M 4s 2024-06-06T03:40:19.423 INFO:tasks.workunit.client.0.smithi040.stderr: 43050K .......... .......... .......... .......... .......... 31% 39.9M 4s 2024-06-06T03:40:19.423 INFO:tasks.workunit.client.0.smithi040.stderr: 43100K .......... .......... .......... .......... .......... 31% 21.5M 4s 2024-06-06T03:40:19.423 INFO:tasks.workunit.client.0.smithi040.stderr: 43150K .......... .......... .......... .......... .......... 31% 27.5M 4s 2024-06-06T03:40:19.423 INFO:tasks.workunit.client.0.smithi040.stderr: 43200K .......... .......... .......... .......... .......... 31% 38.4M 4s 2024-06-06T03:40:19.423 INFO:tasks.workunit.client.0.smithi040.stderr: 43250K .......... .......... .......... .......... .......... 31% 23.5M 4s 2024-06-06T03:40:19.424 INFO:tasks.workunit.client.0.smithi040.stderr: 43300K .......... .......... .......... .......... .......... 31% 48.1M 4s 2024-06-06T03:40:19.424 INFO:tasks.workunit.client.0.smithi040.stderr: 43350K .......... .......... .......... .......... .......... 31% 21.8M 4s 2024-06-06T03:40:19.425 INFO:tasks.workunit.client.0.smithi040.stderr: 43400K .......... .......... .......... .......... .......... 32% 24.3M 4s 2024-06-06T03:40:19.425 INFO:tasks.workunit.client.0.smithi040.stderr: 43450K .......... .......... .......... .......... .......... 32% 44.2M 4s 2024-06-06T03:40:19.429 INFO:tasks.workunit.client.0.smithi040.stderr: 43500K .......... .......... .......... .......... .......... 32% 22.2M 4s 2024-06-06T03:40:19.429 INFO:tasks.workunit.client.0.smithi040.stderr: 43550K .......... .......... .......... .......... .......... 32% 38.7M 4s 2024-06-06T03:40:19.431 INFO:tasks.workunit.client.0.smithi040.stderr: 43600K .......... .......... .......... .......... .......... 32% 15.9M 4s 2024-06-06T03:40:19.434 INFO:tasks.workunit.client.0.smithi040.stderr: 43650K .......... .......... .......... .......... .......... 32% 23.9M 4s 2024-06-06T03:40:19.439 INFO:tasks.workunit.client.0.smithi040.stderr: 43700K .......... .......... .......... .......... .......... 32% 29.0M 4s 2024-06-06T03:40:19.439 INFO:tasks.workunit.client.0.smithi040.stderr: 43750K .......... .......... .......... .......... .......... 32% 25.1M 4s 2024-06-06T03:40:19.439 INFO:tasks.workunit.client.0.smithi040.stderr: 43800K .......... .......... .......... .......... .......... 32% 45.7M 4s 2024-06-06T03:40:19.451 INFO:tasks.workunit.client.0.smithi040.stderr: 43850K .......... .......... .......... .......... .......... 32% 3.84M 4s 2024-06-06T03:40:19.453 INFO:tasks.workunit.client.0.smithi040.stderr: 43900K .......... .......... .......... .......... .......... 32% 25.9M 4s 2024-06-06T03:40:19.454 INFO:tasks.workunit.client.0.smithi040.stderr: 43950K .......... .......... .......... .......... .......... 32% 37.0M 4s 2024-06-06T03:40:19.457 INFO:tasks.workunit.client.0.smithi040.stderr: 44000K .......... .......... .......... .......... .......... 32% 26.1M 4s 2024-06-06T03:40:19.458 INFO:tasks.workunit.client.0.smithi040.stderr: 44050K .......... .......... .......... .......... .......... 32% 43.8M 4s 2024-06-06T03:40:19.467 INFO:tasks.workunit.client.0.smithi040.stderr: 44100K .......... .......... .......... .......... .......... 32% 25.6M 4s 2024-06-06T03:40:19.467 INFO:tasks.workunit.client.0.smithi040.stderr: 44150K .......... .......... .......... .......... .......... 32% 31.8M 4s 2024-06-06T03:40:19.467 INFO:tasks.workunit.client.0.smithi040.stderr: 44200K .......... .......... .......... .......... .......... 32% 24.5M 4s 2024-06-06T03:40:19.468 INFO:tasks.workunit.client.0.smithi040.stderr: 44250K .......... .......... .......... .......... .......... 32% 28.0M 4s 2024-06-06T03:40:19.468 INFO:tasks.workunit.client.0.smithi040.stderr: 44300K .......... .......... .......... .......... .......... 32% 41.3M 4s 2024-06-06T03:40:19.468 INFO:tasks.workunit.client.0.smithi040.stderr: 44350K .......... .......... .......... .......... .......... 32% 24.8M 4s 2024-06-06T03:40:19.469 INFO:tasks.workunit.client.0.smithi040.stderr: 44400K .......... .......... .......... .......... .......... 32% 25.1M 4s 2024-06-06T03:40:19.470 INFO:tasks.workunit.client.0.smithi040.stderr: 44450K .......... .......... .......... .......... .......... 32% 33.2M 4s 2024-06-06T03:40:19.475 INFO:tasks.workunit.client.0.smithi040.stderr: 44500K .......... .......... .......... .......... .......... 32% 32.9M 4s 2024-06-06T03:40:19.476 INFO:tasks.workunit.client.0.smithi040.stderr: 44550K .......... .......... .......... .......... .......... 32% 31.1M 4s 2024-06-06T03:40:19.476 INFO:tasks.workunit.client.0.smithi040.stderr: 44600K .......... .......... .......... .......... .......... 32% 18.2M 4s 2024-06-06T03:40:19.486 INFO:tasks.workunit.client.0.smithi040.stderr: 44650K .......... .......... .......... .......... .......... 32% 48.6M 4s 2024-06-06T03:40:19.486 INFO:tasks.workunit.client.0.smithi040.stderr: 44700K .......... .......... .......... .......... .......... 32% 32.7M 4s 2024-06-06T03:40:19.486 INFO:tasks.workunit.client.0.smithi040.stderr: 44750K .......... .......... .......... .......... .......... 33% 24.9M 4s 2024-06-06T03:40:19.486 INFO:tasks.workunit.client.0.smithi040.stderr: 44800K .......... .......... .......... .......... .......... 33% 26.8M 4s 2024-06-06T03:40:19.486 INFO:tasks.workunit.client.0.smithi040.stderr: 44850K .......... .......... .......... .......... .......... 33% 36.0M 4s 2024-06-06T03:40:19.487 INFO:tasks.workunit.client.0.smithi040.stderr: 44900K .......... .......... .......... .......... .......... 33% 30.0M 4s 2024-06-06T03:40:19.501 INFO:tasks.workunit.client.0.smithi040.stderr: 44950K .......... .......... .......... .......... .......... 33% 29.8M 4s 2024-06-06T03:40:19.502 INFO:tasks.workunit.client.0.smithi040.stderr: 45000K .......... .......... .......... .......... .......... 33% 24.1M 4s 2024-06-06T03:40:19.502 INFO:tasks.workunit.client.0.smithi040.stderr: 45050K .......... .......... .......... .......... .......... 33% 18.8M 4s 2024-06-06T03:40:19.502 INFO:tasks.workunit.client.0.smithi040.stderr: 45100K .......... .......... .......... .......... .......... 33% 43.1M 4s 2024-06-06T03:40:19.502 INFO:tasks.workunit.client.0.smithi040.stderr: 45150K .......... .......... .......... .......... .......... 33% 29.1M 4s 2024-06-06T03:40:19.502 INFO:tasks.workunit.client.0.smithi040.stderr: 45200K .......... .......... .......... .......... .......... 33% 23.6M 4s 2024-06-06T03:40:19.502 INFO:tasks.workunit.client.0.smithi040.stderr: 45250K .......... .......... .......... .......... .......... 33% 32.3M 4s 2024-06-06T03:40:19.503 INFO:tasks.workunit.client.0.smithi040.stderr: 45300K .......... .......... .......... .......... .......... 33% 10.3M 4s 2024-06-06T03:40:19.505 INFO:tasks.workunit.client.0.smithi040.stderr: 45350K .......... .......... .......... .......... .......... 33% 17.6M 4s 2024-06-06T03:40:19.509 INFO:tasks.workunit.client.0.smithi040.stderr: 45400K .......... .......... .......... .......... .......... 33% 14.7M 4s 2024-06-06T03:40:19.512 INFO:tasks.workunit.client.0.smithi040.stderr: 45450K .......... .......... .......... .......... .......... 33% 14.1M 4s 2024-06-06T03:40:19.516 INFO:tasks.workunit.client.0.smithi040.stderr: 45500K .......... .......... .......... .......... .......... 33% 13.3M 4s 2024-06-06T03:40:19.520 INFO:tasks.workunit.client.0.smithi040.stderr: 45550K .......... .......... .......... .......... .......... 33% 12.4M 4s 2024-06-06T03:40:19.521 INFO:tasks.workunit.client.0.smithi040.stderr: 45600K .......... .......... .......... .......... .......... 33% 45.8M 4s 2024-06-06T03:40:19.524 INFO:tasks.workunit.client.0.smithi040.stderr: 45650K .......... .......... .......... .......... .......... 33% 24.7M 4s 2024-06-06T03:40:19.525 INFO:tasks.workunit.client.0.smithi040.stderr: 45700K .......... .......... .......... .......... .......... 33% 24.2M 4s 2024-06-06T03:40:19.529 INFO:tasks.workunit.client.0.smithi040.stderr: 45750K .......... .......... .......... .......... .......... 33% 36.6M 4s 2024-06-06T03:40:19.529 INFO:tasks.workunit.client.0.smithi040.stderr: 45800K .......... .......... .......... .......... .......... 33% 23.9M 4s 2024-06-06T03:40:19.530 INFO:tasks.workunit.client.0.smithi040.stderr: 45850K .......... .......... .......... .......... .......... 33% 45.8M 4s 2024-06-06T03:40:19.535 INFO:tasks.workunit.client.0.smithi040.stderr: 45900K .......... .......... .......... .......... .......... 33% 24.8M 4s 2024-06-06T03:40:19.535 INFO:tasks.workunit.client.0.smithi040.stderr: 45950K .......... .......... .......... .......... .......... 33% 35.7M 4s 2024-06-06T03:40:19.535 INFO:tasks.workunit.client.0.smithi040.stderr: 46000K .......... .......... .......... .......... .......... 33% 29.2M 4s 2024-06-06T03:40:19.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:19 smithi131 ceph-mon[36687]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-06T03:40:19.549 INFO:tasks.workunit.client.0.smithi040.stderr: 46050K .......... .......... .......... .......... .......... 33% 20.4M 4s 2024-06-06T03:40:19.549 INFO:tasks.workunit.client.0.smithi040.stderr: 46100K .......... .......... .......... .......... .......... 33% 34.8M 4s 2024-06-06T03:40:19.549 INFO:tasks.workunit.client.0.smithi040.stderr: 46150K .......... .......... .......... .......... .......... 34% 22.9M 4s 2024-06-06T03:40:19.549 INFO:tasks.workunit.client.0.smithi040.stderr: 46200K .......... .......... .......... .......... .......... 34% 23.9M 4s 2024-06-06T03:40:19.549 INFO:tasks.workunit.client.0.smithi040.stderr: 46250K .......... .......... .......... .......... .......... 34% 34.0M 4s 2024-06-06T03:40:19.549 INFO:tasks.workunit.client.0.smithi040.stderr: 46300K .......... .......... .......... .......... .......... 34% 35.0M 4s 2024-06-06T03:40:19.549 INFO:tasks.workunit.client.0.smithi040.stderr: 46350K .......... .......... .......... .......... .......... 34% 39.5M 4s 2024-06-06T03:40:19.550 INFO:tasks.workunit.client.0.smithi040.stderr: 46400K .......... .......... .......... .......... .......... 34% 22.3M 4s 2024-06-06T03:40:19.570 INFO:tasks.workunit.client.0.smithi040.stderr: 46450K .......... .......... .......... .......... .......... 34% 18.2M 4s 2024-06-06T03:40:19.570 INFO:tasks.workunit.client.0.smithi040.stderr: 46500K .......... .......... .......... .......... .......... 34% 36.4M 4s 2024-06-06T03:40:19.570 INFO:tasks.workunit.client.0.smithi040.stderr: 46550K .......... .......... .......... .......... .......... 34% 25.0M 4s 2024-06-06T03:40:19.570 INFO:tasks.workunit.client.0.smithi040.stderr: 46600K .......... .......... .......... .......... .......... 34% 44.3M 4s 2024-06-06T03:40:19.571 INFO:tasks.workunit.client.0.smithi040.stderr: 46650K .......... .......... .......... .......... .......... 34% 25.7M 4s 2024-06-06T03:40:19.571 INFO:tasks.workunit.client.0.smithi040.stderr: 46700K .......... .......... .......... .......... .......... 34% 18.4M 4s 2024-06-06T03:40:19.571 INFO:tasks.workunit.client.0.smithi040.stderr: 46750K .......... .......... .......... .......... .......... 34% 26.5M 4s 2024-06-06T03:40:19.571 INFO:tasks.workunit.client.0.smithi040.stderr: 46800K .......... .......... .......... .......... .......... 34% 15.9M 4s 2024-06-06T03:40:19.571 INFO:tasks.workunit.client.0.smithi040.stderr: 46850K .......... .......... .......... .......... .......... 34% 43.2M 4s 2024-06-06T03:40:19.571 INFO:tasks.workunit.client.0.smithi040.stderr: 46900K .......... .......... .......... .......... .......... 34% 13.9M 4s 2024-06-06T03:40:19.575 INFO:tasks.workunit.client.0.smithi040.stderr: 46950K .......... .......... .......... .......... .......... 34% 15.1M 4s 2024-06-06T03:40:19.576 INFO:tasks.workunit.client.0.smithi040.stderr: 47000K .......... .......... .......... .......... .......... 34% 19.2M 4s 2024-06-06T03:40:19.580 INFO:tasks.workunit.client.0.smithi040.stderr: 47050K .......... .......... .......... .......... .......... 34% 12.0M 4s 2024-06-06T03:40:19.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:19 smithi104 ceph-mon[36569]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-06T03:40:19.673 INFO:tasks.workunit.client.0.smithi040.stderr: 47100K .......... .......... .......... .......... .......... 34% 41.0M 4s 2024-06-06T03:40:19.673 INFO:tasks.workunit.client.0.smithi040.stderr: 47150K .......... .......... .......... .......... .......... 34% 27.9M 4s 2024-06-06T03:40:19.673 INFO:tasks.workunit.client.0.smithi040.stderr: 47200K .......... .......... .......... .......... .......... 34% 23.8M 4s 2024-06-06T03:40:19.674 INFO:tasks.workunit.client.0.smithi040.stderr: 47250K .......... .......... .......... .......... .......... 34% 30.6M 4s 2024-06-06T03:40:19.674 INFO:tasks.workunit.client.0.smithi040.stderr: 47300K .......... .......... .......... .......... .......... 34% 25.1M 4s 2024-06-06T03:40:19.674 INFO:tasks.workunit.client.0.smithi040.stderr: 47350K .......... .......... .......... .......... .......... 34% 36.5M 4s 2024-06-06T03:40:19.674 INFO:tasks.workunit.client.0.smithi040.stderr: 47400K .......... .......... .......... .......... .......... 34% 32.1M 4s 2024-06-06T03:40:19.674 INFO:tasks.workunit.client.0.smithi040.stderr: 47450K .......... .......... .......... .......... .......... 34% 22.4M 4s 2024-06-06T03:40:19.674 INFO:tasks.workunit.client.0.smithi040.stderr: 47500K .......... .......... .......... .......... .......... 35% 34.9M 4s 2024-06-06T03:40:19.674 INFO:tasks.workunit.client.0.smithi040.stderr: 47550K .......... .......... .......... .......... .......... 35% 26.0M 4s 2024-06-06T03:40:19.674 INFO:tasks.workunit.client.0.smithi040.stderr: 47600K .......... .......... .......... .......... .......... 35% 38.7M 4s 2024-06-06T03:40:19.674 INFO:tasks.workunit.client.0.smithi040.stderr: 47650K .......... .......... .......... .......... .......... 35% 29.4M 4s 2024-06-06T03:40:19.674 INFO:tasks.workunit.client.0.smithi040.stderr: 47700K .......... .......... .......... .......... .......... 35% 24.5M 4s 2024-06-06T03:40:19.674 INFO:tasks.workunit.client.0.smithi040.stderr: 47750K .......... .......... .......... .......... .......... 35% 30.6M 4s 2024-06-06T03:40:19.674 INFO:tasks.workunit.client.0.smithi040.stderr: 47800K .......... .......... .......... .......... .......... 35% 24.3M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 47850K .......... .......... .......... .......... .......... 35% 41.4M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 47900K .......... .......... .......... .......... .......... 35% 25.0M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 47950K .......... .......... .......... .......... .......... 35% 26.0M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 48000K .......... .......... .......... .......... .......... 35% 35.7M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 48050K .......... .......... .......... .......... .......... 35% 25.6M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 48100K .......... .......... .......... .......... .......... 35% 36.1M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 48150K .......... .......... .......... .......... .......... 35% 29.7M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 48200K .......... .......... .......... .......... .......... 35% 28.5M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 48250K .......... .......... .......... .......... .......... 35% 28.8M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 48300K .......... .......... .......... .......... .......... 35% 22.9M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 48350K .......... .......... .......... .......... .......... 35% 44.1M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 48400K .......... .......... .......... .......... .......... 35% 27.5M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 48450K .......... .......... .......... .......... .......... 35% 25.5M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 48500K .......... .......... .......... .......... .......... 35% 26.0M 4s 2024-06-06T03:40:19.675 INFO:tasks.workunit.client.0.smithi040.stderr: 48550K .......... .......... .......... .......... .......... 35% 30.3M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 48600K .......... .......... .......... .......... .......... 35% 38.0M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 48650K .......... .......... .......... .......... .......... 35% 21.8M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 48700K .......... .......... .......... .......... .......... 35% 24.4M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 48750K .......... .......... .......... .......... .......... 35% 34.1M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 48800K .......... .......... .......... .......... .......... 35% 25.5M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 48850K .......... .......... .......... .......... .......... 36% 39.8M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 48900K .......... .......... .......... .......... .......... 36% 28.5M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 48950K .......... .......... .......... .......... .......... 36% 21.5M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 49000K .......... .......... .......... .......... .......... 36% 28.7M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 49050K .......... .......... .......... .......... .......... 36% 23.4M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 49100K .......... .......... .......... .......... .......... 36% 40.4M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 49150K .......... .......... .......... .......... .......... 36% 25.6M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 49200K .......... .......... .......... .......... .......... 36% 26.8M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 49250K .......... .......... .......... .......... .......... 36% 32.5M 4s 2024-06-06T03:40:19.676 INFO:tasks.workunit.client.0.smithi040.stderr: 49300K .......... .......... .......... .......... .......... 36% 34.1M 4s 2024-06-06T03:40:19.677 INFO:tasks.workunit.client.0.smithi040.stderr: 49350K .......... .......... .......... .......... .......... 36% 35.7M 4s 2024-06-06T03:40:19.677 INFO:tasks.workunit.client.0.smithi040.stderr: 49400K .......... .......... .......... .......... .......... 36% 28.3M 4s 2024-06-06T03:40:19.677 INFO:tasks.workunit.client.0.smithi040.stderr: 49450K .......... .......... .......... .......... .......... 36% 25.1M 4s 2024-06-06T03:40:19.677 INFO:tasks.workunit.client.0.smithi040.stderr: 49500K .......... .......... .......... .......... .......... 36% 38.4M 4s 2024-06-06T03:40:19.677 INFO:tasks.workunit.client.0.smithi040.stderr: 49550K .......... .......... .......... .......... .......... 36% 21.8M 4s 2024-06-06T03:40:19.677 INFO:tasks.workunit.client.0.smithi040.stderr: 49600K .......... .......... .......... .......... .......... 36% 46.8M 4s 2024-06-06T03:40:19.677 INFO:tasks.workunit.client.0.smithi040.stderr: 49650K .......... .......... .......... .......... .......... 36% 21.3M 4s 2024-06-06T03:40:19.677 INFO:tasks.workunit.client.0.smithi040.stderr: 49700K .......... .......... .......... .......... .......... 36% 22.2M 4s 2024-06-06T03:40:19.677 INFO:tasks.workunit.client.0.smithi040.stderr: 49750K .......... .......... .......... .......... .......... 36% 40.5M 4s 2024-06-06T03:40:19.677 INFO:tasks.workunit.client.0.smithi040.stderr: 49800K .......... .......... .......... .......... .......... 36% 22.4M 4s 2024-06-06T03:40:19.687 INFO:tasks.workunit.client.0.smithi040.stderr: 49850K .......... .......... .......... .......... .......... 36% 26.1M 4s 2024-06-06T03:40:19.687 INFO:tasks.workunit.client.0.smithi040.stderr: 49900K .......... .......... .......... .......... .......... 36% 43.2M 4s 2024-06-06T03:40:19.687 INFO:tasks.workunit.client.0.smithi040.stderr: 49950K .......... .......... .......... .......... .......... 36% 26.4M 4s 2024-06-06T03:40:19.687 INFO:tasks.workunit.client.0.smithi040.stderr: 50000K .......... .......... .......... .......... .......... 36% 40.1M 4s 2024-06-06T03:40:19.687 INFO:tasks.workunit.client.0.smithi040.stderr: 50050K .......... .......... .......... .......... .......... 36% 25.7M 4s 2024-06-06T03:40:19.687 INFO:tasks.workunit.client.0.smithi040.stderr: 50100K .......... .......... .......... .......... .......... 36% 24.4M 4s 2024-06-06T03:40:19.687 INFO:tasks.workunit.client.0.smithi040.stderr: 50150K .......... .......... .......... .......... .......... 36% 42.8M 4s 2024-06-06T03:40:19.702 INFO:tasks.workunit.client.0.smithi040.stderr: 50200K .......... .......... .......... .......... .......... 37% 8.44M 4s 2024-06-06T03:40:19.702 INFO:tasks.workunit.client.0.smithi040.stderr: 50250K .......... .......... .......... .......... .......... 37% 28.3M 4s 2024-06-06T03:40:19.702 INFO:tasks.workunit.client.0.smithi040.stderr: 50300K .......... .......... .......... .......... .......... 37% 25.2M 4s 2024-06-06T03:40:19.702 INFO:tasks.workunit.client.0.smithi040.stderr: 50350K .......... .......... .......... .......... .......... 37% 28.9M 4s 2024-06-06T03:40:19.702 INFO:tasks.workunit.client.0.smithi040.stderr: 50400K .......... .......... .......... .......... .......... 37% 31.4M 4s 2024-06-06T03:40:19.703 INFO:tasks.workunit.client.0.smithi040.stderr: 50450K .......... .......... .......... .......... .......... 37% 21.4M 4s 2024-06-06T03:40:19.703 INFO:tasks.workunit.client.0.smithi040.stderr: 50500K .......... .......... .......... .......... .......... 37% 41.5M 4s 2024-06-06T03:40:19.704 INFO:tasks.workunit.client.0.smithi040.stderr: 50550K .......... .......... .......... .......... .......... 37% 23.3M 4s 2024-06-06T03:40:19.747 INFO:tasks.workunit.client.0.smithi040.stderr: 50600K .......... .......... .......... .......... .......... 37% 26.0M 4s 2024-06-06T03:40:19.747 INFO:tasks.workunit.client.0.smithi040.stderr: 50650K .......... .......... .......... .......... .......... 37% 42.4M 4s 2024-06-06T03:40:19.747 INFO:tasks.workunit.client.0.smithi040.stderr: 50700K .......... .......... .......... .......... .......... 37% 24.1M 4s 2024-06-06T03:40:19.747 INFO:tasks.workunit.client.0.smithi040.stderr: 50750K .......... .......... .......... .......... .......... 37% 29.1M 4s 2024-06-06T03:40:19.747 INFO:tasks.workunit.client.0.smithi040.stderr: 50800K .......... .......... .......... .......... .......... 37% 30.8M 4s 2024-06-06T03:40:19.747 INFO:tasks.workunit.client.0.smithi040.stderr: 50850K .......... .......... .......... .......... .......... 37% 31.0M 4s 2024-06-06T03:40:19.747 INFO:tasks.workunit.client.0.smithi040.stderr: 50900K .......... .......... .......... .......... .......... 37% 34.7M 4s 2024-06-06T03:40:19.747 INFO:tasks.workunit.client.0.smithi040.stderr: 50950K .......... .......... .......... .......... .......... 37% 27.9M 4s 2024-06-06T03:40:19.747 INFO:tasks.workunit.client.0.smithi040.stderr: 51000K .......... .......... .......... .......... .......... 37% 32.9M 4s 2024-06-06T03:40:19.747 INFO:tasks.workunit.client.0.smithi040.stderr: 51050K .......... .......... .......... .......... .......... 37% 22.2M 4s 2024-06-06T03:40:19.747 INFO:tasks.workunit.client.0.smithi040.stderr: 51100K .......... .......... .......... .......... .......... 37% 22.4M 4s 2024-06-06T03:40:19.747 INFO:tasks.workunit.client.0.smithi040.stderr: 51150K .......... .......... .......... .......... .......... 37% 52.0M 4s 2024-06-06T03:40:19.748 INFO:tasks.workunit.client.0.smithi040.stderr: 51200K .......... .......... .......... .......... .......... 37% 33.5M 4s 2024-06-06T03:40:19.748 INFO:tasks.workunit.client.0.smithi040.stderr: 51250K .......... .......... .......... .......... .......... 37% 30.0M 4s 2024-06-06T03:40:19.748 INFO:tasks.workunit.client.0.smithi040.stderr: 51300K .......... .......... .......... .......... .......... 37% 23.3M 4s 2024-06-06T03:40:19.748 INFO:tasks.workunit.client.0.smithi040.stderr: 51350K .......... .......... .......... .......... .......... 37% 24.4M 4s 2024-06-06T03:40:19.748 INFO:tasks.workunit.client.0.smithi040.stderr: 51400K .......... .......... .......... .......... .......... 37% 51.5M 4s 2024-06-06T03:40:19.748 INFO:tasks.workunit.client.0.smithi040.stderr: 51450K .......... .......... .......... .......... .......... 37% 23.5M 4s 2024-06-06T03:40:19.748 INFO:tasks.workunit.client.0.smithi040.stderr: 51500K .......... .......... .......... .......... .......... 37% 35.0M 4s 2024-06-06T03:40:19.748 INFO:tasks.workunit.client.0.smithi040.stderr: 51550K .......... .......... .......... .......... .......... 38% 24.4M 4s 2024-06-06T03:40:19.748 INFO:tasks.workunit.client.0.smithi040.stderr: 51600K .......... .......... .......... .......... .......... 38% 25.9M 4s 2024-06-06T03:40:19.748 INFO:tasks.workunit.client.0.smithi040.stderr: 51650K .......... .......... .......... .......... .......... 38% 31.9M 4s 2024-06-06T03:40:19.748 INFO:tasks.workunit.client.0.smithi040.stderr: 51700K .......... .......... .......... .......... .......... 38% 22.5M 4s 2024-06-06T03:40:19.748 INFO:tasks.workunit.client.0.smithi040.stderr: 51750K .......... .......... .......... .......... .......... 38% 29.3M 4s 2024-06-06T03:40:19.806 INFO:tasks.workunit.client.0.smithi040.stderr: 51800K .......... .......... .......... .......... .......... 38% 9.43M 4s 2024-06-06T03:40:19.807 INFO:tasks.workunit.client.0.smithi040.stderr: 51850K .......... .......... .......... .......... .......... 38% 25.2M 4s 2024-06-06T03:40:19.807 INFO:tasks.workunit.client.0.smithi040.stderr: 51900K .......... .......... .......... .......... .......... 38% 50.3M 4s 2024-06-06T03:40:19.807 INFO:tasks.workunit.client.0.smithi040.stderr: 51950K .......... .......... .......... .......... .......... 38% 26.8M 4s 2024-06-06T03:40:19.807 INFO:tasks.workunit.client.0.smithi040.stderr: 52000K .......... .......... .......... .......... .......... 38% 22.9M 4s 2024-06-06T03:40:19.807 INFO:tasks.workunit.client.0.smithi040.stderr: 52050K .......... .......... .......... .......... .......... 38% 35.9M 4s 2024-06-06T03:40:19.807 INFO:tasks.workunit.client.0.smithi040.stderr: 52100K .......... .......... .......... .......... .......... 38% 21.7M 4s 2024-06-06T03:40:19.807 INFO:tasks.workunit.client.0.smithi040.stderr: 52150K .......... .......... .......... .......... .......... 38% 43.8M 4s 2024-06-06T03:40:19.807 INFO:tasks.workunit.client.0.smithi040.stderr: 52200K .......... .......... .......... .......... .......... 38% 31.2M 4s 2024-06-06T03:40:19.807 INFO:tasks.workunit.client.0.smithi040.stderr: 52250K .......... .......... .......... .......... .......... 38% 30.6M 4s 2024-06-06T03:40:19.807 INFO:tasks.workunit.client.0.smithi040.stderr: 52300K .......... .......... .......... .......... .......... 38% 34.3M 4s 2024-06-06T03:40:19.807 INFO:tasks.workunit.client.0.smithi040.stderr: 52350K .......... .......... .......... .......... .......... 38% 19.5M 4s 2024-06-06T03:40:19.807 INFO:tasks.workunit.client.0.smithi040.stderr: 52400K .......... .......... .......... .......... .......... 38% 41.3M 4s 2024-06-06T03:40:19.808 INFO:tasks.workunit.client.0.smithi040.stderr: 52450K .......... .......... .......... .......... .......... 38% 31.3M 4s 2024-06-06T03:40:19.808 INFO:tasks.workunit.client.0.smithi040.stderr: 52500K .......... .......... .......... .......... .......... 38% 29.4M 4s 2024-06-06T03:40:19.808 INFO:tasks.workunit.client.0.smithi040.stderr: 52550K .......... .......... .......... .......... .......... 38% 27.9M 4s 2024-06-06T03:40:19.808 INFO:tasks.workunit.client.0.smithi040.stderr: 52600K .......... .......... .......... .......... .......... 38% 23.5M 4s 2024-06-06T03:40:19.808 INFO:tasks.workunit.client.0.smithi040.stderr: 52650K .......... .......... .......... .......... .......... 38% 43.2M 4s 2024-06-06T03:40:19.808 INFO:tasks.workunit.client.0.smithi040.stderr: 52700K .......... .......... .......... .......... .......... 38% 22.2M 4s 2024-06-06T03:40:19.808 INFO:tasks.workunit.client.0.smithi040.stderr: 52750K .......... .......... .......... .......... .......... 38% 28.7M 4s 2024-06-06T03:40:19.808 INFO:tasks.workunit.client.0.smithi040.stderr: 52800K .......... .......... .......... .......... .......... 38% 33.4M 4s 2024-06-06T03:40:19.808 INFO:tasks.workunit.client.0.smithi040.stderr: 52850K .......... .......... .......... .......... .......... 38% 28.1M 4s 2024-06-06T03:40:19.808 INFO:tasks.workunit.client.0.smithi040.stderr: 52900K .......... .......... .......... .......... .......... 39% 35.3M 4s 2024-06-06T03:40:19.808 INFO:tasks.workunit.client.0.smithi040.stderr: 52950K .......... .......... .......... .......... .......... 39% 24.3M 4s 2024-06-06T03:40:19.808 INFO:tasks.workunit.client.0.smithi040.stderr: 53000K .......... .......... .......... .......... .......... 39% 23.4M 4s 2024-06-06T03:40:19.808 INFO:tasks.workunit.client.0.smithi040.stderr: 53050K .......... .......... .......... .......... .......... 39% 42.2M 4s 2024-06-06T03:40:19.808 INFO:tasks.workunit.client.0.smithi040.stderr: 53100K .......... .......... .......... .......... .......... 39% 28.6M 4s 2024-06-06T03:40:19.809 INFO:tasks.workunit.client.0.smithi040.stderr: 53150K .......... .......... .......... .......... .......... 39% 33.1M 4s 2024-06-06T03:40:19.809 INFO:tasks.workunit.client.0.smithi040.stderr: 53200K .......... .......... .......... .......... .......... 39% 24.7M 4s 2024-06-06T03:40:19.809 INFO:tasks.workunit.client.0.smithi040.stderr: 53250K .......... .......... .......... .......... .......... 39% 25.6M 4s 2024-06-06T03:40:19.809 INFO:tasks.workunit.client.0.smithi040.stderr: 53300K .......... .......... .......... .......... .......... 39% 41.3M 4s 2024-06-06T03:40:19.809 INFO:tasks.workunit.client.0.smithi040.stderr: 53350K .......... .......... .......... .......... .......... 39% 23.0M 4s 2024-06-06T03:40:19.809 INFO:tasks.workunit.client.0.smithi040.stderr: 53400K .......... .......... .......... .......... .......... 39% 23.5M 4s 2024-06-06T03:40:19.816 INFO:tasks.workunit.client.0.smithi040.stderr: 53450K .......... .......... .......... .......... .......... 39% 3.86M 4s 2024-06-06T03:40:19.819 INFO:tasks.workunit.client.0.smithi040.stderr: 53500K .......... .......... .......... .......... .......... 39% 32.5M 4s 2024-06-06T03:40:19.819 INFO:tasks.workunit.client.0.smithi040.stderr: 53550K .......... .......... .......... .......... .......... 39% 36.7M 4s 2024-06-06T03:40:19.826 INFO:tasks.workunit.client.0.smithi040.stderr: 53600K .......... .......... .......... .......... .......... 39% 22.1M 4s 2024-06-06T03:40:19.826 INFO:tasks.workunit.client.0.smithi040.stderr: 53650K .......... .......... .......... .......... .......... 39% 36.9M 4s 2024-06-06T03:40:19.826 INFO:tasks.workunit.client.0.smithi040.stderr: 53700K .......... .......... .......... .......... .......... 39% 24.9M 4s 2024-06-06T03:40:19.827 INFO:tasks.workunit.client.0.smithi040.stderr: 53750K .......... .......... .......... .......... .......... 39% 24.7M 4s 2024-06-06T03:40:19.830 INFO:tasks.workunit.client.0.smithi040.stderr: 53800K .......... .......... .......... .......... .......... 39% 47.6M 4s 2024-06-06T03:40:19.830 INFO:tasks.workunit.client.0.smithi040.stderr: 53850K .......... .......... .......... .......... .......... 39% 20.7M 4s 2024-06-06T03:40:19.871 INFO:tasks.workunit.client.0.smithi040.stderr: 53900K .......... .......... .......... .......... .......... 39% 47.0M 4s 2024-06-06T03:40:19.871 INFO:tasks.workunit.client.0.smithi040.stderr: 53950K .......... .......... .......... .......... .......... 39% 22.8M 4s 2024-06-06T03:40:19.871 INFO:tasks.workunit.client.0.smithi040.stderr: 54000K .......... .......... .......... .......... .......... 39% 27.0M 4s 2024-06-06T03:40:19.871 INFO:tasks.workunit.client.0.smithi040.stderr: 54050K .......... .......... .......... .......... .......... 39% 34.2M 4s 2024-06-06T03:40:19.871 INFO:tasks.workunit.client.0.smithi040.stderr: 54100K .......... .......... .......... .......... .......... 39% 27.9M 4s 2024-06-06T03:40:19.871 INFO:tasks.workunit.client.0.smithi040.stderr: 54150K .......... .......... .......... .......... .......... 39% 35.8M 4s 2024-06-06T03:40:19.871 INFO:tasks.workunit.client.0.smithi040.stderr: 54200K .......... .......... .......... .......... .......... 39% 25.7M 4s 2024-06-06T03:40:19.871 INFO:tasks.workunit.client.0.smithi040.stderr: 54250K .......... .......... .......... .......... .......... 39% 23.6M 4s 2024-06-06T03:40:19.871 INFO:tasks.workunit.client.0.smithi040.stderr: 54300K .......... .......... .......... .......... .......... 40% 37.9M 3s 2024-06-06T03:40:19.871 INFO:tasks.workunit.client.0.smithi040.stderr: 54350K .......... .......... .......... .......... .......... 40% 25.2M 3s 2024-06-06T03:40:19.872 INFO:tasks.workunit.client.0.smithi040.stderr: 54400K .......... .......... .......... .......... .......... 40% 44.1M 3s 2024-06-06T03:40:19.872 INFO:tasks.workunit.client.0.smithi040.stderr: 54450K .......... .......... .......... .......... .......... 40% 22.5M 3s 2024-06-06T03:40:19.872 INFO:tasks.workunit.client.0.smithi040.stderr: 54500K .......... .......... .......... .......... .......... 40% 25.7M 3s 2024-06-06T03:40:19.872 INFO:tasks.workunit.client.0.smithi040.stderr: 54550K .......... .......... .......... .......... .......... 40% 42.5M 3s 2024-06-06T03:40:19.872 INFO:tasks.workunit.client.0.smithi040.stderr: 54600K .......... .......... .......... .......... .......... 40% 25.3M 3s 2024-06-06T03:40:19.872 INFO:tasks.workunit.client.0.smithi040.stderr: 54650K .......... .......... .......... .......... .......... 40% 36.7M 3s 2024-06-06T03:40:19.872 INFO:tasks.workunit.client.0.smithi040.stderr: 54700K .......... .......... .......... .......... .......... 40% 25.4M 3s 2024-06-06T03:40:19.872 INFO:tasks.workunit.client.0.smithi040.stderr: 54750K .......... .......... .......... .......... .......... 40% 29.9M 3s 2024-06-06T03:40:19.872 INFO:tasks.workunit.client.0.smithi040.stderr: 54800K .......... .......... .......... .......... .......... 40% 33.6M 3s 2024-06-06T03:40:19.872 INFO:tasks.workunit.client.0.smithi040.stderr: 54850K .......... .......... .......... .......... .......... 40% 26.4M 3s 2024-06-06T03:40:19.872 INFO:tasks.workunit.client.0.smithi040.stderr: 54900K .......... .......... .......... .......... .......... 40% 23.9M 3s 2024-06-06T03:40:19.872 INFO:tasks.workunit.client.0.smithi040.stderr: 54950K .......... .......... .......... .......... .......... 40% 12.4M 3s 2024-06-06T03:40:19.873 INFO:tasks.workunit.client.0.smithi040.stderr: 55000K .......... .......... .......... .......... .......... 40% 11.2M 3s 2024-06-06T03:40:19.875 INFO:tasks.workunit.client.0.smithi040.stderr: 55050K .......... .......... .......... .......... .......... 40% 32.9M 3s 2024-06-06T03:40:19.879 INFO:tasks.workunit.client.0.smithi040.stderr: 55100K .......... .......... .......... .......... .......... 40% 12.0M 3s 2024-06-06T03:40:19.884 INFO:tasks.workunit.client.0.smithi040.stderr: 55150K .......... .......... .......... .......... .......... 40% 9.99M 3s 2024-06-06T03:40:19.888 INFO:tasks.workunit.client.0.smithi040.stderr: 55200K .......... .......... .......... .......... .......... 40% 17.5M 3s 2024-06-06T03:40:19.888 INFO:tasks.workunit.client.0.smithi040.stderr: 55250K .......... .......... .......... .......... .......... 40% 27.8M 3s 2024-06-06T03:40:19.889 INFO:tasks.workunit.client.0.smithi040.stderr: 55300K .......... .......... .......... .......... .......... 40% 36.6M 3s 2024-06-06T03:40:19.897 INFO:tasks.workunit.client.0.smithi040.stderr: 55350K .......... .......... .......... .......... .......... 40% 22.2M 3s 2024-06-06T03:40:19.897 INFO:tasks.workunit.client.0.smithi040.stderr: 55400K .......... .......... .......... .......... .......... 40% 25.7M 3s 2024-06-06T03:40:19.897 INFO:tasks.workunit.client.0.smithi040.stderr: 55450K .......... .......... .......... .......... .......... 40% 46.2M 3s 2024-06-06T03:40:19.897 INFO:tasks.workunit.client.0.smithi040.stderr: 55500K .......... .......... .......... .......... .......... 40% 23.9M 3s 2024-06-06T03:40:19.907 INFO:tasks.workunit.client.0.smithi040.stderr: 55550K .......... .......... .......... .......... .......... 40% 37.8M 3s 2024-06-06T03:40:19.907 INFO:tasks.workunit.client.0.smithi040.stderr: 55600K .......... .......... .......... .......... .......... 40% 24.3M 3s 2024-06-06T03:40:19.907 INFO:tasks.workunit.client.0.smithi040.stderr: 55650K .......... .......... .......... .......... .......... 41% 25.1M 3s 2024-06-06T03:40:19.907 INFO:tasks.workunit.client.0.smithi040.stderr: 55700K .......... .......... .......... .......... .......... 41% 44.4M 3s 2024-06-06T03:40:19.907 INFO:tasks.workunit.client.0.smithi040.stderr: 55750K .......... .......... .......... .......... .......... 41% 23.8M 3s 2024-06-06T03:40:19.907 INFO:tasks.workunit.client.0.smithi040.stderr: 55800K .......... .......... .......... .......... .......... 41% 32.4M 3s 2024-06-06T03:40:19.917 INFO:tasks.workunit.client.0.smithi040.stderr: 55850K .......... .......... .......... .......... .......... 41% 29.4M 3s 2024-06-06T03:40:19.917 INFO:tasks.workunit.client.0.smithi040.stderr: 55900K .......... .......... .......... .......... .......... 41% 29.3M 3s 2024-06-06T03:40:19.917 INFO:tasks.workunit.client.0.smithi040.stderr: 55950K .......... .......... .......... .......... .......... 41% 33.0M 3s 2024-06-06T03:40:19.917 INFO:tasks.workunit.client.0.smithi040.stderr: 56000K .......... .......... .......... .......... .......... 41% 30.0M 3s 2024-06-06T03:40:19.917 INFO:tasks.workunit.client.0.smithi040.stderr: 56050K .......... .......... .......... .......... .......... 41% 21.4M 3s 2024-06-06T03:40:19.917 INFO:tasks.workunit.client.0.smithi040.stderr: 56100K .......... .......... .......... .......... .......... 41% 29.0M 3s 2024-06-06T03:40:19.920 INFO:tasks.workunit.client.0.smithi040.stderr: 56150K .......... .......... .......... .......... .......... 41% 22.6M 3s 2024-06-06T03:40:19.920 INFO:tasks.workunit.client.0.smithi040.stderr: 56200K .......... .......... .......... .......... .......... 41% 35.7M 3s 2024-06-06T03:40:19.930 INFO:tasks.workunit.client.0.smithi040.stderr: 56250K .......... .......... .......... .......... .......... 41% 25.6M 3s 2024-06-06T03:40:19.930 INFO:tasks.workunit.client.0.smithi040.stderr: 56300K .......... .......... .......... .......... .......... 41% 30.6M 3s 2024-06-06T03:40:19.930 INFO:tasks.workunit.client.0.smithi040.stderr: 56350K .......... .......... .......... .......... .......... 41% 29.6M 3s 2024-06-06T03:40:19.930 INFO:tasks.workunit.client.0.smithi040.stderr: 56400K .......... .......... .......... .......... .......... 41% 19.0M 3s 2024-06-06T03:40:19.930 INFO:tasks.workunit.client.0.smithi040.stderr: 56450K .......... .......... .......... .......... .......... 41% 42.4M 3s 2024-06-06T03:40:19.932 INFO:tasks.workunit.client.0.smithi040.stderr: 56500K .......... .......... .......... .......... .......... 41% 15.8M 3s 2024-06-06T03:40:19.936 INFO:tasks.workunit.client.0.smithi040.stderr: 56550K .......... .......... .......... .......... .......... 41% 20.9M 3s 2024-06-06T03:40:19.937 INFO:tasks.workunit.client.0.smithi040.stderr: 56600K .......... .......... .......... .......... .......... 41% 18.5M 3s 2024-06-06T03:40:19.940 INFO:tasks.workunit.client.0.smithi040.stderr: 56650K .......... .......... .......... .......... .......... 41% 16.1M 3s 2024-06-06T03:40:19.942 INFO:tasks.workunit.client.0.smithi040.stderr: 56700K .......... .......... .......... .......... .......... 41% 21.6M 3s 2024-06-06T03:40:19.947 INFO:tasks.workunit.client.0.smithi040.stderr: 56750K .......... .......... .......... .......... .......... 41% 12.7M 3s 2024-06-06T03:40:19.951 INFO:tasks.workunit.client.0.smithi040.stderr: 56800K .......... .......... .......... .......... .......... 41% 10.4M 3s 2024-06-06T03:40:19.954 INFO:tasks.workunit.client.0.smithi040.stderr: 56850K .......... .......... .......... .......... .......... 41% 17.2M 3s 2024-06-06T03:40:19.958 INFO:tasks.workunit.client.0.smithi040.stderr: 56900K .......... .......... .......... .......... .......... 41% 26.2M 3s 2024-06-06T03:40:19.958 INFO:tasks.workunit.client.0.smithi040.stderr: 56950K .......... .......... .......... .......... .......... 41% 39.9M 3s 2024-06-06T03:40:20.004 INFO:tasks.workunit.client.0.smithi040.stderr: 57000K .......... .......... .......... .......... .......... 42% 26.0M 3s 2024-06-06T03:40:20.004 INFO:tasks.workunit.client.0.smithi040.stderr: 57050K .......... .......... .......... .......... .......... 42% 23.7M 3s 2024-06-06T03:40:20.004 INFO:tasks.workunit.client.0.smithi040.stderr: 57100K .......... .......... .......... .......... .......... 42% 37.3M 3s 2024-06-06T03:40:20.005 INFO:tasks.workunit.client.0.smithi040.stderr: 57150K .......... .......... .......... .......... .......... 42% 24.8M 3s 2024-06-06T03:40:20.005 INFO:tasks.workunit.client.0.smithi040.stderr: 57200K .......... .......... .......... .......... .......... 42% 38.6M 3s 2024-06-06T03:40:20.005 INFO:tasks.workunit.client.0.smithi040.stderr: 57250K .......... .......... .......... .......... .......... 42% 25.9M 3s 2024-06-06T03:40:20.005 INFO:tasks.workunit.client.0.smithi040.stderr: 57300K .......... .......... .......... .......... .......... 42% 24.5M 3s 2024-06-06T03:40:20.005 INFO:tasks.workunit.client.0.smithi040.stderr: 57350K .......... .......... .......... .......... .......... 42% 37.0M 3s 2024-06-06T03:40:20.005 INFO:tasks.workunit.client.0.smithi040.stderr: 57400K .......... .......... .......... .......... .......... 42% 23.6M 3s 2024-06-06T03:40:20.005 INFO:tasks.workunit.client.0.smithi040.stderr: 57450K .......... .......... .......... .......... .......... 42% 44.4M 3s 2024-06-06T03:40:20.005 INFO:tasks.workunit.client.0.smithi040.stderr: 57500K .......... .......... .......... .......... .......... 42% 21.6M 3s 2024-06-06T03:40:20.005 INFO:tasks.workunit.client.0.smithi040.stderr: 57550K .......... .......... .......... .......... .......... 42% 25.4M 3s 2024-06-06T03:40:20.005 INFO:tasks.workunit.client.0.smithi040.stderr: 57600K .......... .......... .......... .......... .......... 42% 36.3M 3s 2024-06-06T03:40:20.005 INFO:tasks.workunit.client.0.smithi040.stderr: 57650K .......... .......... .......... .......... .......... 42% 24.0M 3s 2024-06-06T03:40:20.005 INFO:tasks.workunit.client.0.smithi040.stderr: 57700K .......... .......... .......... .......... .......... 42% 39.9M 3s 2024-06-06T03:40:20.005 INFO:tasks.workunit.client.0.smithi040.stderr: 57750K .......... .......... .......... .......... .......... 42% 23.8M 3s 2024-06-06T03:40:20.005 INFO:tasks.workunit.client.0.smithi040.stderr: 57800K .......... .......... .......... .......... .......... 42% 24.7M 3s 2024-06-06T03:40:20.006 INFO:tasks.workunit.client.0.smithi040.stderr: 57850K .......... .......... .......... .......... .......... 42% 32.4M 3s 2024-06-06T03:40:20.006 INFO:tasks.workunit.client.0.smithi040.stderr: 57900K .......... .......... .......... .......... .......... 42% 27.6M 3s 2024-06-06T03:40:20.006 INFO:tasks.workunit.client.0.smithi040.stderr: 57950K .......... .......... .......... .......... .......... 42% 32.9M 3s 2024-06-06T03:40:20.006 INFO:tasks.workunit.client.0.smithi040.stderr: 58000K .......... .......... .......... .......... .......... 42% 18.0M 3s 2024-06-06T03:40:20.006 INFO:tasks.workunit.client.0.smithi040.stderr: 58050K .......... .......... .......... .......... .......... 42% 15.7M 3s 2024-06-06T03:40:20.006 INFO:tasks.workunit.client.0.smithi040.stderr: 58100K .......... .......... .......... .......... .......... 42% 33.3M 3s 2024-06-06T03:40:20.006 INFO:tasks.workunit.client.0.smithi040.stderr: 58150K .......... .......... .......... .......... .......... 42% 16.2M 3s 2024-06-06T03:40:20.006 INFO:tasks.workunit.client.0.smithi040.stderr: 58200K .......... .......... .......... .......... .......... 42% 30.9M 3s 2024-06-06T03:40:20.012 INFO:tasks.workunit.client.0.smithi040.stderr: 58250K .......... .......... .......... .......... .......... 42% 15.1M 3s 2024-06-06T03:40:20.012 INFO:tasks.workunit.client.0.smithi040.stderr: 58300K .......... .......... .......... .......... .......... 42% 22.2M 3s 2024-06-06T03:40:20.012 INFO:tasks.workunit.client.0.smithi040.stderr: 58350K .......... .......... .......... .......... .......... 43% 13.3M 3s 2024-06-06T03:40:20.016 INFO:tasks.workunit.client.0.smithi040.stderr: 58400K .......... .......... .......... .......... .......... 43% 31.6M 3s 2024-06-06T03:40:20.018 INFO:tasks.workunit.client.0.smithi040.stderr: 58450K .......... .......... .......... .......... .......... 43% 13.3M 3s 2024-06-06T03:40:20.021 INFO:tasks.workunit.client.0.smithi040.stderr: 58500K .......... .......... .......... .......... .......... 43% 16.5M 3s 2024-06-06T03:40:20.026 INFO:tasks.workunit.client.0.smithi040.stderr: 58550K .......... .......... .......... .......... .......... 43% 14.9M 3s 2024-06-06T03:40:20.026 INFO:tasks.workunit.client.0.smithi040.stderr: 58600K .......... .......... .......... .......... .......... 43% 39.7M 3s 2024-06-06T03:40:20.027 INFO:tasks.workunit.client.0.smithi040.stderr: 58650K .......... .......... .......... .......... .......... 43% 28.0M 3s 2024-06-06T03:40:20.029 INFO:tasks.workunit.client.0.smithi040.stderr: 58700K .......... .......... .......... .......... .......... 43% 29.5M 3s 2024-06-06T03:40:20.044 INFO:tasks.workunit.client.0.smithi040.stderr: 58750K .......... .......... .......... .......... .......... 43% 26.3M 3s 2024-06-06T03:40:20.044 INFO:tasks.workunit.client.0.smithi040.stderr: 58800K .......... .......... .......... .......... .......... 43% 26.9M 3s 2024-06-06T03:40:20.045 INFO:tasks.workunit.client.0.smithi040.stderr: 58850K .......... .......... .......... .......... .......... 43% 39.2M 3s 2024-06-06T03:40:20.045 INFO:tasks.workunit.client.0.smithi040.stderr: 58900K .......... .......... .......... .......... .......... 43% 30.5M 3s 2024-06-06T03:40:20.045 INFO:tasks.workunit.client.0.smithi040.stderr: 58950K .......... .......... .......... .......... .......... 43% 25.6M 3s 2024-06-06T03:40:20.045 INFO:tasks.workunit.client.0.smithi040.stderr: 59000K .......... .......... .......... .......... .......... 43% 22.2M 3s 2024-06-06T03:40:20.045 INFO:tasks.workunit.client.0.smithi040.stderr: 59050K .......... .......... .......... .......... .......... 43% 31.6M 3s 2024-06-06T03:40:20.045 INFO:tasks.workunit.client.0.smithi040.stderr: 59100K .......... .......... .......... .......... .......... 43% 29.4M 3s 2024-06-06T03:40:20.045 INFO:tasks.workunit.client.0.smithi040.stderr: 59150K .......... .......... .......... .......... .......... 43% 23.7M 3s 2024-06-06T03:40:20.068 INFO:tasks.workunit.client.0.smithi040.stderr: 59200K .......... .......... .......... .......... .......... 43% 47.0M 3s 2024-06-06T03:40:20.068 INFO:tasks.workunit.client.0.smithi040.stderr: 59250K .......... .......... .......... .......... .......... 43% 22.0M 3s 2024-06-06T03:40:20.068 INFO:tasks.workunit.client.0.smithi040.stderr: 59300K .......... .......... .......... .......... .......... 43% 25.3M 3s 2024-06-06T03:40:20.068 INFO:tasks.workunit.client.0.smithi040.stderr: 59350K .......... .......... .......... .......... .......... 43% 35.2M 3s 2024-06-06T03:40:20.068 INFO:tasks.workunit.client.0.smithi040.stderr: 59400K .......... .......... .......... .......... .......... 43% 32.3M 3s 2024-06-06T03:40:20.068 INFO:tasks.workunit.client.0.smithi040.stderr: 59450K .......... .......... .......... .......... .......... 43% 28.3M 3s 2024-06-06T03:40:20.068 INFO:tasks.workunit.client.0.smithi040.stderr: 59500K .......... .......... .......... .......... .......... 43% 22.3M 3s 2024-06-06T03:40:20.069 INFO:tasks.workunit.client.0.smithi040.stderr: 59550K .......... .......... .......... .......... .......... 43% 23.4M 3s 2024-06-06T03:40:20.069 INFO:tasks.workunit.client.0.smithi040.stderr: 59600K .......... .......... .......... .......... .......... 43% 21.7M 3s 2024-06-06T03:40:20.069 INFO:tasks.workunit.client.0.smithi040.stderr: 59650K .......... .......... .......... .......... .......... 43% 28.2M 3s 2024-06-06T03:40:20.069 INFO:tasks.workunit.client.0.smithi040.stderr: 59700K .......... .......... .......... .......... .......... 44% 27.4M 3s 2024-06-06T03:40:20.069 INFO:tasks.workunit.client.0.smithi040.stderr: 59750K .......... .......... .......... .......... .......... 44% 16.7M 3s 2024-06-06T03:40:20.071 INFO:tasks.workunit.client.0.smithi040.stderr: 59800K .......... .......... .......... .......... .......... 44% 19.7M 3s 2024-06-06T03:40:20.073 INFO:tasks.workunit.client.0.smithi040.stderr: 59850K .......... .......... .......... .......... .......... 44% 16.5M 3s 2024-06-06T03:40:20.075 INFO:tasks.workunit.client.0.smithi040.stderr: 59900K .......... .......... .......... .......... .......... 44% 28.2M 3s 2024-06-06T03:40:20.078 INFO:tasks.workunit.client.0.smithi040.stderr: 59950K .......... .......... .......... .......... .......... 44% 22.7M 3s 2024-06-06T03:40:20.080 INFO:tasks.workunit.client.0.smithi040.stderr: 60000K .......... .......... .......... .......... .......... 44% 17.3M 3s 2024-06-06T03:40:20.084 INFO:tasks.workunit.client.0.smithi040.stderr: 60050K .......... .......... .......... .......... .......... 44% 11.9M 3s 2024-06-06T03:40:20.089 INFO:tasks.workunit.client.0.smithi040.stderr: 60100K .......... .......... .......... .......... .......... 44% 14.8M 3s 2024-06-06T03:40:20.089 INFO:tasks.workunit.client.0.smithi040.stderr: 60150K .......... .......... .......... .......... .......... 44% 40.8M 3s 2024-06-06T03:40:20.107 INFO:tasks.workunit.client.0.smithi040.stderr: 60200K .......... .......... .......... .......... .......... 44% 15.3M 3s 2024-06-06T03:40:20.108 INFO:tasks.workunit.client.0.smithi040.stderr: 60250K .......... .......... .......... .......... .......... 44% 26.9M 3s 2024-06-06T03:40:20.108 INFO:tasks.workunit.client.0.smithi040.stderr: 60300K .......... .......... .......... .......... .......... 44% 24.5M 3s 2024-06-06T03:40:20.108 INFO:tasks.workunit.client.0.smithi040.stderr: 60350K .......... .......... .......... .......... .......... 44% 29.9M 3s 2024-06-06T03:40:20.108 INFO:tasks.workunit.client.0.smithi040.stderr: 60400K .......... .......... .......... .......... .......... 44% 30.4M 3s 2024-06-06T03:40:20.108 INFO:tasks.workunit.client.0.smithi040.stderr: 60450K .......... .......... .......... .......... .......... 44% 27.1M 3s 2024-06-06T03:40:20.108 INFO:tasks.workunit.client.0.smithi040.stderr: 60500K .......... .......... .......... .......... .......... 44% 44.5M 3s 2024-06-06T03:40:20.108 INFO:tasks.workunit.client.0.smithi040.stderr: 60550K .......... .......... .......... .......... .......... 44% 21.1M 3s 2024-06-06T03:40:20.108 INFO:tasks.workunit.client.0.smithi040.stderr: 60600K .......... .......... .......... .......... .......... 44% 39.6M 3s 2024-06-06T03:40:20.108 INFO:tasks.workunit.client.0.smithi040.stderr: 60650K .......... .......... .......... .......... .......... 44% 25.8M 3s 2024-06-06T03:40:20.114 INFO:tasks.workunit.client.0.smithi040.stderr: 60700K .......... .......... .......... .......... .......... 44% 26.2M 3s 2024-06-06T03:40:20.114 INFO:tasks.workunit.client.0.smithi040.stderr: 60750K .......... .......... .......... .......... .......... 44% 40.6M 3s 2024-06-06T03:40:20.114 INFO:tasks.workunit.client.0.smithi040.stderr: 60800K .......... .......... .......... .......... .......... 44% 28.6M 3s 2024-06-06T03:40:20.115 INFO:tasks.workunit.client.0.smithi040.stderr: 60850K .......... .......... .......... .......... .......... 44% 30.3M 3s 2024-06-06T03:40:20.117 INFO:tasks.workunit.client.0.smithi040.stderr: 60900K .......... .......... .......... .......... .......... 44% 21.6M 3s 2024-06-06T03:40:20.117 INFO:tasks.workunit.client.0.smithi040.stderr: 60950K .......... .......... .......... .......... .......... 44% 26.2M 3s 2024-06-06T03:40:20.122 INFO:tasks.workunit.client.0.smithi040.stderr: 61000K .......... .......... .......... .......... .......... 44% 36.7M 3s 2024-06-06T03:40:20.122 INFO:tasks.workunit.client.0.smithi040.stderr: 61050K .......... .......... .......... .......... .......... 45% 27.5M 3s 2024-06-06T03:40:20.122 INFO:tasks.workunit.client.0.smithi040.stderr: 61100K .......... .......... .......... .......... .......... 45% 26.5M 3s 2024-06-06T03:40:20.127 INFO:tasks.workunit.client.0.smithi040.stderr: 61150K .......... .......... .......... .......... .......... 45% 18.6M 3s 2024-06-06T03:40:20.127 INFO:tasks.workunit.client.0.smithi040.stderr: 61200K .......... .......... .......... .......... .......... 45% 19.2M 3s 2024-06-06T03:40:20.128 INFO:tasks.workunit.client.0.smithi040.stderr: 61250K .......... .......... .......... .......... .......... 45% 35.6M 3s 2024-06-06T03:40:20.131 INFO:tasks.workunit.client.0.smithi040.stderr: 61300K .......... .......... .......... .......... .......... 45% 20.1M 3s 2024-06-06T03:40:20.134 INFO:tasks.workunit.client.0.smithi040.stderr: 61350K .......... .......... .......... .......... .......... 45% 12.5M 3s 2024-06-06T03:40:20.137 INFO:tasks.workunit.client.0.smithi040.stderr: 61400K .......... .......... .......... .......... .......... 45% 33.7M 3s 2024-06-06T03:40:20.138 INFO:tasks.workunit.client.0.smithi040.stderr: 61450K .......... .......... .......... .......... .......... 45% 23.3M 3s 2024-06-06T03:40:20.140 INFO:tasks.workunit.client.0.smithi040.stderr: 61500K .......... .......... .......... .......... .......... 45% 42.1M 3s 2024-06-06T03:40:20.143 INFO:tasks.workunit.client.0.smithi040.stderr: 61550K .......... .......... .......... .......... .......... 45% 16.1M 3s 2024-06-06T03:40:20.146 INFO:tasks.workunit.client.0.smithi040.stderr: 61600K .......... .......... .......... .......... .......... 45% 17.8M 3s 2024-06-06T03:40:20.148 INFO:tasks.workunit.client.0.smithi040.stderr: 61650K .......... .......... .......... .......... .......... 45% 23.6M 3s 2024-06-06T03:40:20.153 INFO:tasks.workunit.client.0.smithi040.stderr: 61700K .......... .......... .......... .......... .......... 45% 13.5M 3s 2024-06-06T03:40:20.153 INFO:tasks.workunit.client.0.smithi040.stderr: 61750K .......... .......... .......... .......... .......... 45% 30.8M 3s 2024-06-06T03:40:20.156 INFO:tasks.workunit.client.0.smithi040.stderr: 61800K .......... .......... .......... .......... .......... 45% 13.4M 3s 2024-06-06T03:40:20.161 INFO:tasks.workunit.client.0.smithi040.stderr: 61850K .......... .......... .......... .......... .......... 45% 21.5M 3s 2024-06-06T03:40:20.161 INFO:tasks.workunit.client.0.smithi040.stderr: 61900K .......... .......... .......... .......... .......... 45% 31.4M 3s 2024-06-06T03:40:20.166 INFO:tasks.workunit.client.0.smithi040.stderr: 61950K .......... .......... .......... .......... .......... 45% 22.9M 3s 2024-06-06T03:40:20.166 INFO:tasks.workunit.client.0.smithi040.stderr: 62000K .......... .......... .......... .......... .......... 45% 45.0M 3s 2024-06-06T03:40:20.166 INFO:tasks.workunit.client.0.smithi040.stderr: 62050K .......... .......... .......... .......... .......... 45% 24.7M 3s 2024-06-06T03:40:20.168 INFO:tasks.workunit.client.0.smithi040.stderr: 62100K .......... .......... .......... .......... .......... 45% 30.3M 3s 2024-06-06T03:40:20.169 INFO:tasks.workunit.client.0.smithi040.stderr: 62150K .......... .......... .......... .......... .......... 45% 25.9M 3s 2024-06-06T03:40:20.172 INFO:tasks.workunit.client.0.smithi040.stderr: 62200K .......... .......... .......... .......... .......... 45% 24.9M 3s 2024-06-06T03:40:20.172 INFO:tasks.workunit.client.0.smithi040.stderr: 62250K .......... .......... .......... .......... .......... 45% 42.0M 3s 2024-06-06T03:40:20.183 INFO:tasks.workunit.client.0.smithi040.stderr: 62300K .......... .......... .......... .......... .......... 45% 27.0M 3s 2024-06-06T03:40:20.183 INFO:tasks.workunit.client.0.smithi040.stderr: 62350K .......... .......... .......... .......... .......... 45% 30.5M 3s 2024-06-06T03:40:20.183 INFO:tasks.workunit.client.0.smithi040.stderr: 62400K .......... .......... .......... .......... .......... 46% 24.9M 3s 2024-06-06T03:40:20.183 INFO:tasks.workunit.client.0.smithi040.stderr: 62450K .......... .......... .......... .......... .......... 46% 23.2M 3s 2024-06-06T03:40:20.183 INFO:tasks.workunit.client.0.smithi040.stderr: 62500K .......... .......... .......... .......... .......... 46% 50.6M 3s 2024-06-06T03:40:20.183 INFO:tasks.workunit.client.0.smithi040.stderr: 62550K .......... .......... .......... .......... .......... 46% 29.5M 3s 2024-06-06T03:40:20.186 INFO:tasks.workunit.client.0.smithi040.stderr: 62600K .......... .......... .......... .......... .......... 46% 18.4M 3s 2024-06-06T03:40:20.186 INFO:tasks.workunit.client.0.smithi040.stderr: 62650K .......... .......... .......... .......... .......... 46% 38.0M 3s 2024-06-06T03:40:20.190 INFO:tasks.workunit.client.0.smithi040.stderr: 62700K .......... .......... .......... .......... .......... 46% 15.9M 3s 2024-06-06T03:40:20.190 INFO:tasks.workunit.client.0.smithi040.stderr: 62750K .......... .......... .......... .......... .......... 46% 44.3M 3s 2024-06-06T03:40:20.195 INFO:tasks.workunit.client.0.smithi040.stderr: 62800K .......... .......... .......... .......... .......... 46% 22.6M 3s 2024-06-06T03:40:20.195 INFO:tasks.workunit.client.0.smithi040.stderr: 62850K .......... .......... .......... .......... .......... 46% 26.6M 3s 2024-06-06T03:40:20.196 INFO:tasks.workunit.client.0.smithi040.stderr: 62900K .......... .......... .......... .......... .......... 46% 21.3M 3s 2024-06-06T03:40:20.198 INFO:tasks.workunit.client.0.smithi040.stderr: 62950K .......... .......... .......... .......... .......... 46% 22.6M 3s 2024-06-06T03:40:20.201 INFO:tasks.workunit.client.0.smithi040.stderr: 63000K .......... .......... .......... .......... .......... 46% 20.1M 3s 2024-06-06T03:40:20.203 INFO:tasks.workunit.client.0.smithi040.stderr: 63050K .......... .......... .......... .......... .......... 46% 22.4M 3s 2024-06-06T03:40:20.205 INFO:tasks.workunit.client.0.smithi040.stderr: 63100K .......... .......... .......... .......... .......... 46% 24.6M 3s 2024-06-06T03:40:20.210 INFO:tasks.workunit.client.0.smithi040.stderr: 63150K .......... .......... .......... .......... .......... 46% 11.0M 3s 2024-06-06T03:40:20.211 INFO:tasks.workunit.client.0.smithi040.stderr: 63200K .......... .......... .......... .......... .......... 46% 41.4M 3s 2024-06-06T03:40:20.214 INFO:tasks.workunit.client.0.smithi040.stderr: 63250K .......... .......... .......... .......... .......... 46% 27.0M 3s 2024-06-06T03:40:20.216 INFO:tasks.workunit.client.0.smithi040.stderr: 63300K .......... .......... .......... .......... .......... 46% 15.8M 3s 2024-06-06T03:40:20.219 INFO:tasks.workunit.client.0.smithi040.stderr: 63350K .......... .......... .......... .......... .......... 46% 21.4M 3s 2024-06-06T03:40:20.221 INFO:tasks.workunit.client.0.smithi040.stderr: 63400K .......... .......... .......... .......... .......... 46% 24.1M 3s 2024-06-06T03:40:20.223 INFO:tasks.workunit.client.0.smithi040.stderr: 63450K .......... .......... .......... .......... .......... 46% 15.9M 3s 2024-06-06T03:40:20.251 INFO:tasks.workunit.client.0.smithi040.stderr: 63500K .......... .......... .......... .......... .......... 46% 20.9M 3s 2024-06-06T03:40:20.252 INFO:tasks.workunit.client.0.smithi040.stderr: 63550K .......... .......... .......... .......... .......... 46% 23.5M 3s 2024-06-06T03:40:20.252 INFO:tasks.workunit.client.0.smithi040.stderr: 63600K .......... .......... .......... .......... .......... 46% 30.4M 3s 2024-06-06T03:40:20.252 INFO:tasks.workunit.client.0.smithi040.stderr: 63650K .......... .......... .......... .......... .......... 46% 30.6M 3s 2024-06-06T03:40:20.252 INFO:tasks.workunit.client.0.smithi040.stderr: 63700K .......... .......... .......... .......... .......... 46% 25.7M 3s 2024-06-06T03:40:20.252 INFO:tasks.workunit.client.0.smithi040.stderr: 63750K .......... .......... .......... .......... .......... 46% 36.3M 3s 2024-06-06T03:40:20.252 INFO:tasks.workunit.client.0.smithi040.stderr: 63800K .......... .......... .......... .......... .......... 47% 26.0M 3s 2024-06-06T03:40:20.252 INFO:tasks.workunit.client.0.smithi040.stderr: 63850K .......... .......... .......... .......... .......... 47% 25.2M 3s 2024-06-06T03:40:20.252 INFO:tasks.workunit.client.0.smithi040.stderr: 63900K .......... .......... .......... .......... .......... 47% 30.9M 3s 2024-06-06T03:40:20.252 INFO:tasks.workunit.client.0.smithi040.stderr: 63950K .......... .......... .......... .......... .......... 47% 24.1M 3s 2024-06-06T03:40:20.252 INFO:tasks.workunit.client.0.smithi040.stderr: 64000K .......... .......... .......... .......... .......... 47% 46.6M 3s 2024-06-06T03:40:20.252 INFO:tasks.workunit.client.0.smithi040.stderr: 64050K .......... .......... .......... .......... .......... 47% 22.5M 3s 2024-06-06T03:40:20.252 INFO:tasks.workunit.client.0.smithi040.stderr: 64100K .......... .......... .......... .......... .......... 47% 23.5M 3s 2024-06-06T03:40:20.253 INFO:tasks.workunit.client.0.smithi040.stderr: 64150K .......... .......... .......... .......... .......... 47% 35.7M 3s 2024-06-06T03:40:20.253 INFO:tasks.workunit.client.0.smithi040.stderr: 64200K .......... .......... .......... .......... .......... 47% 20.2M 3s 2024-06-06T03:40:20.253 INFO:tasks.workunit.client.0.smithi040.stderr: 64250K .......... .......... .......... .......... .......... 47% 46.0M 3s 2024-06-06T03:40:20.255 INFO:tasks.workunit.client.0.smithi040.stderr: 64300K .......... .......... .......... .......... .......... 47% 14.9M 3s 2024-06-06T03:40:20.262 INFO:tasks.workunit.client.0.smithi040.stderr: 64350K .......... .......... .......... .......... .......... 47% 27.7M 3s 2024-06-06T03:40:20.262 INFO:tasks.workunit.client.0.smithi040.stderr: 64400K .......... .......... .......... .......... .......... 47% 40.0M 3s 2024-06-06T03:40:20.262 INFO:tasks.workunit.client.0.smithi040.stderr: 64450K .......... .......... .......... .......... .......... 47% 19.4M 3s 2024-06-06T03:40:20.262 INFO:tasks.workunit.client.0.smithi040.stderr: 64500K .......... .......... .......... .......... .......... 47% 45.7M 3s 2024-06-06T03:40:20.266 INFO:tasks.workunit.client.0.smithi040.stderr: 64550K .......... .......... .......... .......... .......... 47% 16.9M 3s 2024-06-06T03:40:20.268 INFO:tasks.workunit.client.0.smithi040.stderr: 64600K .......... .......... .......... .......... .......... 47% 14.2M 3s 2024-06-06T03:40:20.272 INFO:tasks.workunit.client.0.smithi040.stderr: 64650K .......... .......... .......... .......... .......... 47% 37.7M 3s 2024-06-06T03:40:20.272 INFO:tasks.workunit.client.0.smithi040.stderr: 64700K .......... .......... .......... .......... .......... 47% 22.7M 3s 2024-06-06T03:40:20.273 INFO:tasks.workunit.client.0.smithi040.stderr: 64750K .......... .......... .......... .......... .......... 47% 28.7M 3s 2024-06-06T03:40:20.276 INFO:tasks.workunit.client.0.smithi040.stderr: 64800K .......... .......... .......... .......... .......... 47% 18.9M 3s 2024-06-06T03:40:20.279 INFO:tasks.workunit.client.0.smithi040.stderr: 64850K .......... .......... .......... .......... .......... 47% 14.5M 3s 2024-06-06T03:40:20.283 INFO:tasks.workunit.client.0.smithi040.stderr: 64900K .......... .......... .......... .......... .......... 47% 24.6M 3s 2024-06-06T03:40:20.283 INFO:tasks.workunit.client.0.smithi040.stderr: 64950K .......... .......... .......... .......... .......... 47% 24.3M 3s 2024-06-06T03:40:20.285 INFO:tasks.workunit.client.0.smithi040.stderr: 65000K .......... .......... .......... .......... .......... 47% 22.1M 3s 2024-06-06T03:40:20.288 INFO:tasks.workunit.client.0.smithi040.stderr: 65050K .......... .......... .......... .......... .......... 47% 21.8M 3s 2024-06-06T03:40:20.290 INFO:tasks.workunit.client.0.smithi040.stderr: 65100K .......... .......... .......... .......... .......... 47% 16.1M 3s 2024-06-06T03:40:20.294 INFO:tasks.workunit.client.0.smithi040.stderr: 65150K .......... .......... .......... .......... .......... 48% 13.8M 3s 2024-06-06T03:40:20.297 INFO:tasks.workunit.client.0.smithi040.stderr: 65200K .......... .......... .......... .......... .......... 48% 30.0M 3s 2024-06-06T03:40:20.297 INFO:tasks.workunit.client.0.smithi040.stderr: 65250K .......... .......... .......... .......... .......... 48% 36.6M 3s 2024-06-06T03:40:20.300 INFO:tasks.workunit.client.0.smithi040.stderr: 65300K .......... .......... .......... .......... .......... 48% 27.0M 3s 2024-06-06T03:40:20.302 INFO:tasks.workunit.client.0.smithi040.stderr: 65350K .......... .......... .......... .......... .......... 48% 16.1M 3s 2024-06-06T03:40:20.303 INFO:tasks.workunit.client.0.smithi040.stderr: 65400K .......... .......... .......... .......... .......... 48% 49.6M 3s 2024-06-06T03:40:20.304 INFO:tasks.workunit.client.0.smithi040.stderr: 65450K .......... .......... .......... .......... .......... 48% 35.9M 3s 2024-06-06T03:40:20.306 INFO:tasks.workunit.client.0.smithi040.stderr: 65500K .......... .......... .......... .......... .......... 48% 32.7M 3s 2024-06-06T03:40:20.307 INFO:tasks.workunit.client.0.smithi040.stderr: 65550K .......... .......... .......... .......... .......... 48% 27.4M 3s 2024-06-06T03:40:20.310 INFO:tasks.workunit.client.0.smithi040.stderr: 65600K .......... .......... .......... .......... .......... 48% 21.0M 3s 2024-06-06T03:40:20.314 INFO:tasks.workunit.client.0.smithi040.stderr: 65650K .......... .......... .......... .......... .......... 48% 28.1M 3s 2024-06-06T03:40:20.314 INFO:tasks.workunit.client.0.smithi040.stderr: 65700K .......... .......... .......... .......... .......... 48% 62.0M 3s 2024-06-06T03:40:20.314 INFO:tasks.workunit.client.0.smithi040.stderr: 65750K .......... .......... .......... .......... .......... 48% 26.2M 3s 2024-06-06T03:40:20.319 INFO:tasks.workunit.client.0.smithi040.stderr: 65800K .......... .......... .......... .......... .......... 48% 21.0M 3s 2024-06-06T03:40:20.319 INFO:tasks.workunit.client.0.smithi040.stderr: 65850K .......... .......... .......... .......... .......... 48% 35.9M 3s 2024-06-06T03:40:20.321 INFO:tasks.workunit.client.0.smithi040.stderr: 65900K .......... .......... .......... .......... .......... 48% 15.4M 3s 2024-06-06T03:40:20.329 INFO:tasks.workunit.client.0.smithi040.stderr: 65950K .......... .......... .......... .......... .......... 48% 32.0M 3s 2024-06-06T03:40:20.329 INFO:tasks.workunit.client.0.smithi040.stderr: 66000K .......... .......... .......... .......... .......... 48% 27.2M 3s 2024-06-06T03:40:20.329 INFO:tasks.workunit.client.0.smithi040.stderr: 66050K .......... .......... .......... .......... .......... 48% 26.9M 3s 2024-06-06T03:40:20.329 INFO:tasks.workunit.client.0.smithi040.stderr: 66100K .......... .......... .......... .......... .......... 48% 26.1M 3s 2024-06-06T03:40:20.332 INFO:tasks.workunit.client.0.smithi040.stderr: 66150K .......... .......... .......... .......... .......... 48% 20.2M 3s 2024-06-06T03:40:20.332 INFO:tasks.workunit.client.0.smithi040.stderr: 66200K .......... .......... .......... .......... .......... 48% 27.6M 3s 2024-06-06T03:40:20.335 INFO:tasks.workunit.client.0.smithi040.stderr: 66250K .......... .......... .......... .......... .......... 48% 16.1M 3s 2024-06-06T03:40:20.337 INFO:tasks.workunit.client.0.smithi040.stderr: 66300K .......... .......... .......... .......... .......... 48% 33.6M 3s 2024-06-06T03:40:20.340 INFO:tasks.workunit.client.0.smithi040.stderr: 66350K .......... .......... .......... .......... .......... 48% 29.8M 3s 2024-06-06T03:40:20.340 INFO:tasks.workunit.client.0.smithi040.stderr: 66400K .......... .......... .......... .......... .......... 48% 23.0M 3s 2024-06-06T03:40:20.343 INFO:tasks.workunit.client.0.smithi040.stderr: 66450K .......... .......... .......... .......... .......... 48% 19.8M 3s 2024-06-06T03:40:20.346 INFO:tasks.workunit.client.0.smithi040.stderr: 66500K .......... .......... .......... .......... .......... 49% 15.0M 3s 2024-06-06T03:40:20.347 INFO:tasks.workunit.client.0.smithi040.stderr: 66550K .......... .......... .......... .......... .......... 49% 26.7M 3s 2024-06-06T03:40:20.350 INFO:tasks.workunit.client.0.smithi040.stderr: 66600K .......... .......... .......... .......... .......... 49% 25.3M 3s 2024-06-06T03:40:20.353 INFO:tasks.workunit.client.0.smithi040.stderr: 66650K .......... .......... .......... .......... .......... 49% 15.5M 3s 2024-06-06T03:40:20.356 INFO:tasks.workunit.client.0.smithi040.stderr: 66700K .......... .......... .......... .......... .......... 49% 15.6M 3s 2024-06-06T03:40:20.357 INFO:tasks.workunit.client.0.smithi040.stderr: 66750K .......... .......... .......... .......... .......... 49% 37.4M 3s 2024-06-06T03:40:20.360 INFO:tasks.workunit.client.0.smithi040.stderr: 66800K .......... .......... .......... .......... .......... 49% 19.3M 3s 2024-06-06T03:40:20.362 INFO:tasks.workunit.client.0.smithi040.stderr: 66850K .......... .......... .......... .......... .......... 49% 23.8M 3s 2024-06-06T03:40:20.364 INFO:tasks.workunit.client.0.smithi040.stderr: 66900K .......... .......... .......... .......... .......... 49% 23.7M 3s 2024-06-06T03:40:20.365 INFO:tasks.workunit.client.0.smithi040.stderr: 66950K .......... .......... .......... .......... .......... 49% 38.1M 3s 2024-06-06T03:40:20.367 INFO:tasks.workunit.client.0.smithi040.stderr: 67000K .......... .......... .......... .......... .......... 49% 31.0M 3s 2024-06-06T03:40:20.369 INFO:tasks.workunit.client.0.smithi040.stderr: 67050K .......... .......... .......... .......... .......... 49% 29.6M 3s 2024-06-06T03:40:20.407 INFO:tasks.workunit.client.0.smithi040.stderr: 67100K .......... .......... .......... .......... .......... 49% 24.4M 3s 2024-06-06T03:40:20.407 INFO:tasks.workunit.client.0.smithi040.stderr: 67150K .......... .......... .......... .......... .......... 49% 21.3M 3s 2024-06-06T03:40:20.408 INFO:tasks.workunit.client.0.smithi040.stderr: 67200K .......... .......... .......... .......... .......... 49% 35.7M 3s 2024-06-06T03:40:20.408 INFO:tasks.workunit.client.0.smithi040.stderr: 67250K .......... .......... .......... .......... .......... 49% 30.9M 3s 2024-06-06T03:40:20.408 INFO:tasks.workunit.client.0.smithi040.stderr: 67300K .......... .......... .......... .......... .......... 49% 40.8M 3s 2024-06-06T03:40:20.408 INFO:tasks.workunit.client.0.smithi040.stderr: 67350K .......... .......... .......... .......... .......... 49% 33.5M 3s 2024-06-06T03:40:20.408 INFO:tasks.workunit.client.0.smithi040.stderr: 67400K .......... .......... .......... .......... .......... 49% 20.2M 3s 2024-06-06T03:40:20.408 INFO:tasks.workunit.client.0.smithi040.stderr: 67450K .......... .......... .......... .......... .......... 49% 51.8M 3s 2024-06-06T03:40:20.408 INFO:tasks.workunit.client.0.smithi040.stderr: 67500K .......... .......... .......... .......... .......... 49% 22.0M 3s 2024-06-06T03:40:20.408 INFO:tasks.workunit.client.0.smithi040.stderr: 67550K .......... .......... .......... .......... .......... 49% 41.0M 3s 2024-06-06T03:40:20.408 INFO:tasks.workunit.client.0.smithi040.stderr: 67600K .......... .......... .......... .......... .......... 49% 30.6M 3s 2024-06-06T03:40:20.408 INFO:tasks.workunit.client.0.smithi040.stderr: 67650K .......... .......... .......... .......... .......... 49% 20.8M 3s 2024-06-06T03:40:20.408 INFO:tasks.workunit.client.0.smithi040.stderr: 67700K .......... .......... .......... .......... .......... 49% 36.3M 3s 2024-06-06T03:40:20.408 INFO:tasks.workunit.client.0.smithi040.stderr: 67750K .......... .......... .......... .......... .......... 49% 29.1M 3s 2024-06-06T03:40:20.408 INFO:tasks.workunit.client.0.smithi040.stderr: 67800K .......... .......... .......... .......... .......... 49% 34.8M 3s 2024-06-06T03:40:20.408 INFO:tasks.workunit.client.0.smithi040.stderr: 67850K .......... .......... .......... .......... .......... 50% 25.0M 3s 2024-06-06T03:40:20.409 INFO:tasks.workunit.client.0.smithi040.stderr: 67900K .......... .......... .......... .......... .......... 50% 24.1M 3s 2024-06-06T03:40:20.409 INFO:tasks.workunit.client.0.smithi040.stderr: 67950K .......... .......... .......... .......... .......... 50% 37.4M 3s 2024-06-06T03:40:20.409 INFO:tasks.workunit.client.0.smithi040.stderr: 68000K .......... .......... .......... .......... .......... 50% 24.2M 3s 2024-06-06T03:40:20.409 INFO:tasks.workunit.client.0.smithi040.stderr: 68050K .......... .......... .......... .......... .......... 50% 40.3M 3s 2024-06-06T03:40:20.409 INFO:tasks.workunit.client.0.smithi040.stderr: 68100K .......... .......... .......... .......... .......... 50% 28.2M 3s 2024-06-06T03:40:20.409 INFO:tasks.workunit.client.0.smithi040.stderr: 68150K .......... .......... .......... .......... .......... 50% 22.4M 3s 2024-06-06T03:40:20.409 INFO:tasks.workunit.client.0.smithi040.stderr: 68200K .......... .......... .......... .......... .......... 50% 35.0M 3s 2024-06-06T03:40:20.419 INFO:tasks.workunit.client.0.smithi040.stderr: 68250K .......... .......... .......... .......... .......... 50% 25.0M 3s 2024-06-06T03:40:20.419 INFO:tasks.workunit.client.0.smithi040.stderr: 68300K .......... .......... .......... .......... .......... 50% 45.7M 3s 2024-06-06T03:40:20.419 INFO:tasks.workunit.client.0.smithi040.stderr: 68350K .......... .......... .......... .......... .......... 50% 31.2M 3s 2024-06-06T03:40:20.419 INFO:tasks.workunit.client.0.smithi040.stderr: 68400K .......... .......... .......... .......... .......... 50% 18.8M 3s 2024-06-06T03:40:20.419 INFO:tasks.workunit.client.0.smithi040.stderr: 68450K .......... .......... .......... .......... .......... 50% 35.3M 3s 2024-06-06T03:40:20.419 INFO:tasks.workunit.client.0.smithi040.stderr: 68500K .......... .......... .......... .......... .......... 50% 25.6M 3s 2024-06-06T03:40:20.430 INFO:tasks.workunit.client.0.smithi040.stderr: 68550K .......... .......... .......... .......... .......... 50% 3.79M 3s 2024-06-06T03:40:20.433 INFO:tasks.workunit.client.0.smithi040.stderr: 68600K .......... .......... .......... .......... .......... 50% 23.4M 3s 2024-06-06T03:40:20.435 INFO:tasks.workunit.client.0.smithi040.stderr: 68650K .......... .......... .......... .......... .......... 50% 24.4M 3s 2024-06-06T03:40:20.436 INFO:tasks.workunit.client.0.smithi040.stderr: 68700K .......... .......... .......... .......... .......... 50% 36.9M 3s 2024-06-06T03:40:20.438 INFO:tasks.workunit.client.0.smithi040.stderr: 68750K .......... .......... .......... .......... .......... 50% 26.4M 3s 2024-06-06T03:40:20.439 INFO:tasks.workunit.client.0.smithi040.stderr: 68800K .......... .......... .......... .......... .......... 50% 39.9M 3s 2024-06-06T03:40:20.450 INFO:tasks.workunit.client.0.smithi040.stderr: 68850K .......... .......... .......... .......... .......... 50% 26.4M 3s 2024-06-06T03:40:20.451 INFO:tasks.workunit.client.0.smithi040.stderr: 68900K .......... .......... .......... .......... .......... 50% 25.7M 3s 2024-06-06T03:40:20.451 INFO:tasks.workunit.client.0.smithi040.stderr: 68950K .......... .......... .......... .......... .......... 50% 31.0M 3s 2024-06-06T03:40:20.451 INFO:tasks.workunit.client.0.smithi040.stderr: 69000K .......... .......... .......... .......... .......... 50% 26.3M 3s 2024-06-06T03:40:20.451 INFO:tasks.workunit.client.0.smithi040.stderr: 69050K .......... .......... .......... .......... .......... 50% 42.3M 3s 2024-06-06T03:40:20.451 INFO:tasks.workunit.client.0.smithi040.stderr: 69100K .......... .......... .......... .......... .......... 50% 23.2M 3s 2024-06-06T03:40:20.466 INFO:tasks.workunit.client.0.smithi040.stderr: 69150K .......... .......... .......... .......... .......... 50% 25.1M 3s 2024-06-06T03:40:20.466 INFO:tasks.workunit.client.0.smithi040.stderr: 69200K .......... .......... .......... .......... .......... 51% 37.9M 3s 2024-06-06T03:40:20.466 INFO:tasks.workunit.client.0.smithi040.stderr: 69250K .......... .......... .......... .......... .......... 51% 26.6M 3s 2024-06-06T03:40:20.466 INFO:tasks.workunit.client.0.smithi040.stderr: 69300K .......... .......... .......... .......... .......... 51% 42.5M 3s 2024-06-06T03:40:20.466 INFO:tasks.workunit.client.0.smithi040.stderr: 69350K .......... .......... .......... .......... .......... 51% 26.8M 3s 2024-06-06T03:40:20.467 INFO:tasks.workunit.client.0.smithi040.stderr: 69400K .......... .......... .......... .......... .......... 51% 23.4M 3s 2024-06-06T03:40:20.467 INFO:tasks.workunit.client.0.smithi040.stderr: 69450K .......... .......... .......... .......... .......... 51% 35.8M 3s 2024-06-06T03:40:20.467 INFO:tasks.workunit.client.0.smithi040.stderr: 69500K .......... .......... .......... .......... .......... 51% 24.2M 3s 2024-06-06T03:40:20.467 INFO:tasks.workunit.client.0.smithi040.stderr: 69550K .......... .......... .......... .......... .......... 51% 42.7M 3s 2024-06-06T03:40:20.467 INFO:tasks.workunit.client.0.smithi040.stderr: 69600K .......... .......... .......... .......... .......... 51% 25.2M 3s 2024-06-06T03:40:20.471 INFO:tasks.workunit.client.0.smithi040.stderr: 69650K .......... .......... .......... .......... .......... 51% 24.5M 3s 2024-06-06T03:40:20.471 INFO:tasks.workunit.client.0.smithi040.stderr: 69700K .......... .......... .......... .......... .......... 51% 32.9M 3s 2024-06-06T03:40:20.471 INFO:tasks.workunit.client.0.smithi040.stderr: 69750K .......... .......... .......... .......... .......... 51% 24.3M 3s 2024-06-06T03:40:20.472 INFO:tasks.workunit.client.0.smithi040.stderr: 69800K .......... .......... .......... .......... .......... 51% 54.0M 3s 2024-06-06T03:40:20.475 INFO:tasks.workunit.client.0.smithi040.stderr: 69850K .......... .......... .......... .......... .......... 51% 25.7M 3s 2024-06-06T03:40:20.477 INFO:tasks.workunit.client.0.smithi040.stderr: 69900K .......... .......... .......... .......... .......... 51% 21.8M 3s 2024-06-06T03:40:20.479 INFO:tasks.workunit.client.0.smithi040.stderr: 69950K .......... .......... .......... .......... .......... 51% 43.0M 3s 2024-06-06T03:40:20.485 INFO:tasks.workunit.client.0.smithi040.stderr: 70000K .......... .......... .......... .......... .......... 51% 23.6M 3s 2024-06-06T03:40:20.486 INFO:tasks.workunit.client.0.smithi040.stderr: 70050K .......... .......... .......... .......... .......... 51% 41.1M 3s 2024-06-06T03:40:20.486 INFO:tasks.workunit.client.0.smithi040.stderr: 70100K .......... .......... .......... .......... .......... 51% 27.5M 3s 2024-06-06T03:40:20.486 INFO:tasks.workunit.client.0.smithi040.stderr: 70150K .......... .......... .......... .......... .......... 51% 29.7M 3s 2024-06-06T03:40:20.487 INFO:tasks.workunit.client.0.smithi040.stderr: 70200K .......... .......... .......... .......... .......... 51% 18.5M 3s 2024-06-06T03:40:20.491 INFO:tasks.workunit.client.0.smithi040.stderr: 70250K .......... .......... .......... .......... .......... 51% 22.7M 3s 2024-06-06T03:40:20.491 INFO:tasks.workunit.client.0.smithi040.stderr: 70300K .......... .......... .......... .......... .......... 51% 53.0M 3s 2024-06-06T03:40:20.492 INFO:tasks.workunit.client.0.smithi040.stderr: 70350K .......... .......... .......... .......... .......... 51% 21.4M 3s 2024-06-06T03:40:20.493 INFO:tasks.workunit.client.0.smithi040.stderr: 70400K .......... .......... .......... .......... .......... 51% 38.9M 3s 2024-06-06T03:40:20.496 INFO:tasks.workunit.client.0.smithi040.stderr: 70450K .......... .......... .......... .......... .......... 51% 26.1M 3s 2024-06-06T03:40:20.498 INFO:tasks.workunit.client.0.smithi040.stderr: 70500K .......... .......... .......... .......... .......... 51% 23.7M 3s 2024-06-06T03:40:20.510 INFO:tasks.workunit.client.0.smithi040.stderr: 70550K .......... .......... .......... .......... .......... 52% 31.9M 3s 2024-06-06T03:40:20.513 INFO:tasks.workunit.client.0.smithi040.stderr: 70600K .......... .......... .......... .......... .......... 52% 3.81M 3s 2024-06-06T03:40:20.514 INFO:tasks.workunit.client.0.smithi040.stderr: 70650K .......... .......... .......... .......... .......... 52% 24.4M 3s 2024-06-06T03:40:20.515 INFO:tasks.workunit.client.0.smithi040.stderr: 70700K .......... .......... .......... .......... .......... 52% 30.7M 3s 2024-06-06T03:40:20.520 INFO:tasks.workunit.client.0.smithi040.stderr: 70750K .......... .......... .......... .......... .......... 52% 30.4M 3s 2024-06-06T03:40:20.520 INFO:tasks.workunit.client.0.smithi040.stderr: 70800K .......... .......... .......... .......... .......... 52% 35.3M 3s 2024-06-06T03:40:20.521 INFO:tasks.workunit.client.0.smithi040.stderr: 70850K .......... .......... .......... .......... .......... 52% 26.7M 3s 2024-06-06T03:40:20.527 INFO:tasks.workunit.client.0.smithi040.stderr: 70900K .......... .......... .......... .......... .......... 52% 25.8M 3s 2024-06-06T03:40:20.527 INFO:tasks.workunit.client.0.smithi040.stderr: 70950K .......... .......... .......... .......... .......... 52% 35.1M 3s 2024-06-06T03:40:20.527 INFO:tasks.workunit.client.0.smithi040.stderr: 71000K .......... .......... .......... .......... .......... 52% 24.6M 3s 2024-06-06T03:40:20.528 INFO:tasks.workunit.client.0.smithi040.stderr: 71050K .......... .......... .......... .......... .......... 52% 22.8M 3s 2024-06-06T03:40:20.529 INFO:tasks.workunit.client.0.smithi040.stderr: 71100K .......... .......... .......... .......... .......... 52% 57.2M 3s 2024-06-06T03:40:20.542 INFO:tasks.workunit.client.0.smithi040.stderr: 71150K .......... .......... .......... .......... .......... 52% 25.9M 3s 2024-06-06T03:40:20.542 INFO:tasks.workunit.client.0.smithi040.stderr: 71200K .......... .......... .......... .......... .......... 52% 32.5M 3s 2024-06-06T03:40:20.543 INFO:tasks.workunit.client.0.smithi040.stderr: 71250K .......... .......... .......... .......... .......... 52% 23.5M 3s 2024-06-06T03:40:20.543 INFO:tasks.workunit.client.0.smithi040.stderr: 71300K .......... .......... .......... .......... .......... 52% 24.4M 3s 2024-06-06T03:40:20.543 INFO:tasks.workunit.client.0.smithi040.stderr: 71350K .......... .......... .......... .......... .......... 52% 37.2M 3s 2024-06-06T03:40:20.543 INFO:tasks.workunit.client.0.smithi040.stderr: 71400K .......... .......... .......... .......... .......... 52% 27.6M 3s 2024-06-06T03:40:20.543 INFO:tasks.workunit.client.0.smithi040.stderr: 71450K .......... .......... .......... .......... .......... 52% 32.4M 3s 2024-06-06T03:40:20.543 INFO:tasks.workunit.client.0.smithi040.stderr: 71500K .......... .......... .......... .......... .......... 52% 25.0M 3s 2024-06-06T03:40:20.546 INFO:tasks.workunit.client.0.smithi040.stderr: 71550K .......... .......... .......... .......... .......... 52% 26.5M 3s 2024-06-06T03:40:20.546 INFO:tasks.workunit.client.0.smithi040.stderr: 71600K .......... .......... .......... .......... .......... 52% 39.6M 3s 2024-06-06T03:40:20.551 INFO:tasks.workunit.client.0.smithi040.stderr: 71650K .......... .......... .......... .......... .......... 52% 25.5M 3s 2024-06-06T03:40:20.551 INFO:tasks.workunit.client.0.smithi040.stderr: 71700K .......... .......... .......... .......... .......... 52% 41.1M 3s 2024-06-06T03:40:20.551 INFO:tasks.workunit.client.0.smithi040.stderr: 71750K .......... .......... .......... .......... .......... 52% 23.2M 3s 2024-06-06T03:40:20.554 INFO:tasks.workunit.client.0.smithi040.stderr: 71800K .......... .......... .......... .......... .......... 52% 20.7M 3s 2024-06-06T03:40:20.555 INFO:tasks.workunit.client.0.smithi040.stderr: 71850K .......... .......... .......... .......... .......... 52% 35.6M 3s 2024-06-06T03:40:20.560 INFO:tasks.workunit.client.0.smithi040.stderr: 71900K .......... .......... .......... .......... .......... 53% 24.1M 3s 2024-06-06T03:40:20.560 INFO:tasks.workunit.client.0.smithi040.stderr: 71950K .......... .......... .......... .......... .......... 53% 36.8M 3s 2024-06-06T03:40:20.560 INFO:tasks.workunit.client.0.smithi040.stderr: 72000K .......... .......... .......... .......... .......... 53% 23.7M 3s 2024-06-06T03:40:20.562 INFO:tasks.workunit.client.0.smithi040.stderr: 72050K .......... .......... .......... .......... .......... 53% 25.8M 3s 2024-06-06T03:40:20.564 INFO:tasks.workunit.client.0.smithi040.stderr: 72100K .......... .......... .......... .......... .......... 53% 38.2M 3s 2024-06-06T03:40:20.565 INFO:tasks.workunit.client.0.smithi040.stderr: 72150K .......... .......... .......... .......... .......... 53% 26.2M 3s 2024-06-06T03:40:20.569 INFO:tasks.workunit.client.0.smithi040.stderr: 72200K .......... .......... .......... .......... .......... 53% 27.0M 3s 2024-06-06T03:40:20.570 INFO:tasks.workunit.client.0.smithi040.stderr: 72250K .......... .......... .......... .......... .......... 53% 15.2M 3s 2024-06-06T03:40:20.575 INFO:tasks.workunit.client.0.smithi040.stderr: 72300K .......... .......... .......... .......... .......... 53% 12.2M 3s 2024-06-06T03:40:20.578 INFO:tasks.workunit.client.0.smithi040.stderr: 72350K .......... .......... .......... .......... .......... 53% 14.5M 3s 2024-06-06T03:40:20.585 INFO:tasks.workunit.client.0.smithi040.stderr: 72400K .......... .......... .......... .......... .......... 53% 13.3M 3s 2024-06-06T03:40:20.585 INFO:tasks.workunit.client.0.smithi040.stderr: 72450K .......... .......... .......... .......... .......... 53% 15.9M 3s 2024-06-06T03:40:20.589 INFO:tasks.workunit.client.0.smithi040.stderr: 72500K .......... .......... .......... .......... .......... 53% 13.2M 3s 2024-06-06T03:40:20.599 INFO:tasks.workunit.client.0.smithi040.stderr: 72550K .......... .......... .......... .......... .......... 53% 24.0M 3s 2024-06-06T03:40:20.599 INFO:tasks.workunit.client.0.smithi040.stderr: 72600K .......... .......... .......... .......... .......... 53% 40.4M 3s 2024-06-06T03:40:20.599 INFO:tasks.workunit.client.0.smithi040.stderr: 72650K .......... .......... .......... .......... .......... 53% 25.8M 3s 2024-06-06T03:40:20.599 INFO:tasks.workunit.client.0.smithi040.stderr: 72700K .......... .......... .......... .......... .......... 53% 23.4M 3s 2024-06-06T03:40:20.599 INFO:tasks.workunit.client.0.smithi040.stderr: 72750K .......... .......... .......... .......... .......... 53% 39.7M 3s 2024-06-06T03:40:20.599 INFO:tasks.workunit.client.0.smithi040.stderr: 72800K .......... .......... .......... .......... .......... 53% 25.2M 3s 2024-06-06T03:40:20.603 INFO:tasks.workunit.client.0.smithi040.stderr: 72850K .......... .......... .......... .......... .......... 53% 39.6M 3s 2024-06-06T03:40:20.603 INFO:tasks.workunit.client.0.smithi040.stderr: 72900K .......... .......... .......... .......... .......... 53% 25.9M 3s 2024-06-06T03:40:20.606 INFO:tasks.workunit.client.0.smithi040.stderr: 72950K .......... .......... .......... .......... .......... 53% 23.3M 3s 2024-06-06T03:40:20.606 INFO:tasks.workunit.client.0.smithi040.stderr: 73000K .......... .......... .......... .......... .......... 53% 42.7M 3s 2024-06-06T03:40:20.619 INFO:tasks.workunit.client.0.smithi040.stderr: 73050K .......... .......... .......... .......... .......... 53% 24.0M 3s 2024-06-06T03:40:20.619 INFO:tasks.workunit.client.0.smithi040.stderr: 73100K .......... .......... .......... .......... .......... 53% 39.3M 3s 2024-06-06T03:40:20.619 INFO:tasks.workunit.client.0.smithi040.stderr: 73150K .......... .......... .......... .......... .......... 53% 24.3M 3s 2024-06-06T03:40:20.620 INFO:tasks.workunit.client.0.smithi040.stderr: 73200K .......... .......... .......... .......... .......... 53% 25.7M 3s 2024-06-06T03:40:20.620 INFO:tasks.workunit.client.0.smithi040.stderr: 73250K .......... .......... .......... .......... .......... 53% 40.7M 3s 2024-06-06T03:40:20.620 INFO:tasks.workunit.client.0.smithi040.stderr: 73300K .......... .......... .......... .......... .......... 54% 24.6M 3s 2024-06-06T03:40:20.620 INFO:tasks.workunit.client.0.smithi040.stderr: 73350K .......... .......... .......... .......... .......... 54% 40.9M 3s 2024-06-06T03:40:20.620 INFO:tasks.workunit.client.0.smithi040.stderr: 73400K .......... .......... .......... .......... .......... 54% 23.3M 3s 2024-06-06T03:40:20.621 INFO:tasks.workunit.client.0.smithi040.stderr: 73450K .......... .......... .......... .......... .......... 54% 20.6M 3s 2024-06-06T03:40:20.623 INFO:tasks.workunit.client.0.smithi040.stderr: 73500K .......... .......... .......... .......... .......... 54% 27.3M 3s 2024-06-06T03:40:20.624 INFO:tasks.workunit.client.0.smithi040.stderr: 73550K .......... .......... .......... .......... .......... 54% 25.5M 3s 2024-06-06T03:40:20.631 INFO:tasks.workunit.client.0.smithi040.stderr: 73600K .......... .......... .......... .......... .......... 54% 47.2M 3s 2024-06-06T03:40:20.631 INFO:tasks.workunit.client.0.smithi040.stderr: 73650K .......... .......... .......... .......... .......... 54% 24.7M 3s 2024-06-06T03:40:20.631 INFO:tasks.workunit.client.0.smithi040.stderr: 73700K .......... .......... .......... .......... .......... 54% 28.3M 3s 2024-06-06T03:40:20.631 INFO:tasks.workunit.client.0.smithi040.stderr: 73750K .......... .......... .......... .......... .......... 54% 27.8M 3s 2024-06-06T03:40:20.633 INFO:tasks.workunit.client.0.smithi040.stderr: 73800K .......... .......... .......... .......... .......... 54% 20.4M 3s 2024-06-06T03:40:20.634 INFO:tasks.workunit.client.0.smithi040.stderr: 73850K .......... .......... .......... .......... .......... 54% 46.5M 3s 2024-06-06T03:40:20.641 INFO:tasks.workunit.client.0.smithi040.stderr: 73900K .......... .......... .......... .......... .......... 54% 9.74M 3s 2024-06-06T03:40:20.641 INFO:tasks.workunit.client.0.smithi040.stderr: 73950K .......... .......... .......... .......... .......... 54% 86.6M 3s 2024-06-06T03:40:20.643 INFO:tasks.workunit.client.0.smithi040.stderr: 74000K .......... .......... .......... .......... .......... 54% 24.8M 3s 2024-06-06T03:40:20.646 INFO:tasks.workunit.client.0.smithi040.stderr: 74050K .......... .......... .......... .......... .......... 54% 15.7M 3s 2024-06-06T03:40:20.646 INFO:tasks.workunit.client.0.smithi040.stderr: 74100K .......... .......... .......... .......... .......... 54% 34.2M 3s 2024-06-06T03:40:20.651 INFO:tasks.workunit.client.0.smithi040.stderr: 74150K .......... .......... .......... .......... .......... 54% 12.8M 3s 2024-06-06T03:40:20.654 INFO:tasks.workunit.client.0.smithi040.stderr: 74200K .......... .......... .......... .......... .......... 54% 13.1M 3s 2024-06-06T03:40:20.660 INFO:tasks.workunit.client.0.smithi040.stderr: 74250K .......... .......... .......... .......... .......... 54% 9.54M 3s 2024-06-06T03:40:20.660 INFO:tasks.workunit.client.0.smithi040.stderr: 74300K .......... .......... .......... .......... .......... 54% 44.6M 3s 2024-06-06T03:40:20.664 INFO:tasks.workunit.client.0.smithi040.stderr: 74350K .......... .......... .......... .......... .......... 54% 26.6M 3s 2024-06-06T03:40:20.664 INFO:tasks.workunit.client.0.smithi040.stderr: 74400K .......... .......... .......... .......... .......... 54% 32.6M 3s 2024-06-06T03:40:20.666 INFO:tasks.workunit.client.0.smithi040.stderr: 74450K .......... .......... .......... .......... .......... 54% 27.1M 3s 2024-06-06T03:40:20.669 INFO:tasks.workunit.client.0.smithi040.stderr: 74500K .......... .......... .......... .......... .......... 54% 25.1M 3s 2024-06-06T03:40:20.669 INFO:tasks.workunit.client.0.smithi040.stderr: 74550K .......... .......... .......... .......... .......... 54% 21.7M 3s 2024-06-06T03:40:20.671 INFO:tasks.workunit.client.0.smithi040.stderr: 74600K .......... .......... .......... .......... .......... 54% 44.7M 3s 2024-06-06T03:40:20.677 INFO:tasks.workunit.client.0.smithi040.stderr: 74650K .......... .......... .......... .......... .......... 55% 24.1M 3s 2024-06-06T03:40:20.678 INFO:tasks.workunit.client.0.smithi040.stderr: 74700K .......... .......... .......... .......... .......... 55% 31.5M 3s 2024-06-06T03:40:20.678 INFO:tasks.workunit.client.0.smithi040.stderr: 74750K .......... .......... .......... .......... .......... 55% 24.9M 3s 2024-06-06T03:40:20.678 INFO:tasks.workunit.client.0.smithi040.stderr: 74800K .......... .......... .......... .......... .......... 55% 32.6M 3s 2024-06-06T03:40:20.679 INFO:tasks.workunit.client.0.smithi040.stderr: 74850K .......... .......... .......... .......... .......... 55% 32.1M 3s 2024-06-06T03:40:20.681 INFO:tasks.workunit.client.0.smithi040.stderr: 74900K .......... .......... .......... .......... .......... 55% 25.5M 3s 2024-06-06T03:40:20.683 INFO:tasks.workunit.client.0.smithi040.stderr: 74950K .......... .......... .......... .......... .......... 55% 31.9M 3s 2024-06-06T03:40:20.688 INFO:tasks.workunit.client.0.smithi040.stderr: 75000K .......... .......... .......... .......... .......... 55% 26.4M 3s 2024-06-06T03:40:20.688 INFO:tasks.workunit.client.0.smithi040.stderr: 75050K .......... .......... .......... .......... .......... 55% 30.7M 3s 2024-06-06T03:40:20.688 INFO:tasks.workunit.client.0.smithi040.stderr: 75100K .......... .......... .......... .......... .......... 55% 31.2M 3s 2024-06-06T03:40:20.691 INFO:tasks.workunit.client.0.smithi040.stderr: 75150K .......... .......... .......... .......... .......... 55% 20.5M 3s 2024-06-06T03:40:20.702 INFO:tasks.workunit.client.0.smithi040.stderr: 75200K .......... .......... .......... .......... .......... 55% 22.2M 3s 2024-06-06T03:40:20.702 INFO:tasks.workunit.client.0.smithi040.stderr: 75250K .......... .......... .......... .......... .......... 55% 28.9M 3s 2024-06-06T03:40:20.702 INFO:tasks.workunit.client.0.smithi040.stderr: 75300K .......... .......... .......... .......... .......... 55% 48.5M 3s 2024-06-06T03:40:20.702 INFO:tasks.workunit.client.0.smithi040.stderr: 75350K .......... .......... .......... .......... .......... 55% 27.0M 3s 2024-06-06T03:40:20.703 INFO:tasks.workunit.client.0.smithi040.stderr: 75400K .......... .......... .......... .......... .......... 55% 26.2M 3s 2024-06-06T03:40:20.703 INFO:tasks.workunit.client.0.smithi040.stderr: 75450K .......... .......... .......... .......... .......... 55% 29.1M 3s 2024-06-06T03:40:20.703 INFO:tasks.workunit.client.0.smithi040.stderr: 75500K .......... .......... .......... .......... .......... 55% 23.8M 3s 2024-06-06T03:40:20.704 INFO:tasks.workunit.client.0.smithi040.stderr: 75550K .......... .......... .......... .......... .......... 55% 29.3M 3s 2024-06-06T03:40:20.706 INFO:tasks.workunit.client.0.smithi040.stderr: 75600K .......... .......... .......... .......... .......... 55% 20.5M 3s 2024-06-06T03:40:20.710 INFO:tasks.workunit.client.0.smithi040.stderr: 75650K .......... .......... .......... .......... .......... 55% 20.6M 3s 2024-06-06T03:40:20.712 INFO:tasks.workunit.client.0.smithi040.stderr: 75700K .......... .......... .......... .......... .......... 55% 16.0M 3s 2024-06-06T03:40:20.715 INFO:tasks.workunit.client.0.smithi040.stderr: 75750K .......... .......... .......... .......... .......... 55% 16.9M 3s 2024-06-06T03:40:20.716 INFO:tasks.workunit.client.0.smithi040.stderr: 75800K .......... .......... .......... .......... .......... 55% 32.2M 3s 2024-06-06T03:40:20.722 INFO:tasks.workunit.client.0.smithi040.stderr: 75850K .......... .......... .......... .......... .......... 55% 18.2M 3s 2024-06-06T03:40:20.723 INFO:tasks.workunit.client.0.smithi040.stderr: 75900K .......... .......... .......... .......... .......... 55% 18.9M 3s 2024-06-06T03:40:20.726 INFO:tasks.workunit.client.0.smithi040.stderr: 75950K .......... .......... .......... .......... .......... 55% 12.9M 3s 2024-06-06T03:40:20.728 INFO:tasks.workunit.client.0.smithi040.stderr: 76000K .......... .......... .......... .......... .......... 56% 20.9M 3s 2024-06-06T03:40:20.731 INFO:tasks.workunit.client.0.smithi040.stderr: 76050K .......... .......... .......... .......... .......... 56% 12.6M 3s 2024-06-06T03:40:20.738 INFO:tasks.workunit.client.0.smithi040.stderr: 76100K .......... .......... .......... .......... .......... 56% 19.4M 3s 2024-06-06T03:40:20.738 INFO:tasks.workunit.client.0.smithi040.stderr: 76150K .......... .......... .......... .......... .......... 56% 40.0M 3s 2024-06-06T03:40:20.738 INFO:tasks.workunit.client.0.smithi040.stderr: 76200K .......... .......... .......... .......... .......... 56% 24.6M 3s 2024-06-06T03:40:20.739 INFO:tasks.workunit.client.0.smithi040.stderr: 76250K .......... .......... .......... .......... .......... 56% 36.9M 3s 2024-06-06T03:40:20.740 INFO:tasks.workunit.client.0.smithi040.stderr: 76300K .......... .......... .......... .......... .......... 56% 24.3M 3s 2024-06-06T03:40:20.742 INFO:tasks.workunit.client.0.smithi040.stderr: 76350K .......... .......... .......... .......... .......... 56% 25.7M 3s 2024-06-06T03:40:20.745 INFO:tasks.workunit.client.0.smithi040.stderr: 76400K .......... .......... .......... .......... .......... 56% 19.2M 3s 2024-06-06T03:40:20.761 INFO:tasks.workunit.client.0.smithi040.stderr: 76450K .......... .......... .......... .......... .......... 56% 12.8M 3s 2024-06-06T03:40:20.761 INFO:tasks.workunit.client.0.smithi040.stderr: 76500K .......... .......... .......... .......... .......... 56% 117M 3s 2024-06-06T03:40:20.761 INFO:tasks.workunit.client.0.smithi040.stderr: 76550K .......... .......... .......... .......... .......... 56% 118M 3s 2024-06-06T03:40:20.761 INFO:tasks.workunit.client.0.smithi040.stderr: 76600K .......... .......... .......... .......... .......... 56% 31.4M 3s 2024-06-06T03:40:20.761 INFO:tasks.workunit.client.0.smithi040.stderr: 76650K .......... .......... .......... .......... .......... 56% 38.9M 2s 2024-06-06T03:40:20.761 INFO:tasks.workunit.client.0.smithi040.stderr: 76700K .......... .......... .......... .......... .......... 56% 26.7M 2s 2024-06-06T03:40:20.762 INFO:tasks.workunit.client.0.smithi040.stderr: 76750K .......... .......... .......... .......... .......... 56% 38.5M 2s 2024-06-06T03:40:20.762 INFO:tasks.workunit.client.0.smithi040.stderr: 76800K .......... .......... .......... .......... .......... 56% 20.3M 2s 2024-06-06T03:40:20.762 INFO:tasks.workunit.client.0.smithi040.stderr: 76850K .......... .......... .......... .......... .......... 56% 21.5M 2s 2024-06-06T03:40:20.762 INFO:tasks.workunit.client.0.smithi040.stderr: 76900K .......... .......... .......... .......... .......... 56% 45.2M 2s 2024-06-06T03:40:20.763 INFO:tasks.workunit.client.0.smithi040.stderr: 76950K .......... .......... .......... .......... .......... 56% 22.5M 2s 2024-06-06T03:40:20.767 INFO:tasks.workunit.client.0.smithi040.stderr: 77000K .......... .......... .......... .......... .......... 56% 43.1M 2s 2024-06-06T03:40:20.767 INFO:tasks.workunit.client.0.smithi040.stderr: 77050K .......... .......... .......... .......... .......... 56% 23.2M 2s 2024-06-06T03:40:20.775 INFO:tasks.workunit.client.0.smithi040.stderr: 77100K .......... .......... .......... .......... .......... 56% 26.5M 2s 2024-06-06T03:40:20.775 INFO:tasks.workunit.client.0.smithi040.stderr: 77150K .......... .......... .......... .......... .......... 56% 47.6M 2s 2024-06-06T03:40:20.775 INFO:tasks.workunit.client.0.smithi040.stderr: 77200K .......... .......... .......... .......... .......... 56% 22.7M 2s 2024-06-06T03:40:20.775 INFO:tasks.workunit.client.0.smithi040.stderr: 77250K .......... .......... .......... .......... .......... 56% 31.8M 2s 2024-06-06T03:40:20.777 INFO:tasks.workunit.client.0.smithi040.stderr: 77300K .......... .......... .......... .......... .......... 56% 16.8M 2s 2024-06-06T03:40:20.778 INFO:tasks.workunit.client.0.smithi040.stderr: 77350K .......... .......... .......... .......... .......... 57% 22.9M 2s 2024-06-06T03:40:20.782 INFO:tasks.workunit.client.0.smithi040.stderr: 77400K .......... .......... .......... .......... .......... 57% 27.0M 2s 2024-06-06T03:40:20.783 INFO:tasks.workunit.client.0.smithi040.stderr: 77450K .......... .......... .......... .......... .......... 57% 16.5M 2s 2024-06-06T03:40:20.786 INFO:tasks.workunit.client.0.smithi040.stderr: 77500K .......... .......... .......... .......... .......... 57% 29.2M 2s 2024-06-06T03:40:20.787 INFO:tasks.workunit.client.0.smithi040.stderr: 77550K .......... .......... .......... .......... .......... 57% 19.9M 2s 2024-06-06T03:40:20.791 INFO:tasks.workunit.client.0.smithi040.stderr: 77600K .......... .......... .......... .......... .......... 57% 12.8M 2s 2024-06-06T03:40:20.794 INFO:tasks.workunit.client.0.smithi040.stderr: 77650K .......... .......... .......... .......... .......... 57% 28.3M 2s 2024-06-06T03:40:20.796 INFO:tasks.workunit.client.0.smithi040.stderr: 77700K .......... .......... .......... .......... .......... 57% 13.3M 2s 2024-06-06T03:40:20.798 INFO:tasks.workunit.client.0.smithi040.stderr: 77750K .......... .......... .......... .......... .......... 57% 31.3M 2s 2024-06-06T03:40:20.800 INFO:tasks.workunit.client.0.smithi040.stderr: 77800K .......... .......... .......... .......... .......... 57% 21.8M 2s 2024-06-06T03:40:20.805 INFO:tasks.workunit.client.0.smithi040.stderr: 77850K .......... .......... .......... .......... .......... 57% 10.6M 2s 2024-06-06T03:40:20.806 INFO:tasks.workunit.client.0.smithi040.stderr: 77900K .......... .......... .......... .......... .......... 57% 27.6M 2s 2024-06-06T03:40:20.826 INFO:tasks.workunit.client.0.smithi040.stderr: 77950K .......... .......... .......... .......... .......... 57% 27.2M 2s 2024-06-06T03:40:20.826 INFO:tasks.workunit.client.0.smithi040.stderr: 78000K .......... .......... .......... .......... .......... 57% 32.7M 2s 2024-06-06T03:40:20.826 INFO:tasks.workunit.client.0.smithi040.stderr: 78050K .......... .......... .......... .......... .......... 57% 26.5M 2s 2024-06-06T03:40:20.826 INFO:tasks.workunit.client.0.smithi040.stderr: 78100K .......... .......... .......... .......... .......... 57% 24.4M 2s 2024-06-06T03:40:20.826 INFO:tasks.workunit.client.0.smithi040.stderr: 78150K .......... .......... .......... .......... .......... 57% 42.9M 2s 2024-06-06T03:40:20.826 INFO:tasks.workunit.client.0.smithi040.stderr: 78200K .......... .......... .......... .......... .......... 57% 26.9M 2s 2024-06-06T03:40:20.826 INFO:tasks.workunit.client.0.smithi040.stderr: 78250K .......... .......... .......... .......... .......... 57% 28.9M 2s 2024-06-06T03:40:20.826 INFO:tasks.workunit.client.0.smithi040.stderr: 78300K .......... .......... .......... .......... .......... 57% 28.8M 2s 2024-06-06T03:40:20.826 INFO:tasks.workunit.client.0.smithi040.stderr: 78350K .......... .......... .......... .......... .......... 57% 24.2M 2s 2024-06-06T03:40:20.826 INFO:tasks.workunit.client.0.smithi040.stderr: 78400K .......... .......... .......... .......... .......... 57% 37.3M 2s 2024-06-06T03:40:20.827 INFO:tasks.workunit.client.0.smithi040.stderr: 78450K .......... .......... .......... .......... .......... 57% 22.8M 2s 2024-06-06T03:40:20.834 INFO:tasks.workunit.client.0.smithi040.stderr: 78500K .......... .......... .......... .......... .......... 57% 24.7M 2s 2024-06-06T03:40:20.834 INFO:tasks.workunit.client.0.smithi040.stderr: 78550K .......... .......... .......... .......... .......... 57% 40.5M 2s 2024-06-06T03:40:20.834 INFO:tasks.workunit.client.0.smithi040.stderr: 78600K .......... .......... .......... .......... .......... 57% 20.1M 2s 2024-06-06T03:40:20.835 INFO:tasks.workunit.client.0.smithi040.stderr: 78650K .......... .......... .......... .......... .......... 57% 41.9M 2s 2024-06-06T03:40:20.835 INFO:tasks.workunit.client.0.smithi040.stderr: 78700K .......... .......... .......... .......... .......... 58% 22.9M 2s 2024-06-06T03:40:20.839 INFO:tasks.workunit.client.0.smithi040.stderr: 78750K .......... .......... .......... .......... .......... 58% 26.5M 2s 2024-06-06T03:40:20.839 INFO:tasks.workunit.client.0.smithi040.stderr: 78800K .......... .......... .......... .......... .......... 58% 33.6M 2s 2024-06-06T03:40:20.840 INFO:tasks.workunit.client.0.smithi040.stderr: 78850K .......... .......... .......... .......... .......... 58% 24.4M 2s 2024-06-06T03:40:20.841 INFO:tasks.workunit.client.0.smithi040.stderr: 78900K .......... .......... .......... .......... .......... 58% 40.2M 2s 2024-06-06T03:40:20.843 INFO:tasks.workunit.client.0.smithi040.stderr: 78950K .......... .......... .......... .......... .......... 58% 20.3M 2s 2024-06-06T03:40:20.846 INFO:tasks.workunit.client.0.smithi040.stderr: 79000K .......... .......... .......... .......... .......... 58% 20.2M 2s 2024-06-06T03:40:20.854 INFO:tasks.workunit.client.0.smithi040.stderr: 79050K .......... .......... .......... .......... .......... 58% 37.8M 2s 2024-06-06T03:40:20.854 INFO:tasks.workunit.client.0.smithi040.stderr: 79100K .......... .......... .......... .......... .......... 58% 23.4M 2s 2024-06-06T03:40:20.854 INFO:tasks.workunit.client.0.smithi040.stderr: 79150K .......... .......... .......... .......... .......... 58% 30.2M 2s 2024-06-06T03:40:20.854 INFO:tasks.workunit.client.0.smithi040.stderr: 79200K .......... .......... .......... .......... .......... 58% 19.6M 2s 2024-06-06T03:40:20.856 INFO:tasks.workunit.client.0.smithi040.stderr: 79250K .......... .......... .......... .......... .......... 58% 17.2M 2s 2024-06-06T03:40:20.858 INFO:tasks.workunit.client.0.smithi040.stderr: 79300K .......... .......... .......... .......... .......... 58% 23.2M 2s 2024-06-06T03:40:20.860 INFO:tasks.workunit.client.0.smithi040.stderr: 79350K .......... .......... .......... .......... .......... 58% 18.8M 2s 2024-06-06T03:40:20.863 INFO:tasks.workunit.client.0.smithi040.stderr: 79400K .......... .......... .......... .......... .......... 58% 20.8M 2s 2024-06-06T03:40:20.866 INFO:tasks.workunit.client.0.smithi040.stderr: 79450K .......... .......... .......... .......... .......... 58% 22.3M 2s 2024-06-06T03:40:20.869 INFO:tasks.workunit.client.0.smithi040.stderr: 79500K .......... .......... .......... .......... .......... 58% 15.0M 2s 2024-06-06T03:40:20.870 INFO:tasks.workunit.client.0.smithi040.stderr: 79550K .......... .......... .......... .......... .......... 58% 23.8M 2s 2024-06-06T03:40:20.874 INFO:tasks.workunit.client.0.smithi040.stderr: 79600K .......... .......... .......... .......... .......... 58% 14.0M 2s 2024-06-06T03:40:20.876 INFO:tasks.workunit.client.0.smithi040.stderr: 79650K .......... .......... .......... .......... .......... 58% 22.0M 2s 2024-06-06T03:40:20.880 INFO:tasks.workunit.client.0.smithi040.stderr: 79700K .......... .......... .......... .......... .......... 58% 16.5M 2s 2024-06-06T03:40:20.881 INFO:tasks.workunit.client.0.smithi040.stderr: 79750K .......... .......... .......... .......... .......... 58% 23.1M 2s 2024-06-06T03:40:20.882 INFO:tasks.workunit.client.0.smithi040.stderr: 79800K .......... .......... .......... .......... .......... 58% 36.4M 2s 2024-06-06T03:40:20.885 INFO:tasks.workunit.client.0.smithi040.stderr: 79850K .......... .......... .......... .......... .......... 58% 23.8M 2s 2024-06-06T03:40:20.886 INFO:tasks.workunit.client.0.smithi040.stderr: 79900K .......... .......... .......... .......... .......... 58% 44.7M 2s 2024-06-06T03:40:20.888 INFO:tasks.workunit.client.0.smithi040.stderr: 79950K .......... .......... .......... .......... .......... 58% 24.5M 2s 2024-06-06T03:40:20.890 INFO:tasks.workunit.client.0.smithi040.stderr: 80000K .......... .......... .......... .......... .......... 58% 23.9M 2s 2024-06-06T03:40:20.892 INFO:tasks.workunit.client.0.smithi040.stderr: 80050K .......... .......... .......... .......... .......... 59% 41.8M 2s 2024-06-06T03:40:20.894 INFO:tasks.workunit.client.0.smithi040.stderr: 80100K .......... .......... .......... .......... .......... 59% 22.5M 2s 2024-06-06T03:40:20.894 INFO:tasks.workunit.client.0.smithi040.stderr: 80150K .......... .......... .......... .......... .......... 59% 40.0M 2s 2024-06-06T03:40:20.899 INFO:tasks.workunit.client.0.smithi040.stderr: 80200K .......... .......... .......... .......... .......... 59% 26.5M 2s 2024-06-06T03:40:20.899 INFO:tasks.workunit.client.0.smithi040.stderr: 80250K .......... .......... .......... .......... .......... 59% 25.2M 2s 2024-06-06T03:40:20.902 INFO:tasks.workunit.client.0.smithi040.stderr: 80300K .......... .......... .......... .......... .......... 59% 27.2M 2s 2024-06-06T03:40:20.902 INFO:tasks.workunit.client.0.smithi040.stderr: 80350K .......... .......... .......... .......... .......... 59% 23.6M 2s 2024-06-06T03:40:20.903 INFO:tasks.workunit.client.0.smithi040.stderr: 80400K .......... .......... .......... .......... .......... 59% 38.0M 2s 2024-06-06T03:40:20.905 INFO:tasks.workunit.client.0.smithi040.stderr: 80450K .......... .......... .......... .......... .......... 59% 24.6M 2s 2024-06-06T03:40:20.907 INFO:tasks.workunit.client.0.smithi040.stderr: 80500K .......... .......... .......... .......... .......... 59% 25.9M 2s 2024-06-06T03:40:20.910 INFO:tasks.workunit.client.0.smithi040.stderr: 80550K .......... .......... .......... .......... .......... 59% 23.2M 2s 2024-06-06T03:40:20.911 INFO:tasks.workunit.client.0.smithi040.stderr: 80600K .......... .......... .......... .......... .......... 59% 35.4M 2s 2024-06-06T03:40:20.912 INFO:tasks.workunit.client.0.smithi040.stderr: 80650K .......... .......... .......... .......... .......... 59% 35.0M 2s 2024-06-06T03:40:20.914 INFO:tasks.workunit.client.0.smithi040.stderr: 80700K .......... .......... .......... .......... .......... 59% 20.0M 2s 2024-06-06T03:40:20.916 INFO:tasks.workunit.client.0.smithi040.stderr: 80750K .......... .......... .......... .......... .......... 59% 24.6M 2s 2024-06-06T03:40:20.919 INFO:tasks.workunit.client.0.smithi040.stderr: 80800K .......... .......... .......... .......... .......... 59% 37.4M 2s 2024-06-06T03:40:20.923 INFO:tasks.workunit.client.0.smithi040.stderr: 80850K .......... .......... .......... .......... .......... 59% 17.8M 2s 2024-06-06T03:40:20.923 INFO:tasks.workunit.client.0.smithi040.stderr: 80900K .......... .......... .......... .......... .......... 59% 46.4M 2s 2024-06-06T03:40:20.925 INFO:tasks.workunit.client.0.smithi040.stderr: 80950K .......... .......... .......... .......... .......... 59% 17.4M 2s 2024-06-06T03:40:20.927 INFO:tasks.workunit.client.0.smithi040.stderr: 81000K .......... .......... .......... .......... .......... 59% 18.4M 2s 2024-06-06T03:40:20.929 INFO:tasks.workunit.client.0.smithi040.stderr: 81050K .......... .......... .......... .......... .......... 59% 23.2M 2s 2024-06-06T03:40:20.932 INFO:tasks.workunit.client.0.smithi040.stderr: 81100K .......... .......... .......... .......... .......... 59% 20.6M 2s 2024-06-06T03:40:20.935 INFO:tasks.workunit.client.0.smithi040.stderr: 81150K .......... .......... .......... .......... .......... 59% 24.9M 2s 2024-06-06T03:40:20.937 INFO:tasks.workunit.client.0.smithi040.stderr: 81200K .......... .......... .......... .......... .......... 59% 18.3M 2s 2024-06-06T03:40:20.939 INFO:tasks.workunit.client.0.smithi040.stderr: 81250K .......... .......... .......... .......... .......... 59% 20.8M 2s 2024-06-06T03:40:20.941 INFO:tasks.workunit.client.0.smithi040.stderr: 81300K .......... .......... .......... .......... .......... 59% 17.2M 2s 2024-06-06T03:40:20.944 INFO:tasks.workunit.client.0.smithi040.stderr: 81350K .......... .......... .......... .......... .......... 59% 19.8M 2s 2024-06-06T03:40:20.947 INFO:tasks.workunit.client.0.smithi040.stderr: 81400K .......... .......... .......... .......... .......... 59% 12.6M 2s 2024-06-06T03:40:20.949 INFO:tasks.workunit.client.0.smithi040.stderr: 81450K .......... .......... .......... .......... .......... 60% 36.5M 2s 2024-06-06T03:40:20.952 INFO:tasks.workunit.client.0.smithi040.stderr: 81500K .......... .......... .......... .......... .......... 60% 19.1M 2s 2024-06-06T03:40:20.953 INFO:tasks.workunit.client.0.smithi040.stderr: 81550K .......... .......... .......... .......... .......... 60% 22.0M 2s 2024-06-06T03:40:20.960 INFO:tasks.workunit.client.0.smithi040.stderr: 81600K .......... .......... .......... .......... .......... 60% 25.8M 2s 2024-06-06T03:40:20.960 INFO:tasks.workunit.client.0.smithi040.stderr: 81650K .......... .......... .......... .......... .......... 60% 22.3M 2s 2024-06-06T03:40:20.960 INFO:tasks.workunit.client.0.smithi040.stderr: 81700K .......... .......... .......... .......... .......... 60% 38.1M 2s 2024-06-06T03:40:20.961 INFO:tasks.workunit.client.0.smithi040.stderr: 81750K .......... .......... .......... .......... .......... 60% 25.7M 2s 2024-06-06T03:40:20.965 INFO:tasks.workunit.client.0.smithi040.stderr: 81800K .......... .......... .......... .......... .......... 60% 37.8M 2s 2024-06-06T03:40:20.965 INFO:tasks.workunit.client.0.smithi040.stderr: 81850K .......... .......... .......... .......... .......... 60% 23.1M 2s 2024-06-06T03:40:20.979 INFO:tasks.workunit.client.0.smithi040.stderr: 81900K .......... .......... .......... .......... .......... 60% 25.7M 2s 2024-06-06T03:40:20.979 INFO:tasks.workunit.client.0.smithi040.stderr: 81950K .......... .......... .......... .......... .......... 60% 39.3M 2s 2024-06-06T03:40:20.979 INFO:tasks.workunit.client.0.smithi040.stderr: 82000K .......... .......... .......... .......... .......... 60% 19.4M 2s 2024-06-06T03:40:20.979 INFO:tasks.workunit.client.0.smithi040.stderr: 82050K .......... .......... .......... .......... .......... 60% 43.4M 2s 2024-06-06T03:40:20.979 INFO:tasks.workunit.client.0.smithi040.stderr: 82100K .......... .......... .......... .......... .......... 60% 22.7M 2s 2024-06-06T03:40:20.979 INFO:tasks.workunit.client.0.smithi040.stderr: 82150K .......... .......... .......... .......... .......... 60% 29.6M 2s 2024-06-06T03:40:20.979 INFO:tasks.workunit.client.0.smithi040.stderr: 82200K .......... .......... .......... .......... .......... 60% 42.0M 2s 2024-06-06T03:40:20.979 INFO:tasks.workunit.client.0.smithi040.stderr: 82250K .......... .......... .......... .......... .......... 60% 24.8M 2s 2024-06-06T03:40:20.999 INFO:tasks.workunit.client.0.smithi040.stderr: 82300K .......... .......... .......... .......... .......... 60% 26.2M 2s 2024-06-06T03:40:20.999 INFO:tasks.workunit.client.0.smithi040.stderr: 82350K .......... .......... .......... .......... .......... 60% 20.7M 2s 2024-06-06T03:40:20.999 INFO:tasks.workunit.client.0.smithi040.stderr: 82400K .......... .......... .......... .......... .......... 60% 48.6M 2s 2024-06-06T03:40:20.999 INFO:tasks.workunit.client.0.smithi040.stderr: 82450K .......... .......... .......... .......... .......... 60% 20.2M 2s 2024-06-06T03:40:20.999 INFO:tasks.workunit.client.0.smithi040.stderr: 82500K .......... .......... .......... .......... .......... 60% 25.5M 2s 2024-06-06T03:40:20.999 INFO:tasks.workunit.client.0.smithi040.stderr: 82550K .......... .......... .......... .......... .......... 60% 42.4M 2s 2024-06-06T03:40:20.999 INFO:tasks.workunit.client.0.smithi040.stderr: 82600K .......... .......... .......... .......... .......... 60% 17.2M 2s 2024-06-06T03:40:20.999 INFO:tasks.workunit.client.0.smithi040.stderr: 82650K .......... .......... .......... .......... .......... 60% 31.0M 2s 2024-06-06T03:40:20.999 INFO:tasks.workunit.client.0.smithi040.stderr: 82700K .......... .......... .......... .......... .......... 60% 20.9M 2s 2024-06-06T03:40:21.000 INFO:tasks.workunit.client.0.smithi040.stderr: 82750K .......... .......... .......... .......... .......... 60% 22.1M 2s 2024-06-06T03:40:21.001 INFO:tasks.workunit.client.0.smithi040.stderr: 82800K .......... .......... .......... .......... .......... 61% 21.3M 2s 2024-06-06T03:40:21.004 INFO:tasks.workunit.client.0.smithi040.stderr: 82850K .......... .......... .......... .......... .......... 61% 22.7M 2s 2024-06-06T03:40:21.004 INFO:tasks.workunit.client.0.smithi040.stderr: 82900K .......... .......... .......... .......... .......... 61% 31.1M 2s 2024-06-06T03:40:21.007 INFO:tasks.workunit.client.0.smithi040.stderr: 82950K .......... .......... .......... .......... .......... 61% 16.8M 2s 2024-06-06T03:40:21.009 INFO:tasks.workunit.client.0.smithi040.stderr: 83000K .......... .......... .......... .......... .......... 61% 18.1M 2s 2024-06-06T03:40:21.012 INFO:tasks.workunit.client.0.smithi040.stderr: 83050K .......... .......... .......... .......... .......... 61% 17.5M 2s 2024-06-06T03:40:21.016 INFO:tasks.workunit.client.0.smithi040.stderr: 83100K .......... .......... .......... .......... .......... 61% 18.6M 2s 2024-06-06T03:40:21.017 INFO:tasks.workunit.client.0.smithi040.stderr: 83150K .......... .......... .......... .......... .......... 61% 22.5M 2s 2024-06-06T03:40:21.019 INFO:tasks.workunit.client.0.smithi040.stderr: 83200K .......... .......... .......... .......... .......... 61% 29.3M 2s 2024-06-06T03:40:21.022 INFO:tasks.workunit.client.0.smithi040.stderr: 83250K .......... .......... .......... .......... .......... 61% 18.1M 2s 2024-06-06T03:40:21.025 INFO:tasks.workunit.client.0.smithi040.stderr: 83300K .......... .......... .......... .......... .......... 61% 13.4M 2s 2024-06-06T03:40:21.027 INFO:tasks.workunit.client.0.smithi040.stderr: 83350K .......... .......... .......... .......... .......... 61% 33.8M 2s 2024-06-06T03:40:21.030 INFO:tasks.workunit.client.0.smithi040.stderr: 83400K .......... .......... .......... .......... .......... 61% 20.9M 2s 2024-06-06T03:40:21.030 INFO:tasks.workunit.client.0.smithi040.stderr: 83450K .......... .......... .......... .......... .......... 61% 39.4M 2s 2024-06-06T03:40:21.064 INFO:tasks.workunit.client.0.smithi040.stderr: 83500K .......... .......... .......... .......... .......... 61% 21.3M 2s 2024-06-06T03:40:21.064 INFO:tasks.workunit.client.0.smithi040.stderr: 83550K .......... .......... .......... .......... .......... 61% 27.3M 2s 2024-06-06T03:40:21.065 INFO:tasks.workunit.client.0.smithi040.stderr: 83600K .......... .......... .......... .......... .......... 61% 31.6M 2s 2024-06-06T03:40:21.065 INFO:tasks.workunit.client.0.smithi040.stderr: 83650K .......... .......... .......... .......... .......... 61% 29.1M 2s 2024-06-06T03:40:21.065 INFO:tasks.workunit.client.0.smithi040.stderr: 83700K .......... .......... .......... .......... .......... 61% 26.1M 2s 2024-06-06T03:40:21.065 INFO:tasks.workunit.client.0.smithi040.stderr: 83750K .......... .......... .......... .......... .......... 61% 24.0M 2s 2024-06-06T03:40:21.065 INFO:tasks.workunit.client.0.smithi040.stderr: 83800K .......... .......... .......... .......... .......... 61% 26.8M 2s 2024-06-06T03:40:21.065 INFO:tasks.workunit.client.0.smithi040.stderr: 83850K .......... .......... .......... .......... .......... 61% 26.8M 2s 2024-06-06T03:40:21.065 INFO:tasks.workunit.client.0.smithi040.stderr: 83900K .......... .......... .......... .......... .......... 61% 30.0M 2s 2024-06-06T03:40:21.065 INFO:tasks.workunit.client.0.smithi040.stderr: 83950K .......... .......... .......... .......... .......... 61% 44.5M 2s 2024-06-06T03:40:21.065 INFO:tasks.workunit.client.0.smithi040.stderr: 84000K .......... .......... .......... .......... .......... 61% 21.0M 2s 2024-06-06T03:40:21.065 INFO:tasks.workunit.client.0.smithi040.stderr: 84050K .......... .......... .......... .......... .......... 61% 23.3M 2s 2024-06-06T03:40:21.065 INFO:tasks.workunit.client.0.smithi040.stderr: 84100K .......... .......... .......... .......... .......... 61% 29.3M 2s 2024-06-06T03:40:21.065 INFO:tasks.workunit.client.0.smithi040.stderr: 84150K .......... .......... .......... .......... .......... 62% 20.4M 2s 2024-06-06T03:40:21.065 INFO:tasks.workunit.client.0.smithi040.stderr: 84200K .......... .......... .......... .......... .......... 62% 38.4M 2s 2024-06-06T03:40:21.066 INFO:tasks.workunit.client.0.smithi040.stderr: 84250K .......... .......... .......... .......... .......... 62% 25.2M 2s 2024-06-06T03:40:21.066 INFO:tasks.workunit.client.0.smithi040.stderr: 84300K .......... .......... .......... .......... .......... 62% 25.6M 2s 2024-06-06T03:40:21.066 INFO:tasks.workunit.client.0.smithi040.stderr: 84350K .......... .......... .......... .......... .......... 62% 35.6M 2s 2024-06-06T03:40:21.069 INFO:tasks.workunit.client.0.smithi040.stderr: 84400K .......... .......... .......... .......... .......... 62% 15.2M 2s 2024-06-06T03:40:21.069 INFO:tasks.workunit.client.0.smithi040.stderr: 84450K .......... .......... .......... .......... .......... 62% 42.3M 2s 2024-06-06T03:40:21.069 INFO:tasks.workunit.client.0.smithi040.stderr: 84500K .......... .......... .......... .......... .......... 62% 22.8M 2s 2024-06-06T03:40:21.072 INFO:tasks.workunit.client.0.smithi040.stderr: 84550K .......... .......... .......... .......... .......... 62% 21.4M 2s 2024-06-06T03:40:21.073 INFO:tasks.workunit.client.0.smithi040.stderr: 84600K .......... .......... .......... .......... .......... 62% 30.3M 2s 2024-06-06T03:40:21.076 INFO:tasks.workunit.client.0.smithi040.stderr: 84650K .......... .......... .......... .......... .......... 62% 19.1M 2s 2024-06-06T03:40:21.077 INFO:tasks.workunit.client.0.smithi040.stderr: 84700K .......... .......... .......... .......... .......... 62% 34.3M 2s 2024-06-06T03:40:21.080 INFO:tasks.workunit.client.0.smithi040.stderr: 84750K .......... .......... .......... .......... .......... 62% 18.2M 2s 2024-06-06T03:40:21.083 INFO:tasks.workunit.client.0.smithi040.stderr: 84800K .......... .......... .......... .......... .......... 62% 17.9M 2s 2024-06-06T03:40:21.084 INFO:tasks.workunit.client.0.smithi040.stderr: 84850K .......... .......... .......... .......... .......... 62% 25.5M 2s 2024-06-06T03:40:21.087 INFO:tasks.workunit.client.0.smithi040.stderr: 84900K .......... .......... .......... .......... .......... 62% 15.7M 2s 2024-06-06T03:40:21.090 INFO:tasks.workunit.client.0.smithi040.stderr: 84950K .......... .......... .......... .......... .......... 62% 33.5M 2s 2024-06-06T03:40:21.092 INFO:tasks.workunit.client.0.smithi040.stderr: 85000K .......... .......... .......... .......... .......... 62% 18.9M 2s 2024-06-06T03:40:21.094 INFO:tasks.workunit.client.0.smithi040.stderr: 85050K .......... .......... .......... .......... .......... 62% 16.8M 2s 2024-06-06T03:40:21.097 INFO:tasks.workunit.client.0.smithi040.stderr: 85100K .......... .......... .......... .......... .......... 62% 19.7M 2s 2024-06-06T03:40:21.100 INFO:tasks.workunit.client.0.smithi040.stderr: 85150K .......... .......... .......... .......... .......... 62% 15.9M 2s 2024-06-06T03:40:21.101 INFO:tasks.workunit.client.0.smithi040.stderr: 85200K .......... .......... .......... .......... .......... 62% 43.1M 2s 2024-06-06T03:40:21.103 INFO:tasks.workunit.client.0.smithi040.stderr: 85250K .......... .......... .......... .......... .......... 62% 22.4M 2s 2024-06-06T03:40:21.108 INFO:tasks.workunit.client.0.smithi040.stderr: 85300K .......... .......... .......... .......... .......... 62% 16.9M 2s 2024-06-06T03:40:21.108 INFO:tasks.workunit.client.0.smithi040.stderr: 85350K .......... .......... .......... .......... .......... 62% 54.8M 2s 2024-06-06T03:40:21.112 INFO:tasks.workunit.client.0.smithi040.stderr: 85400K .......... .......... .......... .......... .......... 62% 28.1M 2s 2024-06-06T03:40:21.112 INFO:tasks.workunit.client.0.smithi040.stderr: 85450K .......... .......... .......... .......... .......... 62% 42.6M 2s 2024-06-06T03:40:21.112 INFO:tasks.workunit.client.0.smithi040.stderr: 85500K .......... .......... .......... .......... .......... 63% 26.3M 2s 2024-06-06T03:40:21.115 INFO:tasks.workunit.client.0.smithi040.stderr: 85550K .......... .......... .......... .......... .......... 63% 21.2M 2s 2024-06-06T03:40:21.116 INFO:tasks.workunit.client.0.smithi040.stderr: 85600K .......... .......... .......... .......... .......... 63% 34.4M 2s 2024-06-06T03:40:21.121 INFO:tasks.workunit.client.0.smithi040.stderr: 85650K .......... .......... .......... .......... .......... 63% 22.7M 2s 2024-06-06T03:40:21.121 INFO:tasks.workunit.client.0.smithi040.stderr: 85700K .......... .......... .......... .......... .......... 63% 46.3M 2s 2024-06-06T03:40:21.121 INFO:tasks.workunit.client.0.smithi040.stderr: 85750K .......... .......... .......... .......... .......... 63% 21.4M 2s 2024-06-06T03:40:21.134 INFO:tasks.workunit.client.0.smithi040.stderr: 85800K .......... .......... .......... .......... .......... 63% 24.8M 2s 2024-06-06T03:40:21.134 INFO:tasks.workunit.client.0.smithi040.stderr: 85850K .......... .......... .......... .......... .......... 63% 29.5M 2s 2024-06-06T03:40:21.134 INFO:tasks.workunit.client.0.smithi040.stderr: 85900K .......... .......... .......... .......... .......... 63% 24.7M 2s 2024-06-06T03:40:21.134 INFO:tasks.workunit.client.0.smithi040.stderr: 85950K .......... .......... .......... .......... .......... 63% 28.5M 2s 2024-06-06T03:40:21.134 INFO:tasks.workunit.client.0.smithi040.stderr: 86000K .......... .......... .......... .......... .......... 63% 25.9M 2s 2024-06-06T03:40:21.134 INFO:tasks.workunit.client.0.smithi040.stderr: 86050K .......... .......... .......... .......... .......... 63% 39.8M 2s 2024-06-06T03:40:21.143 INFO:tasks.workunit.client.0.smithi040.stderr: 86100K .......... .......... .......... .......... .......... 63% 8.76M 2s 2024-06-06T03:40:21.143 INFO:tasks.workunit.client.0.smithi040.stderr: 86150K .......... .......... .......... .......... .......... 63% 64.0M 2s 2024-06-06T03:40:21.143 INFO:tasks.workunit.client.0.smithi040.stderr: 86200K .......... .......... .......... .......... .......... 63% 42.7M 2s 2024-06-06T03:40:21.143 INFO:tasks.workunit.client.0.smithi040.stderr: 86250K .......... .......... .......... .......... .......... 63% 35.0M 2s 2024-06-06T03:40:21.143 INFO:tasks.workunit.client.0.smithi040.stderr: 86300K .......... .......... .......... .......... .......... 63% 23.9M 2s 2024-06-06T03:40:21.144 INFO:tasks.workunit.client.0.smithi040.stderr: 86350K .......... .......... .......... .......... .......... 63% 31.3M 2s 2024-06-06T03:40:21.147 INFO:tasks.workunit.client.0.smithi040.stderr: 86400K .......... .......... .......... .......... .......... 63% 16.6M 2s 2024-06-06T03:40:21.149 INFO:tasks.workunit.client.0.smithi040.stderr: 86450K .......... .......... .......... .......... .......... 63% 49.1M 2s 2024-06-06T03:40:21.150 INFO:tasks.workunit.client.0.smithi040.stderr: 86500K .......... .......... .......... .......... .......... 63% 22.2M 2s 2024-06-06T03:40:21.153 INFO:tasks.workunit.client.0.smithi040.stderr: 86550K .......... .......... .......... .......... .......... 63% 21.0M 2s 2024-06-06T03:40:21.157 INFO:tasks.workunit.client.0.smithi040.stderr: 86600K .......... .......... .......... .......... .......... 63% 19.1M 2s 2024-06-06T03:40:21.157 INFO:tasks.workunit.client.0.smithi040.stderr: 86650K .......... .......... .......... .......... .......... 63% 23.6M 2s 2024-06-06T03:40:21.160 INFO:tasks.workunit.client.0.smithi040.stderr: 86700K .......... .......... .......... .......... .......... 63% 17.2M 2s 2024-06-06T03:40:21.162 INFO:tasks.workunit.client.0.smithi040.stderr: 86750K .......... .......... .......... .......... .......... 63% 21.7M 2s 2024-06-06T03:40:21.164 INFO:tasks.workunit.client.0.smithi040.stderr: 86800K .......... .......... .......... .......... .......... 63% 22.7M 2s 2024-06-06T03:40:21.167 INFO:tasks.workunit.client.0.smithi040.stderr: 86850K .......... .......... .......... .......... .......... 64% 15.8M 2s 2024-06-06T03:40:21.169 INFO:tasks.workunit.client.0.smithi040.stderr: 86900K .......... .......... .......... .......... .......... 64% 28.5M 2s 2024-06-06T03:40:21.173 INFO:tasks.workunit.client.0.smithi040.stderr: 86950K .......... .......... .......... .......... .......... 64% 13.8M 2s 2024-06-06T03:40:21.174 INFO:tasks.workunit.client.0.smithi040.stderr: 87000K .......... .......... .......... .......... .......... 64% 37.1M 2s 2024-06-06T03:40:21.176 INFO:tasks.workunit.client.0.smithi040.stderr: 87050K .......... .......... .......... .......... .......... 64% 19.3M 2s 2024-06-06T03:40:21.178 INFO:tasks.workunit.client.0.smithi040.stderr: 87100K .......... .......... .......... .......... .......... 64% 28.9M 2s 2024-06-06T03:40:21.185 INFO:tasks.workunit.client.0.smithi040.stderr: 87150K .......... .......... .......... .......... .......... 64% 22.7M 2s 2024-06-06T03:40:21.185 INFO:tasks.workunit.client.0.smithi040.stderr: 87200K .......... .......... .......... .......... .......... 64% 28.3M 2s 2024-06-06T03:40:21.185 INFO:tasks.workunit.client.0.smithi040.stderr: 87250K .......... .......... .......... .......... .......... 64% 39.1M 2s 2024-06-06T03:40:21.185 INFO:tasks.workunit.client.0.smithi040.stderr: 87300K .......... .......... .......... .......... .......... 64% 24.2M 2s 2024-06-06T03:40:21.188 INFO:tasks.workunit.client.0.smithi040.stderr: 87350K .......... .......... .......... .......... .......... 64% 24.0M 2s 2024-06-06T03:40:21.191 INFO:tasks.workunit.client.0.smithi040.stderr: 87400K .......... .......... .......... .......... .......... 64% 29.2M 2s 2024-06-06T03:40:21.191 INFO:tasks.workunit.client.0.smithi040.stderr: 87450K .......... .......... .......... .......... .......... 64% 23.5M 2s 2024-06-06T03:40:21.192 INFO:tasks.workunit.client.0.smithi040.stderr: 87500K .......... .......... .......... .......... .......... 64% 36.5M 2s 2024-06-06T03:40:21.196 INFO:tasks.workunit.client.0.smithi040.stderr: 87550K .......... .......... .......... .......... .......... 64% 30.8M 2s 2024-06-06T03:40:21.196 INFO:tasks.workunit.client.0.smithi040.stderr: 87600K .......... .......... .......... .......... .......... 64% 28.8M 2s 2024-06-06T03:40:21.198 INFO:tasks.workunit.client.0.smithi040.stderr: 87650K .......... .......... .......... .......... .......... 64% 21.3M 2s 2024-06-06T03:40:21.201 INFO:tasks.workunit.client.0.smithi040.stderr: 87700K .......... .......... .......... .......... .......... 64% 20.9M 2s 2024-06-06T03:40:21.201 INFO:tasks.workunit.client.0.smithi040.stderr: 87750K .......... .......... .......... .......... .......... 64% 44.6M 2s 2024-06-06T03:40:21.203 INFO:tasks.workunit.client.0.smithi040.stderr: 87800K .......... .......... .......... .......... .......... 64% 29.4M 2s 2024-06-06T03:40:21.206 INFO:tasks.workunit.client.0.smithi040.stderr: 87850K .......... .......... .......... .......... .......... 64% 31.0M 2s 2024-06-06T03:40:21.215 INFO:tasks.workunit.client.0.smithi040.stderr: 87900K .......... .......... .......... .......... .......... 64% 16.7M 2s 2024-06-06T03:40:21.215 INFO:tasks.workunit.client.0.smithi040.stderr: 87950K .......... .......... .......... .......... .......... 64% 25.1M 2s 2024-06-06T03:40:21.215 INFO:tasks.workunit.client.0.smithi040.stderr: 88000K .......... .......... .......... .......... .......... 64% 25.9M 2s 2024-06-06T03:40:21.216 INFO:tasks.workunit.client.0.smithi040.stderr: 88050K .......... .......... .......... .......... .......... 64% 26.7M 2s 2024-06-06T03:40:21.217 INFO:tasks.workunit.client.0.smithi040.stderr: 88100K .......... .......... .......... .......... .......... 64% 14.8M 2s 2024-06-06T03:40:21.220 INFO:tasks.workunit.client.0.smithi040.stderr: 88150K .......... .......... .......... .......... .......... 64% 40.8M 2s 2024-06-06T03:40:21.220 INFO:tasks.workunit.client.0.smithi040.stderr: 88200K .......... .......... .......... .......... .......... 65% 24.3M 2s 2024-06-06T03:40:21.221 INFO:tasks.workunit.client.0.smithi040.stderr: 88250K .......... .......... .......... .......... .......... 65% 36.6M 2s 2024-06-06T03:40:21.223 INFO:tasks.workunit.client.0.smithi040.stderr: 88300K .......... .......... .......... .......... .......... 65% 22.9M 2s 2024-06-06T03:40:21.226 INFO:tasks.workunit.client.0.smithi040.stderr: 88350K .......... .......... .......... .......... .......... 65% 16.9M 2s 2024-06-06T03:40:21.228 INFO:tasks.workunit.client.0.smithi040.stderr: 88400K .......... .......... .......... .......... .......... 65% 27.6M 2s 2024-06-06T03:40:21.230 INFO:tasks.workunit.client.0.smithi040.stderr: 88450K .......... .......... .......... .......... .......... 65% 21.7M 2s 2024-06-06T03:40:21.232 INFO:tasks.workunit.client.0.smithi040.stderr: 88500K .......... .......... .......... .......... .......... 65% 28.5M 2s 2024-06-06T03:40:21.235 INFO:tasks.workunit.client.0.smithi040.stderr: 88550K .......... .......... .......... .......... .......... 65% 16.8M 2s 2024-06-06T03:40:21.238 INFO:tasks.workunit.client.0.smithi040.stderr: 88600K .......... .......... .......... .......... .......... 65% 15.7M 2s 2024-06-06T03:40:21.239 INFO:tasks.workunit.client.0.smithi040.stderr: 88650K .......... .......... .......... .......... .......... 65% 32.1M 2s 2024-06-06T03:40:21.242 INFO:tasks.workunit.client.0.smithi040.stderr: 88700K .......... .......... .......... .......... .......... 65% 19.1M 2s 2024-06-06T03:40:21.243 INFO:tasks.workunit.client.0.smithi040.stderr: 88750K .......... .......... .......... .......... .......... 65% 27.2M 2s 2024-06-06T03:40:21.247 INFO:tasks.workunit.client.0.smithi040.stderr: 88800K .......... .......... .......... .......... .......... 65% 14.6M 2s 2024-06-06T03:40:21.250 INFO:tasks.workunit.client.0.smithi040.stderr: 88850K .......... .......... .......... .......... .......... 65% 22.2M 2s 2024-06-06T03:40:21.250 INFO:tasks.workunit.client.0.smithi040.stderr: 88900K .......... .......... .......... .......... .......... 65% 43.1M 2s 2024-06-06T03:40:21.252 INFO:tasks.workunit.client.0.smithi040.stderr: 88950K .......... .......... .......... .......... .......... 65% 25.6M 2s 2024-06-06T03:40:21.254 INFO:tasks.workunit.client.0.smithi040.stderr: 89000K .......... .......... .......... .......... .......... 65% 28.5M 2s 2024-06-06T03:40:21.257 INFO:tasks.workunit.client.0.smithi040.stderr: 89050K .......... .......... .......... .......... .......... 65% 20.5M 2s 2024-06-06T03:40:21.259 INFO:tasks.workunit.client.0.smithi040.stderr: 89100K .......... .......... .......... .......... .......... 65% 22.1M 2s 2024-06-06T03:40:21.260 INFO:tasks.workunit.client.0.smithi040.stderr: 89150K .......... .......... .......... .......... .......... 65% 39.5M 2s 2024-06-06T03:40:21.267 INFO:tasks.workunit.client.0.smithi040.stderr: 89200K .......... .......... .......... .......... .......... 65% 25.4M 2s 2024-06-06T03:40:21.267 INFO:tasks.workunit.client.0.smithi040.stderr: 89250K .......... .......... .......... .......... .......... 65% 42.3M 2s 2024-06-06T03:40:21.268 INFO:tasks.workunit.client.0.smithi040.stderr: 89300K .......... .......... .......... .......... .......... 65% 20.7M 2s 2024-06-06T03:40:21.268 INFO:tasks.workunit.client.0.smithi040.stderr: 89350K .......... .......... .......... .......... .......... 65% 24.2M 2s 2024-06-06T03:40:21.269 INFO:tasks.workunit.client.0.smithi040.stderr: 89400K .......... .......... .......... .......... .......... 65% 31.5M 2s 2024-06-06T03:40:21.271 INFO:tasks.workunit.client.0.smithi040.stderr: 89450K .......... .......... .......... .......... .......... 65% 22.7M 2s 2024-06-06T03:40:21.277 INFO:tasks.workunit.client.0.smithi040.stderr: 89500K .......... .......... .......... .......... .......... 65% 31.6M 2s 2024-06-06T03:40:21.278 INFO:tasks.workunit.client.0.smithi040.stderr: 89550K .......... .......... .......... .......... .......... 66% 26.0M 2s 2024-06-06T03:40:21.278 INFO:tasks.workunit.client.0.smithi040.stderr: 89600K .......... .......... .......... .......... .......... 66% 23.1M 2s 2024-06-06T03:40:21.278 INFO:tasks.workunit.client.0.smithi040.stderr: 89650K .......... .......... .......... .......... .......... 66% 39.7M 2s 2024-06-06T03:40:21.280 INFO:tasks.workunit.client.0.smithi040.stderr: 89700K .......... .......... .......... .......... .......... 66% 18.3M 2s 2024-06-06T03:40:21.281 INFO:tasks.workunit.client.0.smithi040.stderr: 89750K .......... .......... .......... .......... .......... 66% 38.2M 2s 2024-06-06T03:40:21.284 INFO:tasks.workunit.client.0.smithi040.stderr: 89800K .......... .......... .......... .......... .......... 66% 17.7M 2s 2024-06-06T03:40:21.286 INFO:tasks.workunit.client.0.smithi040.stderr: 89850K .......... .......... .......... .......... .......... 66% 25.6M 2s 2024-06-06T03:40:21.288 INFO:tasks.workunit.client.0.smithi040.stderr: 89900K .......... .......... .......... .......... .......... 66% 34.2M 2s 2024-06-06T03:40:21.300 INFO:tasks.workunit.client.0.smithi040.stderr: 89950K .......... .......... .......... .......... .......... 66% 16.8M 2s 2024-06-06T03:40:21.300 INFO:tasks.workunit.client.0.smithi040.stderr: 90000K .......... .......... .......... .......... .......... 66% 39.8M 2s 2024-06-06T03:40:21.300 INFO:tasks.workunit.client.0.smithi040.stderr: 90050K .......... .......... .......... .......... .......... 66% 24.7M 2s 2024-06-06T03:40:21.300 INFO:tasks.workunit.client.0.smithi040.stderr: 90100K .......... .......... .......... .......... .......... 66% 17.2M 2s 2024-06-06T03:40:21.300 INFO:tasks.workunit.client.0.smithi040.stderr: 90150K .......... .......... .......... .......... .......... 66% 28.3M 2s 2024-06-06T03:40:21.301 INFO:tasks.workunit.client.0.smithi040.stderr: 90200K .......... .......... .......... .......... .......... 66% 17.7M 2s 2024-06-06T03:40:21.303 INFO:tasks.workunit.client.0.smithi040.stderr: 90250K .......... .......... .......... .......... .......... 66% 40.5M 2s 2024-06-06T03:40:21.304 INFO:tasks.workunit.client.0.smithi040.stderr: 90300K .......... .......... .......... .......... .......... 66% 23.0M 2s 2024-06-06T03:40:21.307 INFO:tasks.workunit.client.0.smithi040.stderr: 90350K .......... .......... .......... .......... .......... 66% 16.8M 2s 2024-06-06T03:40:21.309 INFO:tasks.workunit.client.0.smithi040.stderr: 90400K .......... .......... .......... .......... .......... 66% 27.4M 2s 2024-06-06T03:40:21.318 INFO:tasks.workunit.client.0.smithi040.stderr: 90450K .......... .......... .......... .......... .......... 66% 14.5M 2s 2024-06-06T03:40:21.318 INFO:tasks.workunit.client.0.smithi040.stderr: 90500K .......... .......... .......... .......... .......... 66% 39.0M 2s 2024-06-06T03:40:21.318 INFO:tasks.workunit.client.0.smithi040.stderr: 90550K .......... .......... .......... .......... .......... 66% 21.6M 2s 2024-06-06T03:40:21.319 INFO:tasks.workunit.client.0.smithi040.stderr: 90600K .......... .......... .......... .......... .......... 66% 21.9M 2s 2024-06-06T03:40:21.321 INFO:tasks.workunit.client.0.smithi040.stderr: 90650K .......... .......... .......... .......... .......... 66% 14.9M 2s 2024-06-06T03:40:21.324 INFO:tasks.workunit.client.0.smithi040.stderr: 90700K .......... .......... .......... .......... .......... 66% 19.2M 2s 2024-06-06T03:40:21.325 INFO:tasks.workunit.client.0.smithi040.stderr: 90750K .......... .......... .......... .......... .......... 66% 49.0M 2s 2024-06-06T03:40:21.328 INFO:tasks.workunit.client.0.smithi040.stderr: 90800K .......... .......... .......... .......... .......... 66% 19.8M 2s 2024-06-06T03:40:21.332 INFO:tasks.workunit.client.0.smithi040.stderr: 90850K .......... .......... .......... .......... .......... 66% 26.7M 2s 2024-06-06T03:40:21.332 INFO:tasks.workunit.client.0.smithi040.stderr: 90900K .......... .......... .......... .......... .......... 66% 32.7M 2s 2024-06-06T03:40:21.338 INFO:tasks.workunit.client.0.smithi040.stderr: 90950K .......... .......... .......... .......... .......... 67% 24.1M 2s 2024-06-06T03:40:21.338 INFO:tasks.workunit.client.0.smithi040.stderr: 91000K .......... .......... .......... .......... .......... 67% 39.0M 2s 2024-06-06T03:40:21.338 INFO:tasks.workunit.client.0.smithi040.stderr: 91050K .......... .......... .......... .......... .......... 67% 27.7M 2s 2024-06-06T03:40:21.351 INFO:tasks.workunit.client.0.smithi040.stderr: 91100K .......... .......... .......... .......... .......... 67% 23.8M 2s 2024-06-06T03:40:21.351 INFO:tasks.workunit.client.0.smithi040.stderr: 91150K .......... .......... .......... .......... .......... 67% 24.5M 2s 2024-06-06T03:40:21.351 INFO:tasks.workunit.client.0.smithi040.stderr: 91200K .......... .......... .......... .......... .......... 67% 22.8M 2s 2024-06-06T03:40:21.351 INFO:tasks.workunit.client.0.smithi040.stderr: 91250K .......... .......... .......... .......... .......... 67% 48.5M 2s 2024-06-06T03:40:21.351 INFO:tasks.workunit.client.0.smithi040.stderr: 91300K .......... .......... .......... .......... .......... 67% 18.2M 2s 2024-06-06T03:40:21.351 INFO:tasks.workunit.client.0.smithi040.stderr: 91350K .......... .......... .......... .......... .......... 67% 33.7M 2s 2024-06-06T03:40:21.351 INFO:tasks.workunit.client.0.smithi040.stderr: 91400K .......... .......... .......... .......... .......... 67% 21.7M 2s 2024-06-06T03:40:21.351 INFO:tasks.workunit.client.0.smithi040.stderr: 91450K .......... .......... .......... .......... .......... 67% 47.7M 2s 2024-06-06T03:40:21.354 INFO:tasks.workunit.client.0.smithi040.stderr: 91500K .......... .......... .......... .......... .......... 67% 32.8M 2s 2024-06-06T03:40:21.355 INFO:tasks.workunit.client.0.smithi040.stderr: 91550K .......... .......... .......... .......... .......... 67% 17.0M 2s 2024-06-06T03:40:21.358 INFO:tasks.workunit.client.0.smithi040.stderr: 91600K .......... .......... .......... .......... .......... 67% 16.8M 2s 2024-06-06T03:40:21.360 INFO:tasks.workunit.client.0.smithi040.stderr: 91650K .......... .......... .......... .......... .......... 67% 32.9M 2s 2024-06-06T03:40:21.361 INFO:tasks.workunit.client.0.smithi040.stderr: 91700K .......... .......... .......... .......... .......... 67% 34.3M 2s 2024-06-06T03:40:21.363 INFO:tasks.workunit.client.0.smithi040.stderr: 91750K .......... .......... .......... .......... .......... 67% 34.5M 2s 2024-06-06T03:40:21.366 INFO:tasks.workunit.client.0.smithi040.stderr: 91800K .......... .......... .......... .......... .......... 67% 16.6M 2s 2024-06-06T03:40:21.367 INFO:tasks.workunit.client.0.smithi040.stderr: 91850K .......... .......... .......... .......... .......... 67% 28.3M 2s 2024-06-06T03:40:21.372 INFO:tasks.workunit.client.0.smithi040.stderr: 91900K .......... .......... .......... .......... .......... 67% 26.8M 2s 2024-06-06T03:40:21.372 INFO:tasks.workunit.client.0.smithi040.stderr: 91950K .......... .......... .......... .......... .......... 67% 24.8M 2s 2024-06-06T03:40:21.373 INFO:tasks.workunit.client.0.smithi040.stderr: 92000K .......... .......... .......... .......... .......... 67% 18.9M 2s 2024-06-06T03:40:21.378 INFO:tasks.workunit.client.0.smithi040.stderr: 92050K .......... .......... .......... .......... .......... 67% 22.9M 2s 2024-06-06T03:40:21.378 INFO:tasks.workunit.client.0.smithi040.stderr: 92100K .......... .......... .......... .......... .......... 67% 25.4M 2s 2024-06-06T03:40:21.379 INFO:tasks.workunit.client.0.smithi040.stderr: 92150K .......... .......... .......... .......... .......... 67% 19.2M 2s 2024-06-06T03:40:21.385 INFO:tasks.workunit.client.0.smithi040.stderr: 92200K .......... .......... .......... .......... .......... 67% 31.5M 2s 2024-06-06T03:40:21.385 INFO:tasks.workunit.client.0.smithi040.stderr: 92250K .......... .......... .......... .......... .......... 67% 18.0M 2s 2024-06-06T03:40:21.389 INFO:tasks.workunit.client.0.smithi040.stderr: 92300K .......... .......... .......... .......... .......... 68% 22.7M 2s 2024-06-06T03:40:21.389 INFO:tasks.workunit.client.0.smithi040.stderr: 92350K .......... .......... .......... .......... .......... 68% 22.4M 2s 2024-06-06T03:40:21.391 INFO:tasks.workunit.client.0.smithi040.stderr: 92400K .......... .......... .......... .......... .......... 68% 19.6M 2s 2024-06-06T03:40:21.394 INFO:tasks.workunit.client.0.smithi040.stderr: 92450K .......... .......... .......... .......... .......... 68% 22.3M 2s 2024-06-06T03:40:21.396 INFO:tasks.workunit.client.0.smithi040.stderr: 92500K .......... .......... .......... .......... .......... 68% 14.6M 2s 2024-06-06T03:40:21.398 INFO:tasks.workunit.client.0.smithi040.stderr: 92550K .......... .......... .......... .......... .......... 68% 39.4M 2s 2024-06-06T03:40:21.400 INFO:tasks.workunit.client.0.smithi040.stderr: 92600K .......... .......... .......... .......... .......... 68% 23.8M 2s 2024-06-06T03:40:21.403 INFO:tasks.workunit.client.0.smithi040.stderr: 92650K .......... .......... .......... .......... .......... 68% 25.0M 2s 2024-06-06T03:40:21.403 INFO:tasks.workunit.client.0.smithi040.stderr: 92700K .......... .......... .......... .......... .......... 68% 24.1M 2s 2024-06-06T03:40:21.406 INFO:tasks.workunit.client.0.smithi040.stderr: 92750K .......... .......... .......... .......... .......... 68% 20.6M 2s 2024-06-06T03:40:21.407 INFO:tasks.workunit.client.0.smithi040.stderr: 92800K .......... .......... .......... .......... .......... 68% 44.5M 2s 2024-06-06T03:40:21.409 INFO:tasks.workunit.client.0.smithi040.stderr: 92850K .......... .......... .......... .......... .......... 68% 23.4M 2s 2024-06-06T03:40:21.410 INFO:tasks.workunit.client.0.smithi040.stderr: 92900K .......... .......... .......... .......... .......... 68% 43.2M 2s 2024-06-06T03:40:21.413 INFO:tasks.workunit.client.0.smithi040.stderr: 92950K .......... .......... .......... .......... .......... 68% 20.9M 2s 2024-06-06T03:40:21.414 INFO:tasks.workunit.client.0.smithi040.stderr: 93000K .......... .......... .......... .......... .......... 68% 25.0M 2s 2024-06-06T03:40:21.416 INFO:tasks.workunit.client.0.smithi040.stderr: 93050K .......... .......... .......... .......... .......... 68% 34.6M 2s 2024-06-06T03:40:21.419 INFO:tasks.workunit.client.0.smithi040.stderr: 93100K .......... .......... .......... .......... .......... 68% 20.1M 2s 2024-06-06T03:40:21.421 INFO:tasks.workunit.client.0.smithi040.stderr: 93150K .......... .......... .......... .......... .......... 68% 38.5M 2s 2024-06-06T03:40:21.421 INFO:tasks.workunit.client.0.smithi040.stderr: 93200K .......... .......... .......... .......... .......... 68% 24.6M 2s 2024-06-06T03:40:21.425 INFO:tasks.workunit.client.0.smithi040.stderr: 93250K .......... .......... .......... .......... .......... 68% 28.8M 2s 2024-06-06T03:40:21.425 INFO:tasks.workunit.client.0.smithi040.stderr: 93300K .......... .......... .......... .......... .......... 68% 33.0M 2s 2024-06-06T03:40:21.430 INFO:tasks.workunit.client.0.smithi040.stderr: 93350K .......... .......... .......... .......... .......... 68% 17.2M 2s 2024-06-06T03:40:21.430 INFO:tasks.workunit.client.0.smithi040.stderr: 93400K .......... .......... .......... .......... .......... 68% 28.7M 2s 2024-06-06T03:40:21.436 INFO:tasks.workunit.client.0.smithi040.stderr: 93450K .......... .......... .......... .......... .......... 68% 20.7M 2s 2024-06-06T03:40:21.436 INFO:tasks.workunit.client.0.smithi040.stderr: 93500K .......... .......... .......... .......... .......... 68% 23.7M 2s 2024-06-06T03:40:21.436 INFO:tasks.workunit.client.0.smithi040.stderr: 93550K .......... .......... .......... .......... .......... 68% 44.9M 2s 2024-06-06T03:40:21.439 INFO:tasks.workunit.client.0.smithi040.stderr: 93600K .......... .......... .......... .......... .......... 68% 19.4M 2s 2024-06-06T03:40:21.440 INFO:tasks.workunit.client.0.smithi040.stderr: 93650K .......... .......... .......... .......... .......... 69% 19.0M 2s 2024-06-06T03:40:21.442 INFO:tasks.workunit.client.0.smithi040.stderr: 93700K .......... .......... .......... .......... .......... 69% 36.0M 2s 2024-06-06T03:40:21.443 INFO:tasks.workunit.client.0.smithi040.stderr: 93750K .......... .......... .......... .......... .......... 69% 25.0M 2s 2024-06-06T03:40:21.445 INFO:tasks.workunit.client.0.smithi040.stderr: 93800K .......... .......... .......... .......... .......... 69% 28.5M 2s 2024-06-06T03:40:21.448 INFO:tasks.workunit.client.0.smithi040.stderr: 93850K .......... .......... .......... .......... .......... 69% 14.3M 2s 2024-06-06T03:40:21.451 INFO:tasks.workunit.client.0.smithi040.stderr: 93900K .......... .......... .......... .......... .......... 69% 33.4M 2s 2024-06-06T03:40:21.452 INFO:tasks.workunit.client.0.smithi040.stderr: 93950K .......... .......... .......... .......... .......... 69% 26.1M 2s 2024-06-06T03:40:21.456 INFO:tasks.workunit.client.0.smithi040.stderr: 94000K .......... .......... .......... .......... .......... 69% 17.8M 2s 2024-06-06T03:40:21.456 INFO:tasks.workunit.client.0.smithi040.stderr: 94050K .......... .......... .......... .......... .......... 69% 32.0M 2s 2024-06-06T03:40:21.458 INFO:tasks.workunit.client.0.smithi040.stderr: 94100K .......... .......... .......... .......... .......... 69% 22.4M 2s 2024-06-06T03:40:21.463 INFO:tasks.workunit.client.0.smithi040.stderr: 94150K .......... .......... .......... .......... .......... 69% 15.2M 2s 2024-06-06T03:40:21.463 INFO:tasks.workunit.client.0.smithi040.stderr: 94200K .......... .......... .......... .......... .......... 69% 26.6M 2s 2024-06-06T03:40:21.467 INFO:tasks.workunit.client.0.smithi040.stderr: 94250K .......... .......... .......... .......... .......... 69% 20.2M 2s 2024-06-06T03:40:21.468 INFO:tasks.workunit.client.0.smithi040.stderr: 94300K .......... .......... .......... .......... .......... 69% 32.2M 2s 2024-06-06T03:40:21.471 INFO:tasks.workunit.client.0.smithi040.stderr: 94350K .......... .......... .......... .......... .......... 69% 15.8M 2s 2024-06-06T03:40:21.475 INFO:tasks.workunit.client.0.smithi040.stderr: 94400K .......... .......... .......... .......... .......... 69% 17.1M 2s 2024-06-06T03:40:21.475 INFO:tasks.workunit.client.0.smithi040.stderr: 94450K .......... .......... .......... .......... .......... 69% 30.6M 2s 2024-06-06T03:40:21.478 INFO:tasks.workunit.client.0.smithi040.stderr: 94500K .......... .......... .......... .......... .......... 69% 22.2M 2s 2024-06-06T03:40:21.478 INFO:tasks.workunit.client.0.smithi040.stderr: 94550K .......... .......... .......... .......... .......... 69% 42.6M 2s 2024-06-06T03:40:21.480 INFO:tasks.workunit.client.0.smithi040.stderr: 94600K .......... .......... .......... .......... .......... 69% 20.6M 2s 2024-06-06T03:40:21.485 INFO:tasks.workunit.client.0.smithi040.stderr: 94650K .......... .......... .......... .......... .......... 69% 26.7M 2s 2024-06-06T03:40:21.485 INFO:tasks.workunit.client.0.smithi040.stderr: 94700K .......... .......... .......... .......... .......... 69% 39.2M 2s 2024-06-06T03:40:21.487 INFO:tasks.workunit.client.0.smithi040.stderr: 94750K .......... .......... .......... .......... .......... 69% 20.4M 2s 2024-06-06T03:40:21.487 INFO:tasks.workunit.client.0.smithi040.stderr: 94800K .......... .......... .......... .......... .......... 69% 39.8M 2s 2024-06-06T03:40:21.490 INFO:tasks.workunit.client.0.smithi040.stderr: 94850K .......... .......... .......... .......... .......... 69% 22.1M 2s 2024-06-06T03:40:21.491 INFO:tasks.workunit.client.0.smithi040.stderr: 94900K .......... .......... .......... .......... .......... 69% 39.9M 2s 2024-06-06T03:40:21.493 INFO:tasks.workunit.client.0.smithi040.stderr: 94950K .......... .......... .......... .......... .......... 69% 19.9M 2s 2024-06-06T03:40:21.497 INFO:tasks.workunit.client.0.smithi040.stderr: 95000K .......... .......... .......... .......... .......... 70% 25.3M 2s 2024-06-06T03:40:21.497 INFO:tasks.workunit.client.0.smithi040.stderr: 95050K .......... .......... .......... .......... .......... 70% 37.3M 2s 2024-06-06T03:40:21.498 INFO:tasks.workunit.client.0.smithi040.stderr: 95100K .......... .......... .......... .......... .......... 70% 28.0M 2s 2024-06-06T03:40:21.501 INFO:tasks.workunit.client.0.smithi040.stderr: 95150K .......... .......... .......... .......... .......... 70% 20.3M 2s 2024-06-06T03:40:21.502 INFO:tasks.workunit.client.0.smithi040.stderr: 95200K .......... .......... .......... .......... .......... 70% 27.9M 2s 2024-06-06T03:40:21.505 INFO:tasks.workunit.client.0.smithi040.stderr: 95250K .......... .......... .......... .......... .......... 70% 23.1M 2s 2024-06-06T03:40:21.506 INFO:tasks.workunit.client.0.smithi040.stderr: 95300K .......... .......... .......... .......... .......... 70% 26.0M 2s 2024-06-06T03:40:21.508 INFO:tasks.workunit.client.0.smithi040.stderr: 95350K .......... .......... .......... .......... .......... 70% 24.2M 2s 2024-06-06T03:40:21.512 INFO:tasks.workunit.client.0.smithi040.stderr: 95400K .......... .......... .......... .......... .......... 70% 35.6M 2s 2024-06-06T03:40:21.512 INFO:tasks.workunit.client.0.smithi040.stderr: 95450K .......... .......... .......... .......... .......... 70% 17.7M 2s 2024-06-06T03:40:21.515 INFO:tasks.workunit.client.0.smithi040.stderr: 95500K .......... .......... .......... .......... .......... 70% 23.1M 2s 2024-06-06T03:40:21.520 INFO:tasks.workunit.client.0.smithi040.stderr: 95550K .......... .......... .......... .......... .......... 70% 36.7M 2s 2024-06-06T03:40:21.520 INFO:tasks.workunit.client.0.smithi040.stderr: 95600K .......... .......... .......... .......... .......... 70% 22.4M 2s 2024-06-06T03:40:21.520 INFO:tasks.workunit.client.0.smithi040.stderr: 95650K .......... .......... .......... .......... .......... 70% 25.8M 2s 2024-06-06T03:40:21.522 INFO:tasks.workunit.client.0.smithi040.stderr: 95700K .......... .......... .......... .......... .......... 70% 17.6M 2s 2024-06-06T03:40:21.524 INFO:tasks.workunit.client.0.smithi040.stderr: 95750K .......... .......... .......... .......... .......... 70% 27.2M 2s 2024-06-06T03:40:21.526 INFO:tasks.workunit.client.0.smithi040.stderr: 95800K .......... .......... .......... .......... .......... 70% 39.2M 2s 2024-06-06T03:40:21.529 INFO:tasks.workunit.client.0.smithi040.stderr: 95850K .......... .......... .......... .......... .......... 70% 21.2M 2s 2024-06-06T03:40:21.531 INFO:tasks.workunit.client.0.smithi040.stderr: 95900K .......... .......... .......... .......... .......... 70% 18.3M 2s 2024-06-06T03:40:21.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:21 smithi131 ceph-mon[36687]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-06-06T03:40:21.535 INFO:tasks.workunit.client.0.smithi040.stderr: 95950K .......... .......... .......... .......... .......... 70% 16.3M 2s 2024-06-06T03:40:21.537 INFO:tasks.workunit.client.0.smithi040.stderr: 96000K .......... .......... .......... .......... .......... 70% 22.5M 2s 2024-06-06T03:40:21.537 INFO:tasks.workunit.client.0.smithi040.stderr: 96050K .......... .......... .......... .......... .......... 70% 37.7M 2s 2024-06-06T03:40:21.540 INFO:tasks.workunit.client.0.smithi040.stderr: 96100K .......... .......... .......... .......... .......... 70% 19.1M 2s 2024-06-06T03:40:21.541 INFO:tasks.workunit.client.0.smithi040.stderr: 96150K .......... .......... .......... .......... .......... 70% 26.8M 2s 2024-06-06T03:40:21.544 INFO:tasks.workunit.client.0.smithi040.stderr: 96200K .......... .......... .......... .......... .......... 70% 20.9M 2s 2024-06-06T03:40:21.547 INFO:tasks.workunit.client.0.smithi040.stderr: 96250K .......... .......... .......... .......... .......... 70% 16.6M 2s 2024-06-06T03:40:21.548 INFO:tasks.workunit.client.0.smithi040.stderr: 96300K .......... .......... .......... .......... .......... 70% 26.7M 2s 2024-06-06T03:40:21.551 INFO:tasks.workunit.client.0.smithi040.stderr: 96350K .......... .......... .......... .......... .......... 71% 20.3M 2s 2024-06-06T03:40:21.554 INFO:tasks.workunit.client.0.smithi040.stderr: 96400K .......... .......... .......... .......... .......... 71% 24.6M 2s 2024-06-06T03:40:21.563 INFO:tasks.workunit.client.0.smithi040.stderr: 96450K .......... .......... .......... .......... .......... 71% 22.1M 2s 2024-06-06T03:40:21.564 INFO:tasks.workunit.client.0.smithi040.stderr: 96500K .......... .......... .......... .......... .......... 71% 28.9M 2s 2024-06-06T03:40:21.564 INFO:tasks.workunit.client.0.smithi040.stderr: 96550K .......... .......... .......... .......... .......... 71% 32.1M 2s 2024-06-06T03:40:21.564 INFO:tasks.workunit.client.0.smithi040.stderr: 96600K .......... .......... .......... .......... .......... 71% 22.9M 2s 2024-06-06T03:40:21.564 INFO:tasks.workunit.client.0.smithi040.stderr: 96650K .......... .......... .......... .......... .......... 71% 33.0M 2s 2024-06-06T03:40:21.565 INFO:tasks.workunit.client.0.smithi040.stderr: 96700K .......... .......... .......... .......... .......... 71% 27.3M 2s 2024-06-06T03:40:21.567 INFO:tasks.workunit.client.0.smithi040.stderr: 96750K .......... .......... .......... .......... .......... 71% 17.9M 2s 2024-06-06T03:40:21.570 INFO:tasks.workunit.client.0.smithi040.stderr: 96800K .......... .......... .......... .......... .......... 71% 39.5M 2s 2024-06-06T03:40:21.570 INFO:tasks.workunit.client.0.smithi040.stderr: 96850K .......... .......... .......... .......... .......... 71% 27.4M 2s 2024-06-06T03:40:21.573 INFO:tasks.workunit.client.0.smithi040.stderr: 96900K .......... .......... .......... .......... .......... 71% 24.9M 2s 2024-06-06T03:40:21.574 INFO:tasks.workunit.client.0.smithi040.stderr: 96950K .......... .......... .......... .......... .......... 71% 24.5M 2s 2024-06-06T03:40:21.575 INFO:tasks.workunit.client.0.smithi040.stderr: 97000K .......... .......... .......... .......... .......... 71% 27.4M 2s 2024-06-06T03:40:21.577 INFO:tasks.workunit.client.0.smithi040.stderr: 97050K .......... .......... .......... .......... .......... 71% 30.0M 2s 2024-06-06T03:40:21.579 INFO:tasks.workunit.client.0.smithi040.stderr: 97100K .......... .......... .......... .......... .......... 71% 22.1M 2s 2024-06-06T03:40:21.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:21 smithi040 ceph-mon[32220]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-06-06T03:40:21.582 INFO:tasks.workunit.client.0.smithi040.stderr: 97150K .......... .......... .......... .......... .......... 71% 20.2M 2s 2024-06-06T03:40:21.587 INFO:tasks.workunit.client.0.smithi040.stderr: 97200K .......... .......... .......... .......... .......... 71% 31.5M 2s 2024-06-06T03:40:21.587 INFO:tasks.workunit.client.0.smithi040.stderr: 97250K .......... .......... .......... .......... .......... 71% 30.2M 2s 2024-06-06T03:40:21.587 INFO:tasks.workunit.client.0.smithi040.stderr: 97300K .......... .......... .......... .......... .......... 71% 20.9M 2s 2024-06-06T03:40:21.591 INFO:tasks.workunit.client.0.smithi040.stderr: 97350K .......... .......... .......... .......... .......... 71% 21.1M 2s 2024-06-06T03:40:21.591 INFO:tasks.workunit.client.0.smithi040.stderr: 97400K .......... .......... .......... .......... .......... 71% 30.8M 2s 2024-06-06T03:40:21.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:21 smithi104 ceph-mon[36569]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-06-06T03:40:21.593 INFO:tasks.workunit.client.0.smithi040.stderr: 97450K .......... .......... .......... .......... .......... 71% 23.2M 2s 2024-06-06T03:40:21.595 INFO:tasks.workunit.client.0.smithi040.stderr: 97500K .......... .......... .......... .......... .......... 71% 23.7M 2s 2024-06-06T03:40:21.598 INFO:tasks.workunit.client.0.smithi040.stderr: 97550K .......... .......... .......... .......... .......... 71% 17.1M 2s 2024-06-06T03:40:21.602 INFO:tasks.workunit.client.0.smithi040.stderr: 97600K .......... .......... .......... .......... .......... 71% 35.6M 2s 2024-06-06T03:40:21.602 INFO:tasks.workunit.client.0.smithi040.stderr: 97650K .......... .......... .......... .......... .......... 71% 31.5M 2s 2024-06-06T03:40:21.604 INFO:tasks.workunit.client.0.smithi040.stderr: 97700K .......... .......... .......... .......... .......... 72% 23.2M 2s 2024-06-06T03:40:21.605 INFO:tasks.workunit.client.0.smithi040.stderr: 97750K .......... .......... .......... .......... .......... 72% 19.2M 2s 2024-06-06T03:40:21.607 INFO:tasks.workunit.client.0.smithi040.stderr: 97800K .......... .......... .......... .......... .......... 72% 24.3M 2s 2024-06-06T03:40:21.610 INFO:tasks.workunit.client.0.smithi040.stderr: 97850K .......... .......... .......... .......... .......... 72% 21.4M 2s 2024-06-06T03:40:21.612 INFO:tasks.workunit.client.0.smithi040.stderr: 97900K .......... .......... .......... .......... .......... 72% 19.8M 2s 2024-06-06T03:40:21.614 INFO:tasks.workunit.client.0.smithi040.stderr: 97950K .......... .......... .......... .......... .......... 72% 31.4M 2s 2024-06-06T03:40:21.616 INFO:tasks.workunit.client.0.smithi040.stderr: 98000K .......... .......... .......... .......... .......... 72% 24.4M 2s 2024-06-06T03:40:21.618 INFO:tasks.workunit.client.0.smithi040.stderr: 98050K .......... .......... .......... .......... .......... 72% 16.6M 2s 2024-06-06T03:40:21.621 INFO:tasks.workunit.client.0.smithi040.stderr: 98100K .......... .......... .......... .......... .......... 72% 19.5M 2s 2024-06-06T03:40:21.624 INFO:tasks.workunit.client.0.smithi040.stderr: 98150K .......... .......... .......... .......... .......... 72% 17.6M 2s 2024-06-06T03:40:21.626 INFO:tasks.workunit.client.0.smithi040.stderr: 98200K .......... .......... .......... .......... .......... 72% 36.5M 2s 2024-06-06T03:40:21.627 INFO:tasks.workunit.client.0.smithi040.stderr: 98250K .......... .......... .......... .......... .......... 72% 21.1M 2s 2024-06-06T03:40:21.631 INFO:tasks.workunit.client.0.smithi040.stderr: 98300K .......... .......... .......... .......... .......... 72% 20.3M 2s 2024-06-06T03:40:21.631 INFO:tasks.workunit.client.0.smithi040.stderr: 98350K .......... .......... .......... .......... .......... 72% 34.3M 2s 2024-06-06T03:40:21.634 INFO:tasks.workunit.client.0.smithi040.stderr: 98400K .......... .......... .......... .......... .......... 72% 21.1M 2s 2024-06-06T03:40:21.636 INFO:tasks.workunit.client.0.smithi040.stderr: 98450K .......... .......... .......... .......... .......... 72% 39.9M 2s 2024-06-06T03:40:21.637 INFO:tasks.workunit.client.0.smithi040.stderr: 98500K .......... .......... .......... .......... .......... 72% 22.9M 2s 2024-06-06T03:40:21.639 INFO:tasks.workunit.client.0.smithi040.stderr: 98550K .......... .......... .......... .......... .......... 72% 23.4M 2s 2024-06-06T03:40:21.648 INFO:tasks.workunit.client.0.smithi040.stderr: 98600K .......... .......... .......... .......... .......... 72% 28.2M 2s 2024-06-06T03:40:21.649 INFO:tasks.workunit.client.0.smithi040.stderr: 98650K .......... .......... .......... .......... .......... 72% 25.9M 2s 2024-06-06T03:40:21.649 INFO:tasks.workunit.client.0.smithi040.stderr: 98700K .......... .......... .......... .......... .......... 72% 23.6M 2s 2024-06-06T03:40:21.649 INFO:tasks.workunit.client.0.smithi040.stderr: 98750K .......... .......... .......... .......... .......... 72% 42.5M 2s 2024-06-06T03:40:21.649 INFO:tasks.workunit.client.0.smithi040.stderr: 98800K .......... .......... .......... .......... .......... 72% 24.4M 2s 2024-06-06T03:40:21.652 INFO:tasks.workunit.client.0.smithi040.stderr: 98850K .......... .......... .......... .......... .......... 72% 28.6M 2s 2024-06-06T03:40:21.653 INFO:tasks.workunit.client.0.smithi040.stderr: 98900K .......... .......... .......... .......... .......... 72% 23.2M 2s 2024-06-06T03:40:21.656 INFO:tasks.workunit.client.0.smithi040.stderr: 98950K .......... .......... .......... .......... .......... 72% 17.1M 2s 2024-06-06T03:40:21.656 INFO:tasks.workunit.client.0.smithi040.stderr: 99000K .......... .......... .......... .......... .......... 72% 35.3M 2s 2024-06-06T03:40:21.658 INFO:tasks.workunit.client.0.smithi040.stderr: 99050K .......... .......... .......... .......... .......... 73% 24.6M 2s 2024-06-06T03:40:21.661 INFO:tasks.workunit.client.0.smithi040.stderr: 99100K .......... .......... .......... .......... .......... 73% 39.1M 2s 2024-06-06T03:40:21.663 INFO:tasks.workunit.client.0.smithi040.stderr: 99150K .......... .......... .......... .......... .......... 73% 20.5M 2s 2024-06-06T03:40:21.665 INFO:tasks.workunit.client.0.smithi040.stderr: 99200K .......... .......... .......... .......... .......... 73% 22.0M 2s 2024-06-06T03:40:21.669 INFO:tasks.workunit.client.0.smithi040.stderr: 99250K .......... .......... .......... .......... .......... 73% 27.6M 2s 2024-06-06T03:40:21.670 INFO:tasks.workunit.client.0.smithi040.stderr: 99300K .......... .......... .......... .......... .......... 73% 25.5M 2s 2024-06-06T03:40:21.670 INFO:tasks.workunit.client.0.smithi040.stderr: 99350K .......... .......... .......... .......... .......... 73% 40.5M 2s 2024-06-06T03:40:21.672 INFO:tasks.workunit.client.0.smithi040.stderr: 99400K .......... .......... .......... .......... .......... 73% 18.0M 2s 2024-06-06T03:40:21.677 INFO:tasks.workunit.client.0.smithi040.stderr: 99450K .......... .......... .......... .......... .......... 73% 18.3M 2s 2024-06-06T03:40:21.677 INFO:tasks.workunit.client.0.smithi040.stderr: 99500K .......... .......... .......... .......... .......... 73% 30.3M 2s 2024-06-06T03:40:21.679 INFO:tasks.workunit.client.0.smithi040.stderr: 99550K .......... .......... .......... .......... .......... 73% 22.2M 2s 2024-06-06T03:40:21.680 INFO:tasks.workunit.client.0.smithi040.stderr: 99600K .......... .......... .......... .......... .......... 73% 41.1M 2s 2024-06-06T03:40:21.682 INFO:tasks.workunit.client.0.smithi040.stderr: 99650K .......... .......... .......... .......... .......... 73% 18.9M 2s 2024-06-06T03:40:21.685 INFO:tasks.workunit.client.0.smithi040.stderr: 99700K .......... .......... .......... .......... .......... 73% 15.7M 2s 2024-06-06T03:40:21.686 INFO:tasks.workunit.client.0.smithi040.stderr: 99750K .......... .......... .......... .......... .......... 73% 38.5M 2s 2024-06-06T03:40:21.689 INFO:tasks.workunit.client.0.smithi040.stderr: 99800K .......... .......... .......... .......... .......... 73% 17.5M 2s 2024-06-06T03:40:21.690 INFO:tasks.workunit.client.0.smithi040.stderr: 99850K .......... .......... .......... .......... .......... 73% 48.6M 2s 2024-06-06T03:40:21.693 INFO:tasks.workunit.client.0.smithi040.stderr: 99900K .......... .......... .......... .......... .......... 73% 21.0M 1s 2024-06-06T03:40:21.697 INFO:tasks.workunit.client.0.smithi040.stderr: 99950K .......... .......... .......... .......... .......... 73% 17.3M 1s 2024-06-06T03:40:21.697 INFO:tasks.workunit.client.0.smithi040.stderr:100000K .......... .......... .......... .......... .......... 73% 30.0M 1s 2024-06-06T03:40:21.700 INFO:tasks.workunit.client.0.smithi040.stderr:100050K .......... .......... .......... .......... .......... 73% 18.4M 1s 2024-06-06T03:40:21.705 INFO:tasks.workunit.client.0.smithi040.stderr:100100K .......... .......... .......... .......... .......... 73% 22.6M 1s 2024-06-06T03:40:21.705 INFO:tasks.workunit.client.0.smithi040.stderr:100150K .......... .......... .......... .......... .......... 73% 26.0M 1s 2024-06-06T03:40:21.707 INFO:tasks.workunit.client.0.smithi040.stderr:100200K .......... .......... .......... .......... .......... 73% 15.7M 1s 2024-06-06T03:40:21.709 INFO:tasks.workunit.client.0.smithi040.stderr:100250K .......... .......... .......... .......... .......... 73% 45.6M 1s 2024-06-06T03:40:21.710 INFO:tasks.workunit.client.0.smithi040.stderr:100300K .......... .......... .......... .......... .......... 73% 21.6M 1s 2024-06-06T03:40:21.712 INFO:tasks.workunit.client.0.smithi040.stderr:100350K .......... .......... .......... .......... .......... 73% 28.5M 1s 2024-06-06T03:40:21.714 INFO:tasks.workunit.client.0.smithi040.stderr:100400K .......... .......... .......... .......... .......... 73% 26.4M 1s 2024-06-06T03:40:21.717 INFO:tasks.workunit.client.0.smithi040.stderr:100450K .......... .......... .......... .......... .......... 74% 22.4M 1s 2024-06-06T03:40:21.717 INFO:tasks.workunit.client.0.smithi040.stderr:100500K .......... .......... .......... .......... .......... 74% 31.2M 1s 2024-06-06T03:40:21.723 INFO:tasks.workunit.client.0.smithi040.stderr:100550K .......... .......... .......... .......... .......... 74% 23.3M 1s 2024-06-06T03:40:21.723 INFO:tasks.workunit.client.0.smithi040.stderr:100600K .......... .......... .......... .......... .......... 74% 39.0M 1s 2024-06-06T03:40:21.724 INFO:tasks.workunit.client.0.smithi040.stderr:100650K .......... .......... .......... .......... .......... 74% 22.9M 1s 2024-06-06T03:40:21.729 INFO:tasks.workunit.client.0.smithi040.stderr:100700K .......... .......... .......... .......... .......... 74% 25.2M 1s 2024-06-06T03:40:21.729 INFO:tasks.workunit.client.0.smithi040.stderr:100750K .......... .......... .......... .......... .......... 74% 23.4M 1s 2024-06-06T03:40:21.729 INFO:tasks.workunit.client.0.smithi040.stderr:100800K .......... .......... .......... .......... .......... 74% 30.7M 1s 2024-06-06T03:40:21.731 INFO:tasks.workunit.client.0.smithi040.stderr:100850K .......... .......... .......... .......... .......... 74% 22.3M 1s 2024-06-06T03:40:21.733 INFO:tasks.workunit.client.0.smithi040.stderr:100900K .......... .......... .......... .......... .......... 74% 25.0M 1s 2024-06-06T03:40:21.735 INFO:tasks.workunit.client.0.smithi040.stderr:100950K .......... .......... .......... .......... .......... 74% 28.3M 1s 2024-06-06T03:40:21.736 INFO:tasks.workunit.client.0.smithi040.stderr:101000K .......... .......... .......... .......... .......... 74% 22.5M 1s 2024-06-06T03:40:21.738 INFO:tasks.workunit.client.0.smithi040.stderr:101050K .......... .......... .......... .......... .......... 74% 23.9M 1s 2024-06-06T03:40:21.740 INFO:tasks.workunit.client.0.smithi040.stderr:101100K .......... .......... .......... .......... .......... 74% 25.8M 1s 2024-06-06T03:40:21.742 INFO:tasks.workunit.client.0.smithi040.stderr:101150K .......... .......... .......... .......... .......... 74% 24.0M 1s 2024-06-06T03:40:21.744 INFO:tasks.workunit.client.0.smithi040.stderr:101200K .......... .......... .......... .......... .......... 74% 22.5M 1s 2024-06-06T03:40:21.747 INFO:tasks.workunit.client.0.smithi040.stderr:101250K .......... .......... .......... .......... .......... 74% 24.9M 1s 2024-06-06T03:40:21.749 INFO:tasks.workunit.client.0.smithi040.stderr:101300K .......... .......... .......... .......... .......... 74% 18.9M 1s 2024-06-06T03:40:21.750 INFO:tasks.workunit.client.0.smithi040.stderr:101350K .......... .......... .......... .......... .......... 74% 43.0M 1s 2024-06-06T03:40:21.754 INFO:tasks.workunit.client.0.smithi040.stderr:101400K .......... .......... .......... .......... .......... 74% 22.7M 1s 2024-06-06T03:40:21.758 INFO:tasks.workunit.client.0.smithi040.stderr:101450K .......... .......... .......... .......... .......... 74% 21.0M 1s 2024-06-06T03:40:21.758 INFO:tasks.workunit.client.0.smithi040.stderr:101500K .......... .......... .......... .......... .......... 74% 19.9M 1s 2024-06-06T03:40:21.758 INFO:tasks.workunit.client.0.smithi040.stderr:101550K .......... .......... .......... .......... .......... 74% 33.7M 1s 2024-06-06T03:40:21.760 INFO:tasks.workunit.client.0.smithi040.stderr:101600K .......... .......... .......... .......... .......... 74% 21.8M 1s 2024-06-06T03:40:21.764 INFO:tasks.workunit.client.0.smithi040.stderr:101650K .......... .......... .......... .......... .......... 74% 25.1M 1s 2024-06-06T03:40:21.769 INFO:tasks.workunit.client.0.smithi040.stderr:101700K .......... .......... .......... .......... .......... 74% 18.7M 1s 2024-06-06T03:40:21.769 INFO:tasks.workunit.client.0.smithi040.stderr:101750K .......... .......... .......... .......... .......... 74% 32.9M 1s 2024-06-06T03:40:21.769 INFO:tasks.workunit.client.0.smithi040.stderr:101800K .......... .......... .......... .......... .......... 75% 23.8M 1s 2024-06-06T03:40:21.774 INFO:tasks.workunit.client.0.smithi040.stderr:101850K .......... .......... .......... .......... .......... 75% 20.5M 1s 2024-06-06T03:40:21.774 INFO:tasks.workunit.client.0.smithi040.stderr:101900K .......... .......... .......... .......... .......... 75% 23.0M 1s 2024-06-06T03:40:21.777 INFO:tasks.workunit.client.0.smithi040.stderr:101950K .......... .......... .......... .......... .......... 75% 15.5M 1s 2024-06-06T03:40:21.779 INFO:tasks.workunit.client.0.smithi040.stderr:102000K .......... .......... .......... .......... .......... 75% 21.1M 1s 2024-06-06T03:40:21.780 INFO:tasks.workunit.client.0.smithi040.stderr:102050K .......... .......... .......... .......... .......... 75% 37.2M 1s 2024-06-06T03:40:21.783 INFO:tasks.workunit.client.0.smithi040.stderr:102100K .......... .......... .......... .......... .......... 75% 26.5M 1s 2024-06-06T03:40:21.785 INFO:tasks.workunit.client.0.smithi040.stderr:102150K .......... .......... .......... .......... .......... 75% 18.9M 1s 2024-06-06T03:40:21.787 INFO:tasks.workunit.client.0.smithi040.stderr:102200K .......... .......... .......... .......... .......... 75% 20.3M 1s 2024-06-06T03:40:21.791 INFO:tasks.workunit.client.0.smithi040.stderr:102250K .......... .......... .......... .......... .......... 75% 45.9M 1s 2024-06-06T03:40:21.791 INFO:tasks.workunit.client.0.smithi040.stderr:102300K .......... .......... .......... .......... .......... 75% 25.2M 1s 2024-06-06T03:40:21.793 INFO:tasks.workunit.client.0.smithi040.stderr:102350K .......... .......... .......... .......... .......... 75% 17.4M 1s 2024-06-06T03:40:21.801 INFO:tasks.workunit.client.0.smithi040.stderr:102400K .......... .......... .......... .......... .......... 75% 20.9M 1s 2024-06-06T03:40:21.801 INFO:tasks.workunit.client.0.smithi040.stderr:102450K .......... .......... .......... .......... .......... 75% 82.4M 1s 2024-06-06T03:40:21.801 INFO:tasks.workunit.client.0.smithi040.stderr:102500K .......... .......... .......... .......... .......... 75% 23.8M 1s 2024-06-06T03:40:21.801 INFO:tasks.workunit.client.0.smithi040.stderr:102550K .......... .......... .......... .......... .......... 75% 21.5M 1s 2024-06-06T03:40:21.805 INFO:tasks.workunit.client.0.smithi040.stderr:102600K .......... .......... .......... .......... .......... 75% 23.5M 1s 2024-06-06T03:40:21.805 INFO:tasks.workunit.client.0.smithi040.stderr:102650K .......... .......... .......... .......... .......... 75% 45.6M 1s 2024-06-06T03:40:21.810 INFO:tasks.workunit.client.0.smithi040.stderr:102700K .......... .......... .......... .......... .......... 75% 17.1M 1s 2024-06-06T03:40:21.810 INFO:tasks.workunit.client.0.smithi040.stderr:102750K .......... .......... .......... .......... .......... 75% 39.8M 1s 2024-06-06T03:40:21.810 INFO:tasks.workunit.client.0.smithi040.stderr:102800K .......... .......... .......... .......... .......... 75% 23.3M 1s 2024-06-06T03:40:21.811 INFO:tasks.workunit.client.0.smithi040.stderr:102850K .......... .......... .......... .......... .......... 75% 24.8M 1s 2024-06-06T03:40:21.815 INFO:tasks.workunit.client.0.smithi040.stderr:102900K .......... .......... .......... .......... .......... 75% 27.7M 1s 2024-06-06T03:40:21.816 INFO:tasks.workunit.client.0.smithi040.stderr:102950K .......... .......... .......... .......... .......... 75% 17.6M 1s 2024-06-06T03:40:21.817 INFO:tasks.workunit.client.0.smithi040.stderr:103000K .......... .......... .......... .......... .......... 75% 37.9M 1s 2024-06-06T03:40:21.819 INFO:tasks.workunit.client.0.smithi040.stderr:103050K .......... .......... .......... .......... .......... 75% 29.9M 1s 2024-06-06T03:40:21.822 INFO:tasks.workunit.client.0.smithi040.stderr:103100K .......... .......... .......... .......... .......... 75% 15.1M 1s 2024-06-06T03:40:21.824 INFO:tasks.workunit.client.0.smithi040.stderr:103150K .......... .......... .......... .......... .......... 76% 52.1M 1s 2024-06-06T03:40:21.826 INFO:tasks.workunit.client.0.smithi040.stderr:103200K .......... .......... .......... .......... .......... 76% 18.2M 1s 2024-06-06T03:40:21.827 INFO:tasks.workunit.client.0.smithi040.stderr:103250K .......... .......... .......... .......... .......... 76% 32.7M 1s 2024-06-06T03:40:21.829 INFO:tasks.workunit.client.0.smithi040.stderr:103300K .......... .......... .......... .......... .......... 76% 22.8M 1s 2024-06-06T03:40:21.832 INFO:tasks.workunit.client.0.smithi040.stderr:103350K .......... .......... .......... .......... .......... 76% 21.1M 1s 2024-06-06T03:40:21.833 INFO:tasks.workunit.client.0.smithi040.stderr:103400K .......... .......... .......... .......... .......... 76% 26.5M 1s 2024-06-06T03:40:21.840 INFO:tasks.workunit.client.0.smithi040.stderr:103450K .......... .......... .......... .......... .......... 76% 19.3M 1s 2024-06-06T03:40:21.840 INFO:tasks.workunit.client.0.smithi040.stderr:103500K .......... .......... .......... .......... .......... 76% 31.1M 1s 2024-06-06T03:40:21.840 INFO:tasks.workunit.client.0.smithi040.stderr:103550K .......... .......... .......... .......... .......... 76% 24.7M 1s 2024-06-06T03:40:21.843 INFO:tasks.workunit.client.0.smithi040.stderr:103600K .......... .......... .......... .......... .......... 76% 16.5M 1s 2024-06-06T03:40:21.844 INFO:tasks.workunit.client.0.smithi040.stderr:103650K .......... .......... .......... .......... .......... 76% 44.0M 1s 2024-06-06T03:40:21.847 INFO:tasks.workunit.client.0.smithi040.stderr:103700K .......... .......... .......... .......... .......... 76% 23.0M 1s 2024-06-06T03:40:21.849 INFO:tasks.workunit.client.0.smithi040.stderr:103750K .......... .......... .......... .......... .......... 76% 22.6M 1s 2024-06-06T03:40:21.850 INFO:tasks.workunit.client.0.smithi040.stderr:103800K .......... .......... .......... .......... .......... 76% 21.4M 1s 2024-06-06T03:40:21.857 INFO:tasks.workunit.client.0.smithi040.stderr:103850K .......... .......... .......... .......... .......... 76% 15.1M 1s 2024-06-06T03:40:21.857 INFO:tasks.workunit.client.0.smithi040.stderr:103900K .......... .......... .......... .......... .......... 76% 45.6M 1s 2024-06-06T03:40:21.857 INFO:tasks.workunit.client.0.smithi040.stderr:103950K .......... .......... .......... .......... .......... 76% 25.6M 1s 2024-06-06T03:40:21.860 INFO:tasks.workunit.client.0.smithi040.stderr:104000K .......... .......... .......... .......... .......... 76% 16.4M 1s 2024-06-06T03:40:21.887 INFO:tasks.workunit.client.0.smithi040.stderr:104050K .......... .......... .......... .......... .......... 76% 25.6M 1s 2024-06-06T03:40:21.887 INFO:tasks.workunit.client.0.smithi040.stderr:104100K .......... .......... .......... .......... .......... 76% 21.5M 1s 2024-06-06T03:40:21.887 INFO:tasks.workunit.client.0.smithi040.stderr:104150K .......... .......... .......... .......... .......... 76% 43.1M 1s 2024-06-06T03:40:21.887 INFO:tasks.workunit.client.0.smithi040.stderr:104200K .......... .......... .......... .......... .......... 76% 22.4M 1s 2024-06-06T03:40:21.887 INFO:tasks.workunit.client.0.smithi040.stderr:104250K .......... .......... .......... .......... .......... 76% 19.4M 1s 2024-06-06T03:40:21.887 INFO:tasks.workunit.client.0.smithi040.stderr:104300K .......... .......... .......... .......... .......... 76% 36.8M 1s 2024-06-06T03:40:21.887 INFO:tasks.workunit.client.0.smithi040.stderr:104350K .......... .......... .......... .......... .......... 76% 24.9M 1s 2024-06-06T03:40:21.887 INFO:tasks.workunit.client.0.smithi040.stderr:104400K .......... .......... .......... .......... .......... 76% 41.9M 1s 2024-06-06T03:40:21.888 INFO:tasks.workunit.client.0.smithi040.stderr:104450K .......... .......... .......... .......... .......... 76% 19.6M 1s 2024-06-06T03:40:21.888 INFO:tasks.workunit.client.0.smithi040.stderr:104500K .......... .......... .......... .......... .......... 77% 25.2M 1s 2024-06-06T03:40:21.888 INFO:tasks.workunit.client.0.smithi040.stderr:104550K .......... .......... .......... .......... .......... 77% 27.0M 1s 2024-06-06T03:40:21.888 INFO:tasks.workunit.client.0.smithi040.stderr:104600K .......... .......... .......... .......... .......... 77% 19.1M 1s 2024-06-06T03:40:21.888 INFO:tasks.workunit.client.0.smithi040.stderr:104650K .......... .......... .......... .......... .......... 77% 41.8M 1s 2024-06-06T03:40:21.888 INFO:tasks.workunit.client.0.smithi040.stderr:104700K .......... .......... .......... .......... .......... 77% 21.7M 1s 2024-06-06T03:40:21.888 INFO:tasks.workunit.client.0.smithi040.stderr:104750K .......... .......... .......... .......... .......... 77% 45.0M 1s 2024-06-06T03:40:21.890 INFO:tasks.workunit.client.0.smithi040.stderr:104800K .......... .......... .......... .......... .......... 77% 20.2M 1s 2024-06-06T03:40:21.892 INFO:tasks.workunit.client.0.smithi040.stderr:104850K .......... .......... .......... .......... .......... 77% 19.9M 1s 2024-06-06T03:40:21.893 INFO:tasks.workunit.client.0.smithi040.stderr:104900K .......... .......... .......... .......... .......... 77% 34.0M 1s 2024-06-06T03:40:21.896 INFO:tasks.workunit.client.0.smithi040.stderr:104950K .......... .......... .......... .......... .......... 77% 25.8M 1s 2024-06-06T03:40:21.897 INFO:tasks.workunit.client.0.smithi040.stderr:105000K .......... .......... .......... .......... .......... 77% 33.3M 1s 2024-06-06T03:40:21.901 INFO:tasks.workunit.client.0.smithi040.stderr:105050K .......... .......... .......... .......... .......... 77% 18.8M 1s 2024-06-06T03:40:21.901 INFO:tasks.workunit.client.0.smithi040.stderr:105100K .......... .......... .......... .......... .......... 77% 21.9M 1s 2024-06-06T03:40:21.912 INFO:tasks.workunit.client.0.smithi040.stderr:105150K .......... .......... .......... .......... .......... 77% 25.6M 1s 2024-06-06T03:40:21.912 INFO:tasks.workunit.client.0.smithi040.stderr:105200K .......... .......... .......... .......... .......... 77% 27.6M 1s 2024-06-06T03:40:21.912 INFO:tasks.workunit.client.0.smithi040.stderr:105250K .......... .......... .......... .......... .......... 77% 21.7M 1s 2024-06-06T03:40:21.912 INFO:tasks.workunit.client.0.smithi040.stderr:105300K .......... .......... .......... .......... .......... 77% 32.8M 1s 2024-06-06T03:40:21.913 INFO:tasks.workunit.client.0.smithi040.stderr:105350K .......... .......... .......... .......... .......... 77% 15.4M 1s 2024-06-06T03:40:21.928 INFO:tasks.workunit.client.0.smithi040.stderr:105400K .......... .......... .......... .......... .......... 77% 35.5M 1s 2024-06-06T03:40:21.928 INFO:tasks.workunit.client.0.smithi040.stderr:105450K .......... .......... .......... .......... .......... 77% 27.9M 1s 2024-06-06T03:40:21.928 INFO:tasks.workunit.client.0.smithi040.stderr:105500K .......... .......... .......... .......... .......... 77% 25.9M 1s 2024-06-06T03:40:21.928 INFO:tasks.workunit.client.0.smithi040.stderr:105550K .......... .......... .......... .......... .......... 77% 18.8M 1s 2024-06-06T03:40:21.929 INFO:tasks.workunit.client.0.smithi040.stderr:105600K .......... .......... .......... .......... .......... 77% 19.7M 1s 2024-06-06T03:40:21.929 INFO:tasks.workunit.client.0.smithi040.stderr:105650K .......... .......... .......... .......... .......... 77% 32.2M 1s 2024-06-06T03:40:21.929 INFO:tasks.workunit.client.0.smithi040.stderr:105700K .......... .......... .......... .......... .......... 77% 21.7M 1s 2024-06-06T03:40:21.929 INFO:tasks.workunit.client.0.smithi040.stderr:105750K .......... .......... .......... .......... .......... 77% 22.8M 1s 2024-06-06T03:40:21.932 INFO:tasks.workunit.client.0.smithi040.stderr:105800K .......... .......... .......... .......... .......... 77% 17.9M 1s 2024-06-06T03:40:21.935 INFO:tasks.workunit.client.0.smithi040.stderr:105850K .......... .......... .......... .......... .......... 78% 21.6M 1s 2024-06-06T03:40:21.935 INFO:tasks.workunit.client.0.smithi040.stderr:105900K .......... .......... .......... .......... .......... 78% 38.4M 1s 2024-06-06T03:40:21.937 INFO:tasks.workunit.client.0.smithi040.stderr:105950K .......... .......... .......... .......... .......... 78% 19.1M 1s 2024-06-06T03:40:21.940 INFO:tasks.workunit.client.0.smithi040.stderr:106000K .......... .......... .......... .......... .......... 78% 27.3M 1s 2024-06-06T03:40:21.943 INFO:tasks.workunit.client.0.smithi040.stderr:106050K .......... .......... .......... .......... .......... 78% 23.4M 1s 2024-06-06T03:40:21.943 INFO:tasks.workunit.client.0.smithi040.stderr:106100K .......... .......... .......... .......... .......... 78% 24.2M 1s 2024-06-06T03:40:21.945 INFO:tasks.workunit.client.0.smithi040.stderr:106150K .......... .......... .......... .......... .......... 78% 26.7M 1s 2024-06-06T03:40:21.952 INFO:tasks.workunit.client.0.smithi040.stderr:106200K .......... .......... .......... .......... .......... 78% 24.1M 1s 2024-06-06T03:40:21.952 INFO:tasks.workunit.client.0.smithi040.stderr:106250K .......... .......... .......... .......... .......... 78% 32.9M 1s 2024-06-06T03:40:21.952 INFO:tasks.workunit.client.0.smithi040.stderr:106300K .......... .......... .......... .......... .......... 78% 27.6M 1s 2024-06-06T03:40:21.952 INFO:tasks.workunit.client.0.smithi040.stderr:106350K .......... .......... .......... .......... .......... 78% 19.4M 1s 2024-06-06T03:40:21.991 INFO:tasks.workunit.client.0.smithi040.stderr:106400K .......... .......... .......... .......... .......... 78% 31.1M 1s 2024-06-06T03:40:21.991 INFO:tasks.workunit.client.0.smithi040.stderr:106450K .......... .......... .......... .......... .......... 78% 23.0M 1s 2024-06-06T03:40:21.992 INFO:tasks.workunit.client.0.smithi040.stderr:106500K .......... .......... .......... .......... .......... 78% 25.3M 1s 2024-06-06T03:40:21.992 INFO:tasks.workunit.client.0.smithi040.stderr:106550K .......... .......... .......... .......... .......... 78% 25.2M 1s 2024-06-06T03:40:21.992 INFO:tasks.workunit.client.0.smithi040.stderr:106600K .......... .......... .......... .......... .......... 78% 23.8M 1s 2024-06-06T03:40:21.992 INFO:tasks.workunit.client.0.smithi040.stderr:106650K .......... .......... .......... .......... .......... 78% 35.6M 1s 2024-06-06T03:40:21.992 INFO:tasks.workunit.client.0.smithi040.stderr:106700K .......... .......... .......... .......... .......... 78% 22.3M 1s 2024-06-06T03:40:21.992 INFO:tasks.workunit.client.0.smithi040.stderr:106750K .......... .......... .......... .......... .......... 78% 22.4M 1s 2024-06-06T03:40:21.992 INFO:tasks.workunit.client.0.smithi040.stderr:106800K .......... .......... .......... .......... .......... 78% 26.2M 1s 2024-06-06T03:40:21.992 INFO:tasks.workunit.client.0.smithi040.stderr:106850K .......... .......... .......... .......... .......... 78% 32.8M 1s 2024-06-06T03:40:21.992 INFO:tasks.workunit.client.0.smithi040.stderr:106900K .......... .......... .......... .......... .......... 78% 25.9M 1s 2024-06-06T03:40:21.992 INFO:tasks.workunit.client.0.smithi040.stderr:106950K .......... .......... .......... .......... .......... 78% 18.0M 1s 2024-06-06T03:40:21.992 INFO:tasks.workunit.client.0.smithi040.stderr:107000K .......... .......... .......... .......... .......... 78% 19.8M 1s 2024-06-06T03:40:21.992 INFO:tasks.workunit.client.0.smithi040.stderr:107050K .......... .......... .......... .......... .......... 78% 23.0M 1s 2024-06-06T03:40:21.993 INFO:tasks.workunit.client.0.smithi040.stderr:107100K .......... .......... .......... .......... .......... 78% 35.2M 1s 2024-06-06T03:40:21.993 INFO:tasks.workunit.client.0.smithi040.stderr:107150K .......... .......... .......... .......... .......... 78% 36.1M 1s 2024-06-06T03:40:21.993 INFO:tasks.workunit.client.0.smithi040.stderr:107200K .......... .......... .......... .......... .......... 79% 20.6M 1s 2024-06-06T03:40:21.993 INFO:tasks.workunit.client.0.smithi040.stderr:107250K .......... .......... .......... .......... .......... 79% 19.5M 1s 2024-06-06T03:40:21.993 INFO:tasks.workunit.client.0.smithi040.stderr:107300K .......... .......... .......... .......... .......... 79% 25.0M 1s 2024-06-06T03:40:22.001 INFO:tasks.workunit.client.0.smithi040.stderr:107350K .......... .......... .......... .......... .......... 79% 21.1M 1s 2024-06-06T03:40:22.002 INFO:tasks.workunit.client.0.smithi040.stderr:107400K .......... .......... .......... .......... .......... 79% 42.4M 1s 2024-06-06T03:40:22.002 INFO:tasks.workunit.client.0.smithi040.stderr:107450K .......... .......... .......... .......... .......... 79% 21.7M 1s 2024-06-06T03:40:22.002 INFO:tasks.workunit.client.0.smithi040.stderr:107500K .......... .......... .......... .......... .......... 79% 21.1M 1s 2024-06-06T03:40:22.002 INFO:tasks.workunit.client.0.smithi040.stderr:107550K .......... .......... .......... .......... .......... 79% 19.1M 1s 2024-06-06T03:40:22.003 INFO:tasks.workunit.client.0.smithi040.stderr:107600K .......... .......... .......... .......... .......... 79% 23.7M 1s 2024-06-06T03:40:22.006 INFO:tasks.workunit.client.0.smithi040.stderr:107650K .......... .......... .......... .......... .......... 79% 21.6M 1s 2024-06-06T03:40:22.007 INFO:tasks.workunit.client.0.smithi040.stderr:107700K .......... .......... .......... .......... .......... 79% 32.9M 1s 2024-06-06T03:40:22.010 INFO:tasks.workunit.client.0.smithi040.stderr:107750K .......... .......... .......... .......... .......... 79% 18.0M 1s 2024-06-06T03:40:22.012 INFO:tasks.workunit.client.0.smithi040.stderr:107800K .......... .......... .......... .......... .......... 79% 24.7M 1s 2024-06-06T03:40:22.013 INFO:tasks.workunit.client.0.smithi040.stderr:107850K .......... .......... .......... .......... .......... 79% 20.0M 1s 2024-06-06T03:40:22.034 INFO:tasks.workunit.client.0.smithi040.stderr:107900K .......... .......... .......... .......... .......... 79% 20.4M 1s 2024-06-06T03:40:22.035 INFO:tasks.workunit.client.0.smithi040.stderr:107950K .......... .......... .......... .......... .......... 79% 38.8M 1s 2024-06-06T03:40:22.035 INFO:tasks.workunit.client.0.smithi040.stderr:108000K .......... .......... .......... .......... .......... 79% 24.3M 1s 2024-06-06T03:40:22.035 INFO:tasks.workunit.client.0.smithi040.stderr:108050K .......... .......... .......... .......... .......... 79% 27.5M 1s 2024-06-06T03:40:22.035 INFO:tasks.workunit.client.0.smithi040.stderr:108100K .......... .......... .......... .......... .......... 79% 23.8M 1s 2024-06-06T03:40:22.035 INFO:tasks.workunit.client.0.smithi040.stderr:108150K .......... .......... .......... .......... .......... 79% 20.4M 1s 2024-06-06T03:40:22.035 INFO:tasks.workunit.client.0.smithi040.stderr:108200K .......... .......... .......... .......... .......... 79% 45.1M 1s 2024-06-06T03:40:22.035 INFO:tasks.workunit.client.0.smithi040.stderr:108250K .......... .......... .......... .......... .......... 79% 19.3M 1s 2024-06-06T03:40:22.035 INFO:tasks.workunit.client.0.smithi040.stderr:108300K .......... .......... .......... .......... .......... 79% 23.4M 1s 2024-06-06T03:40:22.035 INFO:tasks.workunit.client.0.smithi040.stderr:108350K .......... .......... .......... .......... .......... 79% 41.9M 1s 2024-06-06T03:40:22.074 INFO:tasks.workunit.client.0.smithi040.stderr:108400K .......... .......... .......... .......... .......... 79% 17.3M 1s 2024-06-06T03:40:22.074 INFO:tasks.workunit.client.0.smithi040.stderr:108450K .......... .......... .......... .......... .......... 79% 42.8M 1s 2024-06-06T03:40:22.074 INFO:tasks.workunit.client.0.smithi040.stderr:108500K .......... .......... .......... .......... .......... 79% 24.4M 1s 2024-06-06T03:40:22.074 INFO:tasks.workunit.client.0.smithi040.stderr:108550K .......... .......... .......... .......... .......... 79% 29.3M 1s 2024-06-06T03:40:22.074 INFO:tasks.workunit.client.0.smithi040.stderr:108600K .......... .......... .......... .......... .......... 80% 22.0M 1s 2024-06-06T03:40:22.074 INFO:tasks.workunit.client.0.smithi040.stderr:108650K .......... .......... .......... .......... .......... 80% 17.5M 1s 2024-06-06T03:40:22.074 INFO:tasks.workunit.client.0.smithi040.stderr:108700K .......... .......... .......... .......... .......... 80% 40.4M 1s 2024-06-06T03:40:22.075 INFO:tasks.workunit.client.0.smithi040.stderr:108750K .......... .......... .......... .......... .......... 80% 20.1M 1s 2024-06-06T03:40:22.075 INFO:tasks.workunit.client.0.smithi040.stderr:108800K .......... .......... .......... .......... .......... 80% 41.8M 1s 2024-06-06T03:40:22.075 INFO:tasks.workunit.client.0.smithi040.stderr:108850K .......... .......... .......... .......... .......... 80% 21.9M 1s 2024-06-06T03:40:22.075 INFO:tasks.workunit.client.0.smithi040.stderr:108900K .......... .......... .......... .......... .......... 80% 26.7M 1s 2024-06-06T03:40:22.075 INFO:tasks.workunit.client.0.smithi040.stderr:108950K .......... .......... .......... .......... .......... 80% 23.6M 1s 2024-06-06T03:40:22.075 INFO:tasks.workunit.client.0.smithi040.stderr:109000K .......... .......... .......... .......... .......... 80% 19.2M 1s 2024-06-06T03:40:22.075 INFO:tasks.workunit.client.0.smithi040.stderr:109050K .......... .......... .......... .......... .......... 80% 26.3M 1s 2024-06-06T03:40:22.075 INFO:tasks.workunit.client.0.smithi040.stderr:109100K .......... .......... .......... .......... .......... 80% 31.0M 1s 2024-06-06T03:40:22.075 INFO:tasks.workunit.client.0.smithi040.stderr:109150K .......... .......... .......... .......... .......... 80% 18.5M 1s 2024-06-06T03:40:22.075 INFO:tasks.workunit.client.0.smithi040.stderr:109200K .......... .......... .......... .......... .......... 80% 26.0M 1s 2024-06-06T03:40:22.075 INFO:tasks.workunit.client.0.smithi040.stderr:109250K .......... .......... .......... .......... .......... 80% 21.3M 1s 2024-06-06T03:40:22.075 INFO:tasks.workunit.client.0.smithi040.stderr:109300K .......... .......... .......... .......... .......... 80% 20.2M 1s 2024-06-06T03:40:22.075 INFO:tasks.workunit.client.0.smithi040.stderr:109350K .......... .......... .......... .......... .......... 80% 47.5M 1s 2024-06-06T03:40:22.082 INFO:tasks.workunit.client.0.smithi040.stderr:109400K .......... .......... .......... .......... .......... 80% 15.9M 1s 2024-06-06T03:40:22.082 INFO:tasks.workunit.client.0.smithi040.stderr:109450K .......... .......... .......... .......... .......... 80% 41.3M 1s 2024-06-06T03:40:22.082 INFO:tasks.workunit.client.0.smithi040.stderr:109500K .......... .......... .......... .......... .......... 80% 20.0M 1s 2024-06-06T03:40:22.082 INFO:tasks.workunit.client.0.smithi040.stderr:109550K .......... .......... .......... .......... .......... 80% 20.2M 1s 2024-06-06T03:40:22.084 INFO:tasks.workunit.client.0.smithi040.stderr:109600K .......... .......... .......... .......... .......... 80% 23.2M 1s 2024-06-06T03:40:22.086 INFO:tasks.workunit.client.0.smithi040.stderr:109650K .......... .......... .......... .......... .......... 80% 21.3M 1s 2024-06-06T03:40:22.088 INFO:tasks.workunit.client.0.smithi040.stderr:109700K .......... .......... .......... .......... .......... 80% 24.3M 1s 2024-06-06T03:40:22.117 INFO:tasks.workunit.client.0.smithi040.stderr:109750K .......... .......... .......... .......... .......... 80% 22.9M 1s 2024-06-06T03:40:22.118 INFO:tasks.workunit.client.0.smithi040.stderr:109800K .......... .......... .......... .......... .......... 80% 19.0M 1s 2024-06-06T03:40:22.118 INFO:tasks.workunit.client.0.smithi040.stderr:109850K .......... .......... .......... .......... .......... 80% 26.4M 1s 2024-06-06T03:40:22.118 INFO:tasks.workunit.client.0.smithi040.stderr:109900K .......... .......... .......... .......... .......... 80% 20.6M 1s 2024-06-06T03:40:22.118 INFO:tasks.workunit.client.0.smithi040.stderr:109950K .......... .......... .......... .......... .......... 81% 29.3M 1s 2024-06-06T03:40:22.118 INFO:tasks.workunit.client.0.smithi040.stderr:110000K .......... .......... .......... .......... .......... 81% 18.1M 1s 2024-06-06T03:40:22.118 INFO:tasks.workunit.client.0.smithi040.stderr:110050K .......... .......... .......... .......... .......... 81% 24.5M 1s 2024-06-06T03:40:22.118 INFO:tasks.workunit.client.0.smithi040.stderr:110100K .......... .......... .......... .......... .......... 81% 40.9M 1s 2024-06-06T03:40:22.118 INFO:tasks.workunit.client.0.smithi040.stderr:110150K .......... .......... .......... .......... .......... 81% 21.6M 1s 2024-06-06T03:40:22.118 INFO:tasks.workunit.client.0.smithi040.stderr:110200K .......... .......... .......... .......... .......... 81% 29.7M 1s 2024-06-06T03:40:22.119 INFO:tasks.workunit.client.0.smithi040.stderr:110250K .......... .......... .......... .......... .......... 81% 22.0M 1s 2024-06-06T03:40:22.119 INFO:tasks.workunit.client.0.smithi040.stderr:110300K .......... .......... .......... .......... .......... 81% 25.1M 1s 2024-06-06T03:40:22.119 INFO:tasks.workunit.client.0.smithi040.stderr:110350K .......... .......... .......... .......... .......... 81% 22.4M 1s 2024-06-06T03:40:22.119 INFO:tasks.workunit.client.0.smithi040.stderr:110400K .......... .......... .......... .......... .......... 81% 67.5M 1s 2024-06-06T03:40:22.119 INFO:tasks.workunit.client.0.smithi040.stderr:110450K .......... .......... .......... .......... .......... 81% 21.1M 1s 2024-06-06T03:40:22.135 INFO:tasks.workunit.client.0.smithi040.stderr:110500K .......... .......... .......... .......... .......... 81% 26.8M 1s 2024-06-06T03:40:22.135 INFO:tasks.workunit.client.0.smithi040.stderr:110550K .......... .......... .......... .......... .......... 81% 16.4M 1s 2024-06-06T03:40:22.135 INFO:tasks.workunit.client.0.smithi040.stderr:110600K .......... .......... .......... .......... .......... 81% 34.8M 1s 2024-06-06T03:40:22.135 INFO:tasks.workunit.client.0.smithi040.stderr:110650K .......... .......... .......... .......... .......... 81% 29.3M 1s 2024-06-06T03:40:22.135 INFO:tasks.workunit.client.0.smithi040.stderr:110700K .......... .......... .......... .......... .......... 81% 26.6M 1s 2024-06-06T03:40:22.135 INFO:tasks.workunit.client.0.smithi040.stderr:110750K .......... .......... .......... .......... .......... 81% 22.8M 1s 2024-06-06T03:40:22.135 INFO:tasks.workunit.client.0.smithi040.stderr:110800K .......... .......... .......... .......... .......... 81% 22.7M 1s 2024-06-06T03:40:22.145 INFO:tasks.workunit.client.0.smithi040.stderr:110850K .......... .......... .......... .......... .......... 81% 8.00M 1s 2024-06-06T03:40:22.145 INFO:tasks.workunit.client.0.smithi040.stderr:110900K .......... .......... .......... .......... .......... 81% 76.6M 1s 2024-06-06T03:40:22.146 INFO:tasks.workunit.client.0.smithi040.stderr:110950K .......... .......... .......... .......... .......... 81% 115M 1s 2024-06-06T03:40:22.146 INFO:tasks.workunit.client.0.smithi040.stderr:111000K .......... .......... .......... .......... .......... 81% 41.1M 1s 2024-06-06T03:40:22.146 INFO:tasks.workunit.client.0.smithi040.stderr:111050K .......... .......... .......... .......... .......... 81% 38.3M 1s 2024-06-06T03:40:22.146 INFO:tasks.workunit.client.0.smithi040.stderr:111100K .......... .......... .......... .......... .......... 81% 19.9M 1s 2024-06-06T03:40:22.146 INFO:tasks.workunit.client.0.smithi040.stderr:111150K .......... .......... .......... .......... .......... 81% 23.9M 1s 2024-06-06T03:40:22.147 INFO:tasks.workunit.client.0.smithi040.stderr:111200K .......... .......... .......... .......... .......... 81% 24.9M 1s 2024-06-06T03:40:22.154 INFO:tasks.workunit.client.0.smithi040.stderr:111250K .......... .......... .......... .......... .......... 81% 23.1M 1s 2024-06-06T03:40:22.154 INFO:tasks.workunit.client.0.smithi040.stderr:111300K .......... .......... .......... .......... .......... 82% 23.7M 1s 2024-06-06T03:40:22.154 INFO:tasks.workunit.client.0.smithi040.stderr:111350K .......... .......... .......... .......... .......... 82% 19.1M 1s 2024-06-06T03:40:22.157 INFO:tasks.workunit.client.0.smithi040.stderr:111400K .......... .......... .......... .......... .......... 82% 20.8M 1s 2024-06-06T03:40:22.158 INFO:tasks.workunit.client.0.smithi040.stderr:111450K .......... .......... .......... .......... .......... 82% 55.0M 1s 2024-06-06T03:40:22.160 INFO:tasks.workunit.client.0.smithi040.stderr:111500K .......... .......... .......... .......... .......... 82% 19.2M 1s 2024-06-06T03:40:22.165 INFO:tasks.workunit.client.0.smithi040.stderr:111550K .......... .......... .......... .......... .......... 82% 23.0M 1s 2024-06-06T03:40:22.165 INFO:tasks.workunit.client.0.smithi040.stderr:111600K .......... .......... .......... .......... .......... 82% 20.5M 1s 2024-06-06T03:40:22.166 INFO:tasks.workunit.client.0.smithi040.stderr:111650K .......... .......... .......... .......... .......... 82% 30.0M 1s 2024-06-06T03:40:22.171 INFO:tasks.workunit.client.0.smithi040.stderr:111700K .......... .......... .......... .......... .......... 82% 28.6M 1s 2024-06-06T03:40:22.171 INFO:tasks.workunit.client.0.smithi040.stderr:111750K .......... .......... .......... .......... .......... 82% 19.8M 1s 2024-06-06T03:40:22.173 INFO:tasks.workunit.client.0.smithi040.stderr:111800K .......... .......... .......... .......... .......... 82% 17.1M 1s 2024-06-06T03:40:22.188 INFO:tasks.workunit.client.0.smithi040.stderr:111850K .......... .......... .......... .......... .......... 82% 37.4M 1s 2024-06-06T03:40:22.188 INFO:tasks.workunit.client.0.smithi040.stderr:111900K .......... .......... .......... .......... .......... 82% 18.7M 1s 2024-06-06T03:40:22.188 INFO:tasks.workunit.client.0.smithi040.stderr:111950K .......... .......... .......... .......... .......... 82% 41.9M 1s 2024-06-06T03:40:22.188 INFO:tasks.workunit.client.0.smithi040.stderr:112000K .......... .......... .......... .......... .......... 82% 21.1M 1s 2024-06-06T03:40:22.188 INFO:tasks.workunit.client.0.smithi040.stderr:112050K .......... .......... .......... .......... .......... 82% 31.3M 1s 2024-06-06T03:40:22.189 INFO:tasks.workunit.client.0.smithi040.stderr:112100K .......... .......... .......... .......... .......... 82% 18.4M 1s 2024-06-06T03:40:22.189 INFO:tasks.workunit.client.0.smithi040.stderr:112150K .......... .......... .......... .......... .......... 82% 26.4M 1s 2024-06-06T03:40:22.189 INFO:tasks.workunit.client.0.smithi040.stderr:112200K .......... .......... .......... .......... .......... 82% 41.3M 1s 2024-06-06T03:40:22.193 INFO:tasks.workunit.client.0.smithi040.stderr:112250K .......... .......... .......... .......... .......... 82% 22.0M 1s 2024-06-06T03:40:22.193 INFO:tasks.workunit.client.0.smithi040.stderr:112300K .......... .......... .......... .......... .......... 82% 26.3M 1s 2024-06-06T03:40:22.194 INFO:tasks.workunit.client.0.smithi040.stderr:112350K .......... .......... .......... .......... .......... 82% 19.9M 1s 2024-06-06T03:40:22.196 INFO:tasks.workunit.client.0.smithi040.stderr:112400K .......... .......... .......... .......... .......... 82% 32.0M 1s 2024-06-06T03:40:22.199 INFO:tasks.workunit.client.0.smithi040.stderr:112450K .......... .......... .......... .......... .......... 82% 36.7M 1s 2024-06-06T03:40:22.200 INFO:tasks.workunit.client.0.smithi040.stderr:112500K .......... .......... .......... .......... .......... 82% 17.2M 1s 2024-06-06T03:40:22.201 INFO:tasks.workunit.client.0.smithi040.stderr:112550K .......... .......... .......... .......... .......... 82% 27.8M 1s 2024-06-06T03:40:22.208 INFO:tasks.workunit.client.0.smithi040.stderr:112600K .......... .......... .......... .......... .......... 82% 25.9M 1s 2024-06-06T03:40:22.208 INFO:tasks.workunit.client.0.smithi040.stderr:112650K .......... .......... .......... .......... .......... 83% 29.8M 1s 2024-06-06T03:40:22.208 INFO:tasks.workunit.client.0.smithi040.stderr:112700K .......... .......... .......... .......... .......... 83% 31.9M 1s 2024-06-06T03:40:22.214 INFO:tasks.workunit.client.0.smithi040.stderr:112750K .......... .......... .......... .......... .......... 83% 17.3M 1s 2024-06-06T03:40:22.215 INFO:tasks.workunit.client.0.smithi040.stderr:112800K .......... .......... .......... .......... .......... 83% 22.6M 1s 2024-06-06T03:40:22.215 INFO:tasks.workunit.client.0.smithi040.stderr:112850K .......... .......... .......... .......... .......... 83% 36.9M 1s 2024-06-06T03:40:22.218 INFO:tasks.workunit.client.0.smithi040.stderr:112900K .......... .......... .......... .......... .......... 83% 19.3M 1s 2024-06-06T03:40:22.218 INFO:tasks.workunit.client.0.smithi040.stderr:112950K .......... .......... .......... .......... .......... 83% 22.8M 1s 2024-06-06T03:40:22.236 INFO:tasks.workunit.client.0.smithi040.stderr:113000K .......... .......... .......... .......... .......... 83% 20.0M 1s 2024-06-06T03:40:22.236 INFO:tasks.workunit.client.0.smithi040.stderr:113050K .......... .......... .......... .......... .......... 83% 23.4M 1s 2024-06-06T03:40:22.237 INFO:tasks.workunit.client.0.smithi040.stderr:113100K .......... .......... .......... .......... .......... 83% 36.2M 1s 2024-06-06T03:40:22.237 INFO:tasks.workunit.client.0.smithi040.stderr:113150K .......... .......... .......... .......... .......... 83% 22.9M 1s 2024-06-06T03:40:22.237 INFO:tasks.workunit.client.0.smithi040.stderr:113200K .......... .......... .......... .......... .......... 83% 22.8M 1s 2024-06-06T03:40:22.237 INFO:tasks.workunit.client.0.smithi040.stderr:113250K .......... .......... .......... .......... .......... 83% 39.0M 1s 2024-06-06T03:40:22.237 INFO:tasks.workunit.client.0.smithi040.stderr:113300K .......... .......... .......... .......... .......... 83% 19.6M 1s 2024-06-06T03:40:22.237 INFO:tasks.workunit.client.0.smithi040.stderr:113350K .......... .......... .......... .......... .......... 83% 30.0M 1s 2024-06-06T03:40:22.237 INFO:tasks.workunit.client.0.smithi040.stderr:113400K .......... .......... .......... .......... .......... 83% 20.5M 1s 2024-06-06T03:40:22.243 INFO:tasks.workunit.client.0.smithi040.stderr:113450K .......... .......... .......... .......... .......... 83% 17.7M 1s 2024-06-06T03:40:22.243 INFO:tasks.workunit.client.0.smithi040.stderr:113500K .......... .......... .......... .......... .......... 83% 37.9M 1s 2024-06-06T03:40:22.244 INFO:tasks.workunit.client.0.smithi040.stderr:113550K .......... .......... .......... .......... .......... 83% 17.4M 1s 2024-06-06T03:40:22.245 INFO:tasks.workunit.client.0.smithi040.stderr:113600K .......... .......... .......... .......... .......... 83% 25.5M 1s 2024-06-06T03:40:22.248 INFO:tasks.workunit.client.0.smithi040.stderr:113650K .......... .......... .......... .......... .......... 83% 25.1M 1s 2024-06-06T03:40:22.249 INFO:tasks.workunit.client.0.smithi040.stderr:113700K .......... .......... .......... .......... .......... 83% 20.3M 1s 2024-06-06T03:40:22.251 INFO:tasks.workunit.client.0.smithi040.stderr:113750K .......... .......... .......... .......... .......... 83% 29.7M 1s 2024-06-06T03:40:22.253 INFO:tasks.workunit.client.0.smithi040.stderr:113800K .......... .......... .......... .......... .......... 83% 21.0M 1s 2024-06-06T03:40:22.262 INFO:tasks.workunit.client.0.smithi040.stderr:113850K .......... .......... .......... .......... .......... 83% 25.3M 1s 2024-06-06T03:40:22.262 INFO:tasks.workunit.client.0.smithi040.stderr:113900K .......... .......... .......... .......... .......... 83% 22.2M 1s 2024-06-06T03:40:22.262 INFO:tasks.workunit.client.0.smithi040.stderr:113950K .......... .......... .......... .......... .......... 83% 21.5M 1s 2024-06-06T03:40:22.262 INFO:tasks.workunit.client.0.smithi040.stderr:114000K .......... .......... .......... .......... .......... 84% 39.7M 1s 2024-06-06T03:40:22.264 INFO:tasks.workunit.client.0.smithi040.stderr:114050K .......... .......... .......... .......... .......... 84% 19.6M 1s 2024-06-06T03:40:22.265 INFO:tasks.workunit.client.0.smithi040.stderr:114100K .......... .......... .......... .......... .......... 84% 21.4M 1s 2024-06-06T03:40:22.274 INFO:tasks.workunit.client.0.smithi040.stderr:114150K .......... .......... .......... .......... .......... 84% 35.5M 1s 2024-06-06T03:40:22.274 INFO:tasks.workunit.client.0.smithi040.stderr:114200K .......... .......... .......... .......... .......... 84% 23.0M 1s 2024-06-06T03:40:22.274 INFO:tasks.workunit.client.0.smithi040.stderr:114250K .......... .......... .......... .......... .......... 84% 31.8M 1s 2024-06-06T03:40:22.274 INFO:tasks.workunit.client.0.smithi040.stderr:114300K .......... .......... .......... .......... .......... 84% 24.4M 1s 2024-06-06T03:40:22.276 INFO:tasks.workunit.client.0.smithi040.stderr:114350K .......... .......... .......... .......... .......... 84% 20.2M 1s 2024-06-06T03:40:22.277 INFO:tasks.workunit.client.0.smithi040.stderr:114400K .......... .......... .......... .......... .......... 84% 30.3M 1s 2024-06-06T03:40:22.292 INFO:tasks.workunit.client.0.smithi040.stderr:114450K .......... .......... .......... .......... .......... 84% 19.8M 1s 2024-06-06T03:40:22.292 INFO:tasks.workunit.client.0.smithi040.stderr:114500K .......... .......... .......... .......... .......... 84% 36.2M 1s 2024-06-06T03:40:22.293 INFO:tasks.workunit.client.0.smithi040.stderr:114550K .......... .......... .......... .......... .......... 84% 28.5M 1s 2024-06-06T03:40:22.293 INFO:tasks.workunit.client.0.smithi040.stderr:114600K .......... .......... .......... .......... .......... 84% 27.9M 1s 2024-06-06T03:40:22.293 INFO:tasks.workunit.client.0.smithi040.stderr:114650K .......... .......... .......... .......... .......... 84% 19.2M 1s 2024-06-06T03:40:22.293 INFO:tasks.workunit.client.0.smithi040.stderr:114700K .......... .......... .......... .......... .......... 84% 21.8M 1s 2024-06-06T03:40:22.293 INFO:tasks.workunit.client.0.smithi040.stderr:114750K .......... .......... .......... .......... .......... 84% 34.3M 1s 2024-06-06T03:40:22.293 INFO:tasks.workunit.client.0.smithi040.stderr:114800K .......... .......... .......... .......... .......... 84% 23.1M 1s 2024-06-06T03:40:22.297 INFO:tasks.workunit.client.0.smithi040.stderr:114850K .......... .......... .......... .......... .......... 84% 21.3M 1s 2024-06-06T03:40:22.297 INFO:tasks.workunit.client.0.smithi040.stderr:114900K .......... .......... .......... .......... .......... 84% 34.7M 1s 2024-06-06T03:40:22.306 INFO:tasks.workunit.client.0.smithi040.stderr:114950K .......... .......... .......... .......... .......... 84% 14.0M 1s 2024-06-06T03:40:22.306 INFO:tasks.workunit.client.0.smithi040.stderr:115000K .......... .......... .......... .......... .......... 84% 35.2M 1s 2024-06-06T03:40:22.306 INFO:tasks.workunit.client.0.smithi040.stderr:115050K .......... .......... .......... .......... .......... 84% 25.9M 1s 2024-06-06T03:40:22.306 INFO:tasks.workunit.client.0.smithi040.stderr:115100K .......... .......... .......... .......... .......... 84% 22.3M 1s 2024-06-06T03:40:22.306 INFO:tasks.workunit.client.0.smithi040.stderr:115150K .......... .......... .......... .......... .......... 84% 35.1M 1s 2024-06-06T03:40:22.308 INFO:tasks.workunit.client.0.smithi040.stderr:115200K .......... .......... .......... .......... .......... 84% 18.7M 1s 2024-06-06T03:40:22.315 INFO:tasks.workunit.client.0.smithi040.stderr:115250K .......... .......... .......... .......... .......... 84% 28.6M 1s 2024-06-06T03:40:22.315 INFO:tasks.workunit.client.0.smithi040.stderr:115300K .......... .......... .......... .......... .......... 84% 25.9M 1s 2024-06-06T03:40:22.315 INFO:tasks.workunit.client.0.smithi040.stderr:115350K .......... .......... .......... .......... .......... 85% 19.4M 1s 2024-06-06T03:40:22.317 INFO:tasks.workunit.client.0.smithi040.stderr:115400K .......... .......... .......... .......... .......... 85% 31.2M 1s 2024-06-06T03:40:22.318 INFO:tasks.workunit.client.0.smithi040.stderr:115450K .......... .......... .......... .......... .......... 85% 25.3M 1s 2024-06-06T03:40:22.320 INFO:tasks.workunit.client.0.smithi040.stderr:115500K .......... .......... .......... .......... .......... 85% 31.9M 1s 2024-06-06T03:40:22.322 INFO:tasks.workunit.client.0.smithi040.stderr:115550K .......... .......... .......... .......... .......... 85% 19.9M 1s 2024-06-06T03:40:22.329 INFO:tasks.workunit.client.0.smithi040.stderr:115600K .......... .......... .......... .......... .......... 85% 19.6M 1s 2024-06-06T03:40:22.329 INFO:tasks.workunit.client.0.smithi040.stderr:115650K .......... .......... .......... .......... .......... 85% 34.4M 1s 2024-06-06T03:40:22.329 INFO:tasks.workunit.client.0.smithi040.stderr:115700K .......... .......... .......... .......... .......... 85% 19.2M 1s 2024-06-06T03:40:22.334 INFO:tasks.workunit.client.0.smithi040.stderr:115750K .......... .......... .......... .......... .......... 85% 23.3M 1s 2024-06-06T03:40:22.335 INFO:tasks.workunit.client.0.smithi040.stderr:115800K .......... .......... .......... .......... .......... 85% 19.9M 1s 2024-06-06T03:40:22.336 INFO:tasks.workunit.client.0.smithi040.stderr:115850K .......... .......... .......... .......... .......... 85% 22.4M 1s 2024-06-06T03:40:22.336 INFO:tasks.workunit.client.0.smithi040.stderr:115900K .......... .......... .......... .......... .......... 85% 31.7M 1s 2024-06-06T03:40:22.339 INFO:tasks.workunit.client.0.smithi040.stderr:115950K .......... .......... .......... .......... .......... 85% 23.7M 1s 2024-06-06T03:40:22.342 INFO:tasks.workunit.client.0.smithi040.stderr:116000K .......... .......... .......... .......... .......... 85% 25.2M 1s 2024-06-06T03:40:22.343 INFO:tasks.workunit.client.0.smithi040.stderr:116050K .......... .......... .......... .......... .......... 85% 19.4M 1s 2024-06-06T03:40:22.393 INFO:tasks.workunit.client.0.smithi040.stderr:116100K .......... .......... .......... .......... .......... 85% 31.7M 1s 2024-06-06T03:40:22.393 INFO:tasks.workunit.client.0.smithi040.stderr:116150K .......... .......... .......... .......... .......... 85% 22.6M 1s 2024-06-06T03:40:22.393 INFO:tasks.workunit.client.0.smithi040.stderr:116200K .......... .......... .......... .......... .......... 85% 25.0M 1s 2024-06-06T03:40:22.394 INFO:tasks.workunit.client.0.smithi040.stderr:116250K .......... .......... .......... .......... .......... 85% 34.5M 1s 2024-06-06T03:40:22.394 INFO:tasks.workunit.client.0.smithi040.stderr:116300K .......... .......... .......... .......... .......... 85% 18.9M 1s 2024-06-06T03:40:22.394 INFO:tasks.workunit.client.0.smithi040.stderr:116350K .......... .......... .......... .......... .......... 85% 31.2M 1s 2024-06-06T03:40:22.394 INFO:tasks.workunit.client.0.smithi040.stderr:116400K .......... .......... .......... .......... .......... 85% 24.8M 1s 2024-06-06T03:40:22.394 INFO:tasks.workunit.client.0.smithi040.stderr:116450K .......... .......... .......... .......... .......... 85% 21.3M 1s 2024-06-06T03:40:22.394 INFO:tasks.workunit.client.0.smithi040.stderr:116500K .......... .......... .......... .......... .......... 85% 30.4M 1s 2024-06-06T03:40:22.394 INFO:tasks.workunit.client.0.smithi040.stderr:116550K .......... .......... .......... .......... .......... 85% 23.7M 1s 2024-06-06T03:40:22.394 INFO:tasks.workunit.client.0.smithi040.stderr:116600K .......... .......... .......... .......... .......... 85% 22.4M 1s 2024-06-06T03:40:22.394 INFO:tasks.workunit.client.0.smithi040.stderr:116650K .......... .......... .......... .......... .......... 85% 25.4M 1s 2024-06-06T03:40:22.394 INFO:tasks.workunit.client.0.smithi040.stderr:116700K .......... .......... .......... .......... .......... 86% 23.9M 1s 2024-06-06T03:40:22.395 INFO:tasks.workunit.client.0.smithi040.stderr:116750K .......... .......... .......... .......... .......... 86% 32.8M 1s 2024-06-06T03:40:22.395 INFO:tasks.workunit.client.0.smithi040.stderr:116800K .......... .......... .......... .......... .......... 86% 20.3M 1s 2024-06-06T03:40:22.395 INFO:tasks.workunit.client.0.smithi040.stderr:116850K .......... .......... .......... .......... .......... 86% 32.7M 1s 2024-06-06T03:40:22.395 INFO:tasks.workunit.client.0.smithi040.stderr:116900K .......... .......... .......... .......... .......... 86% 23.3M 1s 2024-06-06T03:40:22.395 INFO:tasks.workunit.client.0.smithi040.stderr:116950K .......... .......... .......... .......... .......... 86% 13.9M 1s 2024-06-06T03:40:22.395 INFO:tasks.workunit.client.0.smithi040.stderr:117000K .......... .......... .......... .......... .......... 86% 43.9M 1s 2024-06-06T03:40:22.395 INFO:tasks.workunit.client.0.smithi040.stderr:117050K .......... .......... .......... .......... .......... 86% 23.7M 1s 2024-06-06T03:40:22.395 INFO:tasks.workunit.client.0.smithi040.stderr:117100K .......... .......... .......... .......... .......... 86% 32.0M 1s 2024-06-06T03:40:22.395 INFO:tasks.workunit.client.0.smithi040.stderr:117150K .......... .......... .......... .......... .......... 86% 22.5M 1s 2024-06-06T03:40:22.395 INFO:tasks.workunit.client.0.smithi040.stderr:117200K .......... .......... .......... .......... .......... 86% 18.3M 1s 2024-06-06T03:40:22.395 INFO:tasks.workunit.client.0.smithi040.stderr:117250K .......... .......... .......... .......... .......... 86% 34.1M 1s 2024-06-06T03:40:22.395 INFO:tasks.workunit.client.0.smithi040.stderr:117300K .......... .......... .......... .......... .......... 86% 20.1M 1s 2024-06-06T03:40:22.400 INFO:tasks.workunit.client.0.smithi040.stderr:117350K .......... .......... .......... .......... .......... 86% 32.7M 1s 2024-06-06T03:40:22.400 INFO:tasks.workunit.client.0.smithi040.stderr:117400K .......... .......... .......... .......... .......... 86% 19.8M 1s 2024-06-06T03:40:22.401 INFO:tasks.workunit.client.0.smithi040.stderr:117450K .......... .......... .......... .......... .......... 86% 20.2M 1s 2024-06-06T03:40:22.401 INFO:tasks.workunit.client.0.smithi040.stderr:117500K .......... .......... .......... .......... .......... 86% 56.7M 1s 2024-06-06T03:40:22.403 INFO:tasks.workunit.client.0.smithi040.stderr:117550K .......... .......... .......... .......... .......... 86% 16.4M 1s 2024-06-06T03:40:22.405 INFO:tasks.workunit.client.0.smithi040.stderr:117600K .......... .......... .......... .......... .......... 86% 23.4M 1s 2024-06-06T03:40:22.407 INFO:tasks.workunit.client.0.smithi040.stderr:117650K .......... .......... .......... .......... .......... 86% 39.3M 1s 2024-06-06T03:40:22.411 INFO:tasks.workunit.client.0.smithi040.stderr:117700K .......... .......... .......... .......... .......... 86% 20.1M 1s 2024-06-06T03:40:22.411 INFO:tasks.workunit.client.0.smithi040.stderr:117750K .......... .......... .......... .......... .......... 86% 29.7M 1s 2024-06-06T03:40:22.463 INFO:tasks.workunit.client.0.smithi040.stderr:117800K .......... .......... .......... .......... .......... 86% 16.9M 1s 2024-06-06T03:40:22.463 INFO:tasks.workunit.client.0.smithi040.stderr:117850K .......... .......... .......... .......... .......... 86% 25.5M 1s 2024-06-06T03:40:22.463 INFO:tasks.workunit.client.0.smithi040.stderr:117900K .......... .......... .......... .......... .......... 86% 19.8M 1s 2024-06-06T03:40:22.463 INFO:tasks.workunit.client.0.smithi040.stderr:117950K .......... .......... .......... .......... .......... 86% 46.9M 1s 2024-06-06T03:40:22.463 INFO:tasks.workunit.client.0.smithi040.stderr:118000K .......... .......... .......... .......... .......... 86% 25.5M 1s 2024-06-06T03:40:22.463 INFO:tasks.workunit.client.0.smithi040.stderr:118050K .......... .......... .......... .......... .......... 86% 19.2M 1s 2024-06-06T03:40:22.463 INFO:tasks.workunit.client.0.smithi040.stderr:118100K .......... .......... .......... .......... .......... 87% 29.2M 1s 2024-06-06T03:40:22.463 INFO:tasks.workunit.client.0.smithi040.stderr:118150K .......... .......... .......... .......... .......... 87% 21.8M 1s 2024-06-06T03:40:22.463 INFO:tasks.workunit.client.0.smithi040.stderr:118200K .......... .......... .......... .......... .......... 87% 30.5M 1s 2024-06-06T03:40:22.463 INFO:tasks.workunit.client.0.smithi040.stderr:118250K .......... .......... .......... .......... .......... 87% 19.0M 1s 2024-06-06T03:40:22.464 INFO:tasks.workunit.client.0.smithi040.stderr:118300K .......... .......... .......... .......... .......... 87% 39.3M 1s 2024-06-06T03:40:22.464 INFO:tasks.workunit.client.0.smithi040.stderr:118350K .......... .......... .......... .......... .......... 87% 23.1M 1s 2024-06-06T03:40:22.464 INFO:tasks.workunit.client.0.smithi040.stderr:118400K .......... .......... .......... .......... .......... 87% 25.3M 1s 2024-06-06T03:40:22.464 INFO:tasks.workunit.client.0.smithi040.stderr:118450K .......... .......... .......... .......... .......... 87% 23.8M 1s 2024-06-06T03:40:22.464 INFO:tasks.workunit.client.0.smithi040.stderr:118500K .......... .......... .......... .......... .......... 87% 26.1M 1s 2024-06-06T03:40:22.464 INFO:tasks.workunit.client.0.smithi040.stderr:118550K .......... .......... .......... .......... .......... 87% 31.9M 1s 2024-06-06T03:40:22.464 INFO:tasks.workunit.client.0.smithi040.stderr:118600K .......... .......... .......... .......... .......... 87% 18.7M 1s 2024-06-06T03:40:22.464 INFO:tasks.workunit.client.0.smithi040.stderr:118650K .......... .......... .......... .......... .......... 87% 33.3M 1s 2024-06-06T03:40:22.464 INFO:tasks.workunit.client.0.smithi040.stderr:118700K .......... .......... .......... .......... .......... 87% 18.1M 1s 2024-06-06T03:40:22.464 INFO:tasks.workunit.client.0.smithi040.stderr:118750K .......... .......... .......... .......... .......... 87% 23.1M 1s 2024-06-06T03:40:22.464 INFO:tasks.workunit.client.0.smithi040.stderr:118800K .......... .......... .......... .......... .......... 87% 45.0M 1s 2024-06-06T03:40:22.464 INFO:tasks.workunit.client.0.smithi040.stderr:118850K .......... .......... .......... .......... .......... 87% 24.8M 1s 2024-06-06T03:40:22.464 INFO:tasks.workunit.client.0.smithi040.stderr:118900K .......... .......... .......... .......... .......... 87% 27.4M 1s 2024-06-06T03:40:22.465 INFO:tasks.workunit.client.0.smithi040.stderr:118950K .......... .......... .......... .......... .......... 87% 14.8M 1s 2024-06-06T03:40:22.465 INFO:tasks.workunit.client.0.smithi040.stderr:119000K .......... .......... .......... .......... .......... 87% 32.9M 1s 2024-06-06T03:40:22.465 INFO:tasks.workunit.client.0.smithi040.stderr:119050K .......... .......... .......... .......... .......... 87% 25.7M 1s 2024-06-06T03:40:22.471 INFO:tasks.workunit.client.0.smithi040.stderr:119100K .......... .......... .......... .......... .......... 87% 27.1M 1s 2024-06-06T03:40:22.472 INFO:tasks.workunit.client.0.smithi040.stderr:119150K .......... .......... .......... .......... .......... 87% 31.3M 1s 2024-06-06T03:40:22.472 INFO:tasks.workunit.client.0.smithi040.stderr:119200K .......... .......... .......... .......... .......... 87% 15.9M 1s 2024-06-06T03:40:22.472 INFO:tasks.workunit.client.0.smithi040.stderr:119250K .......... .......... .......... .......... .......... 87% 23.5M 1s 2024-06-06T03:40:22.480 INFO:tasks.workunit.client.0.smithi040.stderr:119300K .......... .......... .......... .......... .......... 87% 30.9M 1s 2024-06-06T03:40:22.480 INFO:tasks.workunit.client.0.smithi040.stderr:119350K .......... .......... .......... .......... .......... 87% 24.0M 1s 2024-06-06T03:40:22.481 INFO:tasks.workunit.client.0.smithi040.stderr:119400K .......... .......... .......... .......... .......... 87% 31.7M 1s 2024-06-06T03:40:22.481 INFO:tasks.workunit.client.0.smithi040.stderr:119450K .......... .......... .......... .......... .......... 88% 25.0M 1s 2024-06-06T03:40:22.495 INFO:tasks.workunit.client.0.smithi040.stderr:119500K .......... .......... .......... .......... .......... 88% 16.5M 1s 2024-06-06T03:40:22.495 INFO:tasks.workunit.client.0.smithi040.stderr:119550K .......... .......... .......... .......... .......... 88% 29.6M 1s 2024-06-06T03:40:22.495 INFO:tasks.workunit.client.0.smithi040.stderr:119600K .......... .......... .......... .......... .......... 88% 23.3M 1s 2024-06-06T03:40:22.495 INFO:tasks.workunit.client.0.smithi040.stderr:119650K .......... .......... .......... .......... .......... 88% 21.9M 1s 2024-06-06T03:40:22.495 INFO:tasks.workunit.client.0.smithi040.stderr:119700K .......... .......... .......... .......... .......... 88% 33.1M 1s 2024-06-06T03:40:22.495 INFO:tasks.workunit.client.0.smithi040.stderr:119750K .......... .......... .......... .......... .......... 88% 17.0M 1s 2024-06-06T03:40:22.496 INFO:tasks.workunit.client.0.smithi040.stderr:119800K .......... .......... .......... .......... .......... 88% 37.9M 1s 2024-06-06T03:40:22.496 INFO:tasks.workunit.client.0.smithi040.stderr:119850K .......... .......... .......... .......... .......... 88% 20.8M 1s 2024-06-06T03:40:22.500 INFO:tasks.workunit.client.0.smithi040.stderr:119900K .......... .......... .......... .......... .......... 88% 20.5M 1s 2024-06-06T03:40:22.501 INFO:tasks.workunit.client.0.smithi040.stderr:119950K .......... .......... .......... .......... .......... 88% 37.1M 1s 2024-06-06T03:40:22.501 INFO:tasks.workunit.client.0.smithi040.stderr:120000K .......... .......... .......... .......... .......... 88% 25.6M 1s 2024-06-06T03:40:22.504 INFO:tasks.workunit.client.0.smithi040.stderr:120050K .......... .......... .......... .......... .......... 88% 24.6M 1s 2024-06-06T03:40:22.505 INFO:tasks.workunit.client.0.smithi040.stderr:120100K .......... .......... .......... .......... .......... 88% 20.9M 1s 2024-06-06T03:40:22.509 INFO:tasks.workunit.client.0.smithi040.stderr:120150K .......... .......... .......... .......... .......... 88% 26.5M 1s 2024-06-06T03:40:22.509 INFO:tasks.workunit.client.0.smithi040.stderr:120200K .......... .......... .......... .......... .......... 88% 20.4M 1s 2024-06-06T03:40:22.511 INFO:tasks.workunit.client.0.smithi040.stderr:120250K .......... .......... .......... .......... .......... 88% 32.3M 1s 2024-06-06T03:40:22.515 INFO:tasks.workunit.client.0.smithi040.stderr:120300K .......... .......... .......... .......... .......... 88% 28.6M 1s 2024-06-06T03:40:22.515 INFO:tasks.workunit.client.0.smithi040.stderr:120350K .......... .......... .......... .......... .......... 88% 20.8M 1s 2024-06-06T03:40:22.522 INFO:tasks.workunit.client.0.smithi040.stderr:120400K .......... .......... .......... .......... .......... 88% 20.1M 1s 2024-06-06T03:40:22.522 INFO:tasks.workunit.client.0.smithi040.stderr:120450K .......... .......... .......... .......... .......... 88% 26.4M 1s 2024-06-06T03:40:22.522 INFO:tasks.workunit.client.0.smithi040.stderr:120500K .......... .......... .......... .......... .......... 88% 31.1M 1s 2024-06-06T03:40:22.522 INFO:tasks.workunit.client.0.smithi040.stderr:120550K .......... .......... .......... .......... .......... 88% 36.5M 1s 2024-06-06T03:40:22.526 INFO:tasks.workunit.client.0.smithi040.stderr:120600K .......... .......... .......... .......... .......... 88% 18.2M 1s 2024-06-06T03:40:22.527 INFO:tasks.workunit.client.0.smithi040.stderr:120650K .......... .......... .......... .......... .......... 88% 23.3M 1s 2024-06-06T03:40:22.528 INFO:tasks.workunit.client.0.smithi040.stderr:120700K .......... .......... .......... .......... .......... 88% 26.9M 1s 2024-06-06T03:40:22.533 INFO:tasks.workunit.client.0.smithi040.stderr:120750K .......... .......... .......... .......... .......... 88% 21.7M 1s 2024-06-06T03:40:22.533 INFO:tasks.workunit.client.0.smithi040.stderr:120800K .......... .......... .......... .......... .......... 89% 42.8M 1s 2024-06-06T03:40:22.534 INFO:tasks.workunit.client.0.smithi040.stderr:120850K .......... .......... .......... .......... .......... 89% 22.7M 1s 2024-06-06T03:40:22.538 INFO:tasks.workunit.client.0.smithi040.stderr:120900K .......... .......... .......... .......... .......... 89% 21.8M 1s 2024-06-06T03:40:22.541 INFO:tasks.workunit.client.0.smithi040.stderr:120950K .......... .......... .......... .......... .......... 89% 18.8M 1s 2024-06-06T03:40:22.541 INFO:tasks.workunit.client.0.smithi040.stderr:121000K .......... .......... .......... .......... .......... 89% 22.7M 1s 2024-06-06T03:40:22.549 INFO:tasks.workunit.client.0.smithi040.stderr:121050K .......... .......... .......... .......... .......... 89% 44.6M 1s 2024-06-06T03:40:22.549 INFO:tasks.workunit.client.0.smithi040.stderr:121100K .......... .......... .......... .......... .......... 89% 25.1M 1s 2024-06-06T03:40:22.549 INFO:tasks.workunit.client.0.smithi040.stderr:121150K .......... .......... .......... .......... .......... 89% 22.0M 1s 2024-06-06T03:40:22.549 INFO:tasks.workunit.client.0.smithi040.stderr:121200K .......... .......... .......... .......... .......... 89% 19.8M 1s 2024-06-06T03:40:22.550 INFO:tasks.workunit.client.0.smithi040.stderr:121250K .......... .......... .......... .......... .......... 89% 29.3M 1s 2024-06-06T03:40:22.552 INFO:tasks.workunit.client.0.smithi040.stderr:121300K .......... .......... .......... .......... .......... 89% 25.8M 1s 2024-06-06T03:40:22.554 INFO:tasks.workunit.client.0.smithi040.stderr:121350K .......... .......... .......... .......... .......... 89% 24.2M 1s 2024-06-06T03:40:22.557 INFO:tasks.workunit.client.0.smithi040.stderr:121400K .......... .......... .......... .......... .......... 89% 19.0M 1s 2024-06-06T03:40:22.561 INFO:tasks.workunit.client.0.smithi040.stderr:121450K .......... .......... .......... .......... .......... 89% 27.6M 1s 2024-06-06T03:40:22.562 INFO:tasks.workunit.client.0.smithi040.stderr:121500K .......... .......... .......... .......... .......... 89% 23.0M 1s 2024-06-06T03:40:22.567 INFO:tasks.workunit.client.0.smithi040.stderr:121550K .......... .......... .......... .......... .......... 89% 27.8M 1s 2024-06-06T03:40:22.567 INFO:tasks.workunit.client.0.smithi040.stderr:121600K .......... .......... .......... .......... .......... 89% 24.2M 1s 2024-06-06T03:40:22.567 INFO:tasks.workunit.client.0.smithi040.stderr:121650K .......... .......... .......... .......... .......... 89% 21.2M 1s 2024-06-06T03:40:22.571 INFO:tasks.workunit.client.0.smithi040.stderr:121700K .......... .......... .......... .......... .......... 89% 35.3M 1s 2024-06-06T03:40:22.577 INFO:tasks.workunit.client.0.smithi040.stderr:121750K .......... .......... .......... .......... .......... 89% 15.7M 1s 2024-06-06T03:40:22.577 INFO:tasks.workunit.client.0.smithi040.stderr:121800K .......... .......... .......... .......... .......... 89% 42.3M 1s 2024-06-06T03:40:22.577 INFO:tasks.workunit.client.0.smithi040.stderr:121850K .......... .......... .......... .......... .......... 89% 21.6M 1s 2024-06-06T03:40:22.628 INFO:tasks.workunit.client.0.smithi040.stderr:121900K .......... .......... .......... .......... .......... 89% 15.6M 1s 2024-06-06T03:40:22.629 INFO:tasks.workunit.client.0.smithi040.stderr:121950K .......... .......... .......... .......... .......... 89% 42.0M 1s 2024-06-06T03:40:22.629 INFO:tasks.workunit.client.0.smithi040.stderr:122000K .......... .......... .......... .......... .......... 89% 18.7M 1s 2024-06-06T03:40:22.629 INFO:tasks.workunit.client.0.smithi040.stderr:122050K .......... .......... .......... .......... .......... 89% 43.1M 1s 2024-06-06T03:40:22.629 INFO:tasks.workunit.client.0.smithi040.stderr:122100K .......... .......... .......... .......... .......... 89% 25.7M 1s 2024-06-06T03:40:22.629 INFO:tasks.workunit.client.0.smithi040.stderr:122150K .......... .......... .......... .......... .......... 90% 28.9M 1s 2024-06-06T03:40:22.629 INFO:tasks.workunit.client.0.smithi040.stderr:122200K .......... .......... .......... .......... .......... 90% 25.7M 1s 2024-06-06T03:40:22.629 INFO:tasks.workunit.client.0.smithi040.stderr:122250K .......... .......... .......... .......... .......... 90% 22.7M 1s 2024-06-06T03:40:22.629 INFO:tasks.workunit.client.0.smithi040.stderr:122300K .......... .......... .......... .......... .......... 90% 26.0M 1s 2024-06-06T03:40:22.629 INFO:tasks.workunit.client.0.smithi040.stderr:122350K .......... .......... .......... .......... .......... 90% 22.0M 1s 2024-06-06T03:40:22.629 INFO:tasks.workunit.client.0.smithi040.stderr:122400K .......... .......... .......... .......... .......... 90% 22.7M 1s 2024-06-06T03:40:22.629 INFO:tasks.workunit.client.0.smithi040.stderr:122450K .......... .......... .......... .......... .......... 90% 23.6M 1s 2024-06-06T03:40:22.630 INFO:tasks.workunit.client.0.smithi040.stderr:122500K .......... .......... .......... .......... .......... 90% 29.6M 1s 2024-06-06T03:40:22.630 INFO:tasks.workunit.client.0.smithi040.stderr:122550K .......... .......... .......... .......... .......... 90% 36.1M 1s 2024-06-06T03:40:22.630 INFO:tasks.workunit.client.0.smithi040.stderr:122600K .......... .......... .......... .......... .......... 90% 18.0M 1s 2024-06-06T03:40:22.630 INFO:tasks.workunit.client.0.smithi040.stderr:122650K .......... .......... .......... .......... .......... 90% 27.3M 1s 2024-06-06T03:40:22.630 INFO:tasks.workunit.client.0.smithi040.stderr:122700K .......... .......... .......... .......... .......... 90% 24.1M 1s 2024-06-06T03:40:22.630 INFO:tasks.workunit.client.0.smithi040.stderr:122750K .......... .......... .......... .......... .......... 90% 25.9M 1s 2024-06-06T03:40:22.630 INFO:tasks.workunit.client.0.smithi040.stderr:122800K .......... .......... .......... .......... .......... 90% 32.5M 1s 2024-06-06T03:40:22.630 INFO:tasks.workunit.client.0.smithi040.stderr:122850K .......... .......... .......... .......... .......... 90% 21.9M 1s 2024-06-06T03:40:22.630 INFO:tasks.workunit.client.0.smithi040.stderr:122900K .......... .......... .......... .......... .......... 90% 29.3M 1s 2024-06-06T03:40:22.630 INFO:tasks.workunit.client.0.smithi040.stderr:122950K .......... .......... .......... .......... .......... 90% 14.1M 1s 2024-06-06T03:40:22.633 INFO:tasks.workunit.client.0.smithi040.stderr:123000K .......... .......... .......... .......... .......... 90% 34.3M 1s 2024-06-06T03:40:22.634 INFO:tasks.workunit.client.0.smithi040.stderr:123050K .......... .......... .......... .......... .......... 90% 27.5M 1s 2024-06-06T03:40:22.634 INFO:tasks.workunit.client.0.smithi040.stderr:123100K .......... .......... .......... .......... .......... 90% 23.3M 1s 2024-06-06T03:40:22.634 INFO:tasks.workunit.client.0.smithi040.stderr:123150K .......... .......... .......... .......... .......... 90% 28.3M 1s 2024-06-06T03:40:22.658 INFO:tasks.workunit.client.0.smithi040.stderr:123200K .......... .......... .......... .......... .......... 90% 16.6M 1s 2024-06-06T03:40:22.658 INFO:tasks.workunit.client.0.smithi040.stderr:123250K .......... .......... .......... .......... .......... 90% 23.5M 1s 2024-06-06T03:40:22.659 INFO:tasks.workunit.client.0.smithi040.stderr:123300K .......... .......... .......... .......... .......... 90% 31.6M 1s 2024-06-06T03:40:22.659 INFO:tasks.workunit.client.0.smithi040.stderr:123350K .......... .......... .......... .......... .......... 90% 24.9M 1s 2024-06-06T03:40:22.659 INFO:tasks.workunit.client.0.smithi040.stderr:123400K .......... .......... .......... .......... .......... 90% 31.3M 1s 2024-06-06T03:40:22.659 INFO:tasks.workunit.client.0.smithi040.stderr:123450K .......... .......... .......... .......... .......... 90% 24.3M 1s 2024-06-06T03:40:22.659 INFO:tasks.workunit.client.0.smithi040.stderr:123500K .......... .......... .......... .......... .......... 91% 17.1M 1s 2024-06-06T03:40:22.659 INFO:tasks.workunit.client.0.smithi040.stderr:123550K .......... .......... .......... .......... .......... 91% 21.5M 1s 2024-06-06T03:40:22.659 INFO:tasks.workunit.client.0.smithi040.stderr:123600K .......... .......... .......... .......... .......... 91% 38.1M 1s 2024-06-06T03:40:22.659 INFO:tasks.workunit.client.0.smithi040.stderr:123650K .......... .......... .......... .......... .......... 91% 23.5M 1s 2024-06-06T03:40:22.659 INFO:tasks.workunit.client.0.smithi040.stderr:123700K .......... .......... .......... .......... .......... 91% 19.3M 0s 2024-06-06T03:40:22.659 INFO:tasks.workunit.client.0.smithi040.stderr:123750K .......... .......... .......... .......... .......... 91% 40.0M 0s 2024-06-06T03:40:22.659 INFO:tasks.workunit.client.0.smithi040.stderr:123800K .......... .......... .......... .......... .......... 91% 15.1M 0s 2024-06-06T03:40:22.659 INFO:tasks.workunit.client.0.smithi040.stderr:123850K .......... .......... .......... .......... .......... 91% 35.9M 0s 2024-06-06T03:40:22.659 INFO:tasks.workunit.client.0.smithi040.stderr:123900K .......... .......... .......... .......... .......... 91% 24.9M 0s 2024-06-06T03:40:22.683 INFO:tasks.workunit.client.0.smithi040.stderr:123950K .......... .......... .......... .......... .......... 91% 21.6M 0s 2024-06-06T03:40:22.683 INFO:tasks.workunit.client.0.smithi040.stderr:124000K .......... .......... .......... .......... .......... 91% 26.0M 0s 2024-06-06T03:40:22.683 INFO:tasks.workunit.client.0.smithi040.stderr:124050K .......... .......... .......... .......... .......... 91% 20.7M 0s 2024-06-06T03:40:22.683 INFO:tasks.workunit.client.0.smithi040.stderr:124100K .......... .......... .......... .......... .......... 91% 37.6M 0s 2024-06-06T03:40:22.683 INFO:tasks.workunit.client.0.smithi040.stderr:124150K .......... .......... .......... .......... .......... 91% 24.0M 0s 2024-06-06T03:40:22.683 INFO:tasks.workunit.client.0.smithi040.stderr:124200K .......... .......... .......... .......... .......... 91% 30.9M 0s 2024-06-06T03:40:22.683 INFO:tasks.workunit.client.0.smithi040.stderr:124250K .......... .......... .......... .......... .......... 91% 20.5M 0s 2024-06-06T03:40:22.684 INFO:tasks.workunit.client.0.smithi040.stderr:124300K .......... .......... .......... .......... .......... 91% 20.8M 0s 2024-06-06T03:40:22.684 INFO:tasks.workunit.client.0.smithi040.stderr:124350K .......... .......... .......... .......... .......... 91% 33.9M 0s 2024-06-06T03:40:22.684 INFO:tasks.workunit.client.0.smithi040.stderr:124400K .......... .......... .......... .......... .......... 91% 19.7M 0s 2024-06-06T03:40:22.684 INFO:tasks.workunit.client.0.smithi040.stderr:124450K .......... .......... .......... .......... .......... 91% 37.8M 0s 2024-06-06T03:40:22.684 INFO:tasks.workunit.client.0.smithi040.stderr:124500K .......... .......... .......... .......... .......... 91% 23.6M 0s 2024-06-06T03:40:23.061 INFO:tasks.workunit.client.0.smithi040.stderr:124550K .......... .......... .......... .......... .......... 91% 278K 0s 2024-06-06T03:40:23.061 INFO:tasks.workunit.client.0.smithi040.stderr:124600K .......... .......... .......... .......... .......... 91% 453M 0s 2024-06-06T03:40:23.061 INFO:tasks.workunit.client.0.smithi040.stderr:124650K .......... .......... .......... .......... .......... 91% 443M 0s 2024-06-06T03:40:23.061 INFO:tasks.workunit.client.0.smithi040.stderr:124700K .......... .......... .......... .......... .......... 91% 331M 0s 2024-06-06T03:40:23.061 INFO:tasks.workunit.client.0.smithi040.stderr:124750K .......... .......... .......... .......... .......... 91% 357M 0s 2024-06-06T03:40:23.061 INFO:tasks.workunit.client.0.smithi040.stderr:124800K .......... .......... .......... .......... .......... 91% 355M 0s 2024-06-06T03:40:23.061 INFO:tasks.workunit.client.0.smithi040.stderr:124850K .......... .......... .......... .......... .......... 92% 309M 0s 2024-06-06T03:40:23.061 INFO:tasks.workunit.client.0.smithi040.stderr:124900K .......... .......... .......... .......... .......... 92% 439M 0s 2024-06-06T03:40:23.061 INFO:tasks.workunit.client.0.smithi040.stderr:124950K .......... .......... .......... .......... .......... 92% 469M 0s 2024-06-06T03:40:23.062 INFO:tasks.workunit.client.0.smithi040.stderr:125000K .......... .......... .......... .......... .......... 92% 386M 0s 2024-06-06T03:40:23.062 INFO:tasks.workunit.client.0.smithi040.stderr:125050K .......... .......... .......... .......... .......... 92% 402M 0s 2024-06-06T03:40:23.062 INFO:tasks.workunit.client.0.smithi040.stderr:125100K .......... .......... .......... .......... .......... 92% 441M 0s 2024-06-06T03:40:23.062 INFO:tasks.workunit.client.0.smithi040.stderr:125150K .......... .......... .......... .......... .......... 92% 466M 0s 2024-06-06T03:40:23.062 INFO:tasks.workunit.client.0.smithi040.stderr:125200K .......... .......... .......... .......... .......... 92% 474M 0s 2024-06-06T03:40:23.062 INFO:tasks.workunit.client.0.smithi040.stderr:125250K .......... .......... .......... .......... .......... 92% 445M 0s 2024-06-06T03:40:23.062 INFO:tasks.workunit.client.0.smithi040.stderr:125300K .......... .......... .......... .......... .......... 92% 540M 0s 2024-06-06T03:40:23.062 INFO:tasks.workunit.client.0.smithi040.stderr:125350K .......... .......... .......... .......... .......... 92% 539M 0s 2024-06-06T03:40:23.062 INFO:tasks.workunit.client.0.smithi040.stderr:125400K .......... .......... .......... .......... .......... 92% 490M 0s 2024-06-06T03:40:23.062 INFO:tasks.workunit.client.0.smithi040.stderr:125450K .......... .......... .......... .......... .......... 92% 413M 0s 2024-06-06T03:40:23.062 INFO:tasks.workunit.client.0.smithi040.stderr:125500K .......... .......... .......... .......... .......... 92% 456M 0s 2024-06-06T03:40:23.062 INFO:tasks.workunit.client.0.smithi040.stderr:125550K .......... .......... .......... .......... .......... 92% 486M 0s 2024-06-06T03:40:23.062 INFO:tasks.workunit.client.0.smithi040.stderr:125600K .......... .......... .......... .......... .......... 92% 500M 0s 2024-06-06T03:40:23.063 INFO:tasks.workunit.client.0.smithi040.stderr:125650K .......... .......... .......... .......... .......... 92% 475M 0s 2024-06-06T03:40:23.063 INFO:tasks.workunit.client.0.smithi040.stderr:125700K .......... .......... .......... .......... .......... 92% 512M 0s 2024-06-06T03:40:23.063 INFO:tasks.workunit.client.0.smithi040.stderr:125750K .......... .......... .......... .......... .......... 92% 519M 0s 2024-06-06T03:40:23.063 INFO:tasks.workunit.client.0.smithi040.stderr:125800K .......... .......... .......... .......... .......... 92% 542M 0s 2024-06-06T03:40:23.063 INFO:tasks.workunit.client.0.smithi040.stderr:125850K .......... .......... .......... .......... .......... 92% 415M 0s 2024-06-06T03:40:23.063 INFO:tasks.workunit.client.0.smithi040.stderr:125900K .......... .......... .......... .......... .......... 92% 463M 0s 2024-06-06T03:40:23.063 INFO:tasks.workunit.client.0.smithi040.stderr:125950K .......... .......... .......... .......... .......... 92% 456M 0s 2024-06-06T03:40:23.063 INFO:tasks.workunit.client.0.smithi040.stderr:126000K .......... .......... .......... .......... .......... 92% 450M 0s 2024-06-06T03:40:23.063 INFO:tasks.workunit.client.0.smithi040.stderr:126050K .......... .......... .......... .......... .......... 92% 410M 0s 2024-06-06T03:40:23.063 INFO:tasks.workunit.client.0.smithi040.stderr:126100K .......... .......... .......... .......... .......... 92% 542M 0s 2024-06-06T03:40:23.063 INFO:tasks.workunit.client.0.smithi040.stderr:126150K .......... .......... .......... .......... .......... 92% 437M 0s 2024-06-06T03:40:23.063 INFO:tasks.workunit.client.0.smithi040.stderr:126200K .......... .......... .......... .......... .......... 92% 454M 0s 2024-06-06T03:40:23.063 INFO:tasks.workunit.client.0.smithi040.stderr:126250K .......... .......... .......... .......... .......... 93% 422M 0s 2024-06-06T03:40:23.063 INFO:tasks.workunit.client.0.smithi040.stderr:126300K .......... .......... .......... .......... .......... 93% 446M 0s 2024-06-06T03:40:23.064 INFO:tasks.workunit.client.0.smithi040.stderr:126350K .......... .......... .......... .......... .......... 93% 520M 0s 2024-06-06T03:40:23.064 INFO:tasks.workunit.client.0.smithi040.stderr:126400K .......... .......... .......... .......... .......... 93% 496M 0s 2024-06-06T03:40:23.064 INFO:tasks.workunit.client.0.smithi040.stderr:126450K .......... .......... .......... .......... .......... 93% 375M 0s 2024-06-06T03:40:23.064 INFO:tasks.workunit.client.0.smithi040.stderr:126500K .......... .......... .......... .......... .......... 93% 500M 0s 2024-06-06T03:40:23.064 INFO:tasks.workunit.client.0.smithi040.stderr:126550K .......... .......... .......... .......... .......... 93% 480M 0s 2024-06-06T03:40:23.064 INFO:tasks.workunit.client.0.smithi040.stderr:126600K .......... .......... .......... .......... .......... 93% 439M 0s 2024-06-06T03:40:23.064 INFO:tasks.workunit.client.0.smithi040.stderr:126650K .......... .......... .......... .......... .......... 93% 391M 0s 2024-06-06T03:40:23.064 INFO:tasks.workunit.client.0.smithi040.stderr:126700K .......... .......... .......... .......... .......... 93% 469M 0s 2024-06-06T03:40:23.064 INFO:tasks.workunit.client.0.smithi040.stderr:126750K .......... .......... .......... .......... .......... 93% 477M 0s 2024-06-06T03:40:23.064 INFO:tasks.workunit.client.0.smithi040.stderr:126800K .......... .......... .......... .......... .......... 93% 466M 0s 2024-06-06T03:40:23.064 INFO:tasks.workunit.client.0.smithi040.stderr:126850K .......... .......... .......... .......... .......... 93% 433M 0s 2024-06-06T03:40:23.064 INFO:tasks.workunit.client.0.smithi040.stderr:126900K .......... .......... .......... .......... .......... 93% 451M 0s 2024-06-06T03:40:23.064 INFO:tasks.workunit.client.0.smithi040.stderr:126950K .......... .......... .......... .......... .......... 93% 466M 0s 2024-06-06T03:40:23.064 INFO:tasks.workunit.client.0.smithi040.stderr:127000K .......... .......... .......... .......... .......... 93% 483M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127050K .......... .......... .......... .......... .......... 93% 416M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127100K .......... .......... .......... .......... .......... 93% 460M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127150K .......... .......... .......... .......... .......... 93% 499M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127200K .......... .......... .......... .......... .......... 93% 467M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127250K .......... .......... .......... .......... .......... 93% 414M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127300K .......... .......... .......... .......... .......... 93% 520M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127350K .......... .......... .......... .......... .......... 93% 358M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127400K .......... .......... .......... .......... .......... 93% 446M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127450K .......... .......... .......... .......... .......... 93% 379M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127500K .......... .......... .......... .......... .......... 93% 504M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127550K .......... .......... .......... .......... .......... 93% 510M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127600K .......... .......... .......... .......... .......... 94% 421M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127650K .......... .......... .......... .......... .......... 94% 350M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127700K .......... .......... .......... .......... .......... 94% 504M 0s 2024-06-06T03:40:23.065 INFO:tasks.workunit.client.0.smithi040.stderr:127750K .......... .......... .......... .......... .......... 94% 462M 0s 2024-06-06T03:40:23.066 INFO:tasks.workunit.client.0.smithi040.stderr:127800K .......... .......... .......... .......... .......... 94% 445M 0s 2024-06-06T03:40:23.066 INFO:tasks.workunit.client.0.smithi040.stderr:127850K .......... .......... .......... .......... .......... 94% 427M 0s 2024-06-06T03:40:23.066 INFO:tasks.workunit.client.0.smithi040.stderr:127900K .......... .......... .......... .......... .......... 94% 512M 0s 2024-06-06T03:40:23.066 INFO:tasks.workunit.client.0.smithi040.stderr:127950K .......... .......... .......... .......... .......... 94% 507M 0s 2024-06-06T03:40:23.066 INFO:tasks.workunit.client.0.smithi040.stderr:128000K .......... .......... .......... .......... .......... 94% 461M 0s 2024-06-06T03:40:23.066 INFO:tasks.workunit.client.0.smithi040.stderr:128050K .......... .......... .......... .......... .......... 94% 437M 0s 2024-06-06T03:40:23.066 INFO:tasks.workunit.client.0.smithi040.stderr:128100K .......... .......... .......... .......... .......... 94% 453M 0s 2024-06-06T03:40:23.066 INFO:tasks.workunit.client.0.smithi040.stderr:128150K .......... .......... .......... .......... .......... 94% 482M 0s 2024-06-06T03:40:23.066 INFO:tasks.workunit.client.0.smithi040.stderr:128200K .......... .......... .......... .......... .......... 94% 492M 0s 2024-06-06T03:40:23.066 INFO:tasks.workunit.client.0.smithi040.stderr:128250K .......... .......... .......... .......... .......... 94% 414M 0s 2024-06-06T03:40:23.066 INFO:tasks.workunit.client.0.smithi040.stderr:128300K .......... .......... .......... .......... .......... 94% 489M 0s 2024-06-06T03:40:23.066 INFO:tasks.workunit.client.0.smithi040.stderr:128350K .......... .......... .......... .......... .......... 94% 441M 0s 2024-06-06T03:40:23.066 INFO:tasks.workunit.client.0.smithi040.stderr:128400K .......... .......... .......... .......... .......... 94% 451M 0s 2024-06-06T03:40:23.066 INFO:tasks.workunit.client.0.smithi040.stderr:128450K .......... .......... .......... .......... .......... 94% 424M 0s 2024-06-06T03:40:23.067 INFO:tasks.workunit.client.0.smithi040.stderr:128500K .......... .......... .......... .......... .......... 94% 365M 0s 2024-06-06T03:40:23.067 INFO:tasks.workunit.client.0.smithi040.stderr:128550K .......... .......... .......... .......... .......... 94% 6.81M 0s 2024-06-06T03:40:23.067 INFO:tasks.workunit.client.0.smithi040.stderr:128600K .......... .......... .......... .......... .......... 94% 2.48M 0s 2024-06-06T03:40:23.067 INFO:tasks.workunit.client.0.smithi040.stderr:128650K .......... .......... .......... .......... .......... 94% 71.7M 0s 2024-06-06T03:40:23.067 INFO:tasks.workunit.client.0.smithi040.stderr:128700K .......... .......... .......... .......... .......... 94% 113M 0s 2024-06-06T03:40:23.067 INFO:tasks.workunit.client.0.smithi040.stderr:128750K .......... .......... .......... .......... .......... 94% 113M 0s 2024-06-06T03:40:23.067 INFO:tasks.workunit.client.0.smithi040.stderr:128800K .......... .......... .......... .......... .......... 94% 116M 0s 2024-06-06T03:40:23.067 INFO:tasks.workunit.client.0.smithi040.stderr:128850K .......... .......... .......... .......... .......... 94% 131M 0s 2024-06-06T03:40:23.067 INFO:tasks.workunit.client.0.smithi040.stderr:128900K .......... .......... .......... .......... .......... 94% 92.4M 0s 2024-06-06T03:40:23.067 INFO:tasks.workunit.client.0.smithi040.stderr:128950K .......... .......... .......... .......... .......... 95% 137M 0s 2024-06-06T03:40:23.067 INFO:tasks.workunit.client.0.smithi040.stderr:129000K .......... .......... .......... .......... .......... 95% 95.3M 0s 2024-06-06T03:40:23.067 INFO:tasks.workunit.client.0.smithi040.stderr:129050K .......... .......... .......... .......... .......... 95% 26.6M 0s 2024-06-06T03:40:23.067 INFO:tasks.workunit.client.0.smithi040.stderr:129100K .......... .......... .......... .......... .......... 95% 24.3M 0s 2024-06-06T03:40:23.067 INFO:tasks.workunit.client.0.smithi040.stderr:129150K .......... .......... .......... .......... .......... 95% 42.0M 0s 2024-06-06T03:40:23.068 INFO:tasks.workunit.client.0.smithi040.stderr:129200K .......... .......... .......... .......... .......... 95% 25.1M 0s 2024-06-06T03:40:23.068 INFO:tasks.workunit.client.0.smithi040.stderr:129250K .......... .......... .......... .......... .......... 95% 33.0M 0s 2024-06-06T03:40:23.068 INFO:tasks.workunit.client.0.smithi040.stderr:129300K .......... .......... .......... .......... .......... 95% 26.0M 0s 2024-06-06T03:40:23.068 INFO:tasks.workunit.client.0.smithi040.stderr:129350K .......... .......... .......... .......... .......... 95% 26.1M 0s 2024-06-06T03:40:23.068 INFO:tasks.workunit.client.0.smithi040.stderr:129400K .......... .......... .......... .......... .......... 95% 44.4M 0s 2024-06-06T03:40:23.068 INFO:tasks.workunit.client.0.smithi040.stderr:129450K .......... .......... .......... .......... .......... 95% 22.7M 0s 2024-06-06T03:40:23.068 INFO:tasks.workunit.client.0.smithi040.stderr:129500K .......... .......... .......... .......... .......... 95% 40.1M 0s 2024-06-06T03:40:23.068 INFO:tasks.workunit.client.0.smithi040.stderr:129550K .......... .......... .......... .......... .......... 95% 10.5M 0s 2024-06-06T03:40:23.068 INFO:tasks.workunit.client.0.smithi040.stderr:129600K .......... .......... .......... .......... .......... 95% 24.2M 0s 2024-06-06T03:40:23.068 INFO:tasks.workunit.client.0.smithi040.stderr:129650K .......... .......... .......... .......... .......... 95% 46.4M 0s 2024-06-06T03:40:23.068 INFO:tasks.workunit.client.0.smithi040.stderr:129700K .......... .......... .......... .......... .......... 95% 24.4M 0s 2024-06-06T03:40:23.068 INFO:tasks.workunit.client.0.smithi040.stderr:129750K .......... .......... .......... .......... .......... 95% 24.3M 0s 2024-06-06T03:40:23.068 INFO:tasks.workunit.client.0.smithi040.stderr:129800K .......... .......... .......... .......... .......... 95% 36.8M 0s 2024-06-06T03:40:23.068 INFO:tasks.workunit.client.0.smithi040.stderr:129850K .......... .......... .......... .......... .......... 95% 24.7M 0s 2024-06-06T03:40:23.069 INFO:tasks.workunit.client.0.smithi040.stderr:129900K .......... .......... .......... .......... .......... 95% 40.7M 0s 2024-06-06T03:40:23.069 INFO:tasks.workunit.client.0.smithi040.stderr:129950K .......... .......... .......... .......... .......... 95% 20.3M 0s 2024-06-06T03:40:23.069 INFO:tasks.workunit.client.0.smithi040.stderr:130000K .......... .......... .......... .......... .......... 95% 25.3M 0s 2024-06-06T03:40:23.069 INFO:tasks.workunit.client.0.smithi040.stderr:130050K .......... .......... .......... .......... .......... 95% 37.2M 0s 2024-06-06T03:40:23.069 INFO:tasks.workunit.client.0.smithi040.stderr:130100K .......... .......... .......... .......... .......... 95% 24.4M 0s 2024-06-06T03:40:23.069 INFO:tasks.workunit.client.0.smithi040.stderr:130150K .......... .......... .......... .......... .......... 95% 40.8M 0s 2024-06-06T03:40:23.069 INFO:tasks.workunit.client.0.smithi040.stderr:130200K .......... .......... .......... .......... .......... 95% 25.0M 0s 2024-06-06T03:40:23.069 INFO:tasks.workunit.client.0.smithi040.stderr:130250K .......... .......... .......... .......... .......... 95% 26.4M 0s 2024-06-06T03:40:23.069 INFO:tasks.workunit.client.0.smithi040.stderr:130300K .......... .......... .......... .......... .......... 96% 35.1M 0s 2024-06-06T03:40:23.069 INFO:tasks.workunit.client.0.smithi040.stderr:130350K .......... .......... .......... .......... .......... 96% 23.9M 0s 2024-06-06T03:40:23.069 INFO:tasks.workunit.client.0.smithi040.stderr:130400K .......... .......... .......... .......... .......... 96% 43.5M 0s 2024-06-06T03:40:23.069 INFO:tasks.workunit.client.0.smithi040.stderr:130450K .......... .......... .......... .......... .......... 96% 25.1M 0s 2024-06-06T03:40:23.069 INFO:tasks.workunit.client.0.smithi040.stderr:130500K .......... .......... .......... .......... .......... 96% 25.5M 0s 2024-06-06T03:40:23.069 INFO:tasks.workunit.client.0.smithi040.stderr:130550K .......... .......... .......... .......... .......... 96% 16.8M 0s 2024-06-06T03:40:23.070 INFO:tasks.workunit.client.0.smithi040.stderr:130600K .......... .......... .......... .......... .......... 96% 24.3M 0s 2024-06-06T03:40:23.070 INFO:tasks.workunit.client.0.smithi040.stderr:130650K .......... .......... .......... .......... .......... 96% 35.1M 0s 2024-06-06T03:40:23.070 INFO:tasks.workunit.client.0.smithi040.stderr:130700K .......... .......... .......... .......... .......... 96% 6.46M 0s 2024-06-06T03:40:23.070 INFO:tasks.workunit.client.0.smithi040.stderr:130750K .......... .......... .......... .......... .......... 96% 86.4M 0s 2024-06-06T03:40:23.070 INFO:tasks.workunit.client.0.smithi040.stderr:130800K .......... .......... .......... .......... .......... 96% 54.4M 0s 2024-06-06T03:40:23.070 INFO:tasks.workunit.client.0.smithi040.stderr:130850K .......... .......... .......... .......... .......... 96% 88.9M 0s 2024-06-06T03:40:23.070 INFO:tasks.workunit.client.0.smithi040.stderr:130900K .......... .......... .......... .......... .......... 96% 7.89M 0s 2024-06-06T03:40:23.070 INFO:tasks.workunit.client.0.smithi040.stderr:130950K .......... .......... .......... .......... .......... 96% 20.2M 0s 2024-06-06T03:40:23.070 INFO:tasks.workunit.client.0.smithi040.stderr:131000K .......... .......... .......... .......... .......... 96% 26.7M 0s 2024-06-06T03:40:23.070 INFO:tasks.workunit.client.0.smithi040.stderr:131050K .......... .......... .......... .......... .......... 96% 22.4M 0s 2024-06-06T03:40:23.070 INFO:tasks.workunit.client.0.smithi040.stderr:131100K .......... .......... .......... .......... .......... 96% 26.1M 0s 2024-06-06T03:40:23.070 INFO:tasks.workunit.client.0.smithi040.stderr:131150K .......... .......... .......... .......... .......... 96% 110M 0s 2024-06-06T03:40:23.070 INFO:tasks.workunit.client.0.smithi040.stderr:131200K .......... .......... .......... .......... .......... 96% 24.5M 0s 2024-06-06T03:40:23.070 INFO:tasks.workunit.client.0.smithi040.stderr:131250K .......... .......... .......... .......... .......... 96% 25.0M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:131300K .......... .......... .......... .......... .......... 96% 30.9M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:131350K .......... .......... .......... .......... .......... 96% 45.1M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:131400K .......... .......... .......... .......... .......... 96% 33.5M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:131450K .......... .......... .......... .......... .......... 96% 29.1M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:131500K .......... .......... .......... .......... .......... 96% 20.9M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:131550K .......... .......... .......... .......... .......... 96% 26.3M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:131600K .......... .......... .......... .......... .......... 96% 12.5M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:131650K .......... .......... .......... .......... .......... 97% 35.8M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:131700K .......... .......... .......... .......... .......... 97% 22.9M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:131750K .......... .......... .......... .......... .......... 97% 34.4M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:131800K .......... .......... .......... .......... .......... 97% 21.1M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:131850K .......... .......... .......... .......... .......... 97% 15.2M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:131900K .......... .......... .......... .......... .......... 97% 20.5M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:131950K .......... .......... .......... .......... .......... 97% 82.2M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:132000K .......... .......... .......... .......... .......... 97% 72.5M 0s 2024-06-06T03:40:23.071 INFO:tasks.workunit.client.0.smithi040.stderr:132050K .......... .......... .......... .......... .......... 97% 41.2M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132100K .......... .......... .......... .......... .......... 97% 19.8M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132150K .......... .......... .......... .......... .......... 97% 55.2M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132200K .......... .......... .......... .......... .......... 97% 24.8M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132250K .......... .......... .......... .......... .......... 97% 30.7M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132300K .......... .......... .......... .......... .......... 97% 24.1M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132350K .......... .......... .......... .......... .......... 97% 26.5M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132400K .......... .......... .......... .......... .......... 97% 47.8M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132450K .......... .......... .......... .......... .......... 97% 24.3M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132500K .......... .......... .......... .......... .......... 97% 25.7M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132550K .......... .......... .......... .......... .......... 97% 36.5M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132600K .......... .......... .......... .......... .......... 97% 14.0M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132650K .......... .......... .......... .......... .......... 97% 36.7M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132700K .......... .......... .......... .......... .......... 97% 26.7M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132750K .......... .......... .......... .......... .......... 97% 22.8M 0s 2024-06-06T03:40:23.072 INFO:tasks.workunit.client.0.smithi040.stderr:132800K .......... .......... .......... .......... .......... 97% 40.2M 0s 2024-06-06T03:40:23.073 INFO:tasks.workunit.client.0.smithi040.stderr:132850K .......... .......... .......... .......... .......... 97% 23.4M 0s 2024-06-06T03:40:23.073 INFO:tasks.workunit.client.0.smithi040.stderr:132900K .......... .......... .......... .......... .......... 97% 43.1M 0s 2024-06-06T03:40:23.073 INFO:tasks.workunit.client.0.smithi040.stderr:132950K .......... .......... .......... .......... .......... 97% 13.2M 0s 2024-06-06T03:40:23.073 INFO:tasks.workunit.client.0.smithi040.stderr:133000K .......... .......... .......... .......... .......... 98% 6.21M 0s 2024-06-06T03:40:23.073 INFO:tasks.workunit.client.0.smithi040.stderr:133050K .......... .......... .......... .......... .......... 98% 41.6M 0s 2024-06-06T03:40:23.112 INFO:tasks.workunit.client.0.smithi040.stderr:133100K .......... .......... .......... .......... .......... 98% 24.5M 0s 2024-06-06T03:40:23.112 INFO:tasks.workunit.client.0.smithi040.stderr:133150K .......... .......... .......... .......... .......... 98% 46.4M 0s 2024-06-06T03:40:23.112 INFO:tasks.workunit.client.0.smithi040.stderr:133200K .......... .......... .......... .......... .......... 98% 25.7M 0s 2024-06-06T03:40:23.112 INFO:tasks.workunit.client.0.smithi040.stderr:133250K .......... .......... .......... .......... .......... 98% 29.9M 0s 2024-06-06T03:40:23.112 INFO:tasks.workunit.client.0.smithi040.stderr:133300K .......... .......... .......... .......... .......... 98% 21.8M 0s 2024-06-06T03:40:23.112 INFO:tasks.workunit.client.0.smithi040.stderr:133350K .......... .......... .......... .......... .......... 98% 34.7M 0s 2024-06-06T03:40:23.112 INFO:tasks.workunit.client.0.smithi040.stderr:133400K .......... .......... .......... .......... .......... 98% 41.1M 0s 2024-06-06T03:40:23.112 INFO:tasks.workunit.client.0.smithi040.stderr:133450K .......... .......... .......... .......... .......... 98% 24.2M 0s 2024-06-06T03:40:23.112 INFO:tasks.workunit.client.0.smithi040.stderr:133500K .......... .......... .......... .......... .......... 98% 33.7M 0s 2024-06-06T03:40:23.112 INFO:tasks.workunit.client.0.smithi040.stderr:133550K .......... .......... .......... .......... .......... 98% 20.7M 0s 2024-06-06T03:40:23.112 INFO:tasks.workunit.client.0.smithi040.stderr:133600K .......... .......... .......... .......... .......... 98% 17.6M 0s 2024-06-06T03:40:23.113 INFO:tasks.workunit.client.0.smithi040.stderr:133650K .......... .......... .......... .......... .......... 98% 44.8M 0s 2024-06-06T03:40:23.113 INFO:tasks.workunit.client.0.smithi040.stderr:133700K .......... .......... .......... .......... .......... 98% 17.3M 0s 2024-06-06T03:40:23.113 INFO:tasks.workunit.client.0.smithi040.stderr:133750K .......... .......... .......... .......... .......... 98% 17.6M 0s 2024-06-06T03:40:23.113 INFO:tasks.workunit.client.0.smithi040.stderr:133800K .......... .......... .......... .......... .......... 98% 36.2M 0s 2024-06-06T03:40:23.113 INFO:tasks.workunit.client.0.smithi040.stderr:133850K .......... .......... .......... .......... .......... 98% 30.1M 0s 2024-06-06T03:40:23.113 INFO:tasks.workunit.client.0.smithi040.stderr:133900K .......... .......... .......... .......... .......... 98% 30.5M 0s 2024-06-06T03:40:23.113 INFO:tasks.workunit.client.0.smithi040.stderr:133950K .......... .......... .......... .......... .......... 98% 35.4M 0s 2024-06-06T03:40:23.113 INFO:tasks.workunit.client.0.smithi040.stderr:134000K .......... .......... .......... .......... .......... 98% 20.8M 0s 2024-06-06T03:40:23.113 INFO:tasks.workunit.client.0.smithi040.stderr:134050K .......... .......... .......... .......... .......... 98% 26.3M 0s 2024-06-06T03:40:23.113 INFO:tasks.workunit.client.0.smithi040.stderr:134100K .......... .......... .......... .......... .......... 98% 23.7M 0s 2024-06-06T03:40:23.113 INFO:tasks.workunit.client.0.smithi040.stderr:134150K .......... .......... .......... .......... .......... 98% 42.1M 0s 2024-06-06T03:40:23.113 INFO:tasks.workunit.client.0.smithi040.stderr:134200K .......... .......... .......... .......... .......... 98% 23.7M 0s 2024-06-06T03:40:23.113 INFO:tasks.workunit.client.0.smithi040.stderr:134250K .......... .......... .......... .......... .......... 98% 29.8M 0s 2024-06-06T03:40:23.113 INFO:tasks.workunit.client.0.smithi040.stderr:134300K .......... .......... .......... .......... .......... 98% 33.4M 0s 2024-06-06T03:40:23.114 INFO:tasks.workunit.client.0.smithi040.stderr:134350K .......... .......... .......... .......... .......... 99% 25.6M 0s 2024-06-06T03:40:23.114 INFO:tasks.workunit.client.0.smithi040.stderr:134400K .......... .......... .......... .......... .......... 99% 33.6M 0s 2024-06-06T03:40:23.114 INFO:tasks.workunit.client.0.smithi040.stderr:134450K .......... .......... .......... .......... .......... 99% 26.2M 0s 2024-06-06T03:40:23.119 INFO:tasks.workunit.client.0.smithi040.stderr:134500K .......... .......... .......... .......... .......... 99% 28.1M 0s 2024-06-06T03:40:23.120 INFO:tasks.workunit.client.0.smithi040.stderr:134550K .......... .......... .......... .......... .......... 99% 30.3M 0s 2024-06-06T03:40:23.120 INFO:tasks.workunit.client.0.smithi040.stderr:134600K .......... .......... .......... .......... .......... 99% 16.5M 0s 2024-06-06T03:40:23.120 INFO:tasks.workunit.client.0.smithi040.stderr:134650K .......... .......... .......... .......... .......... 99% 30.8M 0s 2024-06-06T03:40:23.122 INFO:tasks.workunit.client.0.smithi040.stderr:134700K .......... .......... .......... .......... .......... 99% 22.4M 0s 2024-06-06T03:40:23.124 INFO:tasks.workunit.client.0.smithi040.stderr:134750K .......... .......... .......... .......... .......... 99% 27.8M 0s 2024-06-06T03:40:23.128 INFO:tasks.workunit.client.0.smithi040.stderr:134800K .......... .......... .......... .......... .......... 99% 26.6M 0s 2024-06-06T03:40:23.128 INFO:tasks.workunit.client.0.smithi040.stderr:134850K .......... .......... .......... .......... .......... 99% 26.8M 0s 2024-06-06T03:40:23.128 INFO:tasks.workunit.client.0.smithi040.stderr:134900K .......... .......... .......... .......... .......... 99% 41.0M 0s 2024-06-06T03:40:23.130 INFO:tasks.workunit.client.0.smithi040.stderr:134950K .......... .......... .......... .......... .......... 99% 23.2M 0s 2024-06-06T03:40:23.136 INFO:tasks.workunit.client.0.smithi040.stderr:135000K .......... .......... .......... .......... .......... 99% 9.26M 0s 2024-06-06T03:40:23.136 INFO:tasks.workunit.client.0.smithi040.stderr:135050K .......... .......... .......... .......... .......... 99% 140M 0s 2024-06-06T03:40:23.139 INFO:tasks.workunit.client.0.smithi040.stderr:135100K .......... .......... .......... .......... .......... 99% 15.7M 0s 2024-06-06T03:40:23.143 INFO:tasks.workunit.client.0.smithi040.stderr:135150K .......... .......... .......... .......... .......... 99% 12.6M 0s 2024-06-06T03:40:23.191 INFO:tasks.workunit.client.0.smithi040.stderr:135200K .......... .......... .......... .......... .......... 99% 1.20M 0s 2024-06-06T03:40:23.191 INFO:tasks.workunit.client.0.smithi040.stderr:135250K .......... .......... .......... .......... .......... 99% 472M 0s 2024-06-06T03:40:23.191 INFO:tasks.workunit.client.0.smithi040.stderr:135300K .......... .......... .......... .......... .......... 99% 525M 0s 2024-06-06T03:40:23.191 INFO:tasks.workunit.client.0.smithi040.stderr:135350K .......... .......... .......... .......... .......... 99% 524M 0s 2024-06-06T03:40:23.191 INFO:tasks.workunit.client.0.smithi040.stderr:135400K .......... .......... .......... .......... .......... 99% 373M 0s 2024-06-06T03:40:23.191 INFO:tasks.workunit.client.0.smithi040.stderr:135450K .......... .......... .......... .......... .......... 99% 507M 0s 2024-06-06T03:40:23.191 INFO:tasks.workunit.client.0.smithi040.stderr:135500K .......... .......... .......... .......... .......... 99% 486M 0s 2024-06-06T03:40:23.192 INFO:tasks.workunit.client.0.smithi040.stderr:135550K .......... .......... .......... .......... .......... 99% 529M 0s 2024-06-06T03:40:23.192 INFO:tasks.workunit.client.0.smithi040.stderr:135600K .......... .......... .......... .......... .......... 99% 386M 0s 2024-06-06T03:40:23.192 INFO:tasks.workunit.client.0.smithi040.stderr:135650K .......... .......... .......... .......... .......... 99% 502M 0s 2024-06-06T03:40:23.192 INFO:tasks.workunit.client.0.smithi040.stderr:135700K .......... .......... .......... .......... .......... 99% 429M 0s 2024-06-06T03:40:23.192 INFO:tasks.workunit.client.0.smithi040.stderr:135750K .. 100% 135M=5.7s 2024-06-06T03:40:23.192 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-06-06T03:40:23.192 INFO:tasks.workunit.client.0.smithi040.stderr:2024-06-06 03:40:23 (23.4 MB/s) - ‘linux.tar.xz’ saved [139010916/139010916] 2024-06-06T03:40:23.192 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-06-06T03:40:23.192 INFO:tasks.workunit.client.0.smithi040.stderr:+ mkdir t 2024-06-06T03:40:23.192 INFO:tasks.workunit.client.0.smithi040.stderr:+ cd t 2024-06-06T03:40:23.192 INFO:tasks.workunit.client.0.smithi040.stderr:+ tar xJf ../linux.tar.xz 2024-06-06T03:40:23.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:23 smithi131 ceph-mon[36687]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-06-06T03:40:23.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:23 smithi040 ceph-mon[32220]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-06-06T03:40:23.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:23 smithi104 ceph-mon[36569]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-06-06T03:40:25.400 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:25 smithi040 ceph-mon[32220]: pgmap v56: 97 pgs: 97 active+clean; 123 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 16 op/s 2024-06-06T03:40:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:25 smithi131 ceph-mon[36687]: pgmap v56: 97 pgs: 97 active+clean; 123 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 16 op/s 2024-06-06T03:40:25.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:25 smithi104 ceph-mon[36569]: pgmap v56: 97 pgs: 97 active+clean; 123 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 16 op/s 2024-06-06T03:40:27.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:27 smithi131 ceph-mon[36687]: pgmap v57: 97 pgs: 97 active+clean; 123 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-06-06T03:40:27.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:27 smithi040 ceph-mon[32220]: pgmap v57: 97 pgs: 97 active+clean; 123 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-06-06T03:40:27.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:27 smithi104 ceph-mon[36569]: pgmap v57: 97 pgs: 97 active+clean; 123 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-06-06T03:40:29.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:29 smithi131 ceph-mon[36687]: pgmap v58: 97 pgs: 97 active+clean; 123 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-06-06T03:40:29.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:29 smithi040 ceph-mon[32220]: pgmap v58: 97 pgs: 97 active+clean; 123 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-06-06T03:40:29.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:29 smithi104 ceph-mon[36569]: pgmap v58: 97 pgs: 97 active+clean; 123 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-06-06T03:40:31.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:31 smithi131 ceph-mon[36687]: pgmap v59: 97 pgs: 97 active+clean; 129 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 11 MiB/s wr, 22 op/s 2024-06-06T03:40:31.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:31 smithi040 ceph-mon[32220]: pgmap v59: 97 pgs: 97 active+clean; 129 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 11 MiB/s wr, 22 op/s 2024-06-06T03:40:31.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:31 smithi104 ceph-mon[36569]: pgmap v59: 97 pgs: 97 active+clean; 129 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 11 MiB/s wr, 22 op/s 2024-06-06T03:40:33.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:33 smithi131 ceph-mon[36687]: pgmap v60: 97 pgs: 97 active+clean; 129 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 11 MiB/s wr, 17 op/s 2024-06-06T03:40:33.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:33 smithi040 ceph-mon[32220]: pgmap v60: 97 pgs: 97 active+clean; 129 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 11 MiB/s wr, 17 op/s 2024-06-06T03:40:33.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:33 smithi104 ceph-mon[36569]: pgmap v60: 97 pgs: 97 active+clean; 129 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 11 MiB/s wr, 17 op/s 2024-06-06T03:40:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:35 smithi131 ceph-mon[36687]: pgmap v61: 97 pgs: 97 active+clean; 134 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 11 MiB/s wr, 34 op/s 2024-06-06T03:40:35.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:35 smithi040 ceph-mon[32220]: pgmap v61: 97 pgs: 97 active+clean; 134 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 11 MiB/s wr, 34 op/s 2024-06-06T03:40:35.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:35 smithi104 ceph-mon[36569]: pgmap v61: 97 pgs: 97 active+clean; 134 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 11 MiB/s wr, 34 op/s 2024-06-06T03:40:37.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:37 smithi131 ceph-mon[36687]: pgmap v62: 97 pgs: 97 active+clean; 134 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 1.0 MiB/s wr, 26 op/s 2024-06-06T03:40:37.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:37 smithi040 ceph-mon[32220]: pgmap v62: 97 pgs: 97 active+clean; 134 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 1.0 MiB/s wr, 26 op/s 2024-06-06T03:40:37.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:37 smithi104 ceph-mon[36569]: pgmap v62: 97 pgs: 97 active+clean; 134 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 1.0 MiB/s wr, 26 op/s 2024-06-06T03:40:39.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:39 smithi131 ceph-mon[36687]: pgmap v63: 97 pgs: 97 active+clean; 134 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 1.0 MiB/s wr, 26 op/s 2024-06-06T03:40:39.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:39 smithi040 ceph-mon[32220]: pgmap v63: 97 pgs: 97 active+clean; 134 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 1.0 MiB/s wr, 26 op/s 2024-06-06T03:40:39.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:39 smithi104 ceph-mon[36569]: pgmap v63: 97 pgs: 97 active+clean; 134 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 1.0 MiB/s wr, 26 op/s 2024-06-06T03:40:41.366 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:41 smithi040 ceph-mon[32220]: pgmap v64: 97 pgs: 97 active+clean; 139 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 43 op/s 2024-06-06T03:40:41.366 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:41 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:40:41.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:41 smithi131 ceph-mon[36687]: pgmap v64: 97 pgs: 97 active+clean; 139 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 43 op/s 2024-06-06T03:40:41.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:41 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:40:41.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:41 smithi104 ceph-mon[36569]: pgmap v64: 97 pgs: 97 active+clean; 139 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 43 op/s 2024-06-06T03:40:41.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:41 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:40:43.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:43 smithi131 ceph-mon[36687]: pgmap v65: 97 pgs: 97 active+clean; 139 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 983 KiB/s wr, 34 op/s 2024-06-06T03:40:43.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:43 smithi040 ceph-mon[32220]: pgmap v65: 97 pgs: 97 active+clean; 139 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 983 KiB/s wr, 34 op/s 2024-06-06T03:40:43.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:43 smithi104 ceph-mon[36569]: pgmap v65: 97 pgs: 97 active+clean; 139 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 983 KiB/s wr, 34 op/s 2024-06-06T03:40:44.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:44 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:40:44.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:44 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:40:44.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:44 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:40:45.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:45 smithi131 ceph-mon[36687]: pgmap v66: 97 pgs: 97 active+clean; 143 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 49 op/s 2024-06-06T03:40:45.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:45 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:40:45.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:45 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:40:45.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:45 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:40:45.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:45 smithi040 ceph-mon[32220]: pgmap v66: 97 pgs: 97 active+clean; 143 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 49 op/s 2024-06-06T03:40:45.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:45 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:40:45.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:45 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:40:45.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:45 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:40:45.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:45 smithi104 ceph-mon[36569]: pgmap v66: 97 pgs: 97 active+clean; 143 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 49 op/s 2024-06-06T03:40:45.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:45 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:40:45.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:45 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:40:45.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:45 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:40:46.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:46 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:40:46.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:46 smithi040 ceph-mon[32220]: pgmap v67: 97 pgs: 97 active+clean; 143 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 957 KiB/s wr, 32 op/s 2024-06-06T03:40:46.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:46 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:40:46.840 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:46 smithi104 ceph-mon[36569]: pgmap v67: 97 pgs: 97 active+clean; 143 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 957 KiB/s wr, 32 op/s 2024-06-06T03:40:47.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:46 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:40:47.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:46 smithi131 ceph-mon[36687]: pgmap v67: 97 pgs: 97 active+clean; 143 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 957 KiB/s wr, 32 op/s 2024-06-06T03:40:49.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:48 smithi131 ceph-mon[36687]: pgmap v68: 97 pgs: 97 active+clean; 143 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 957 KiB/s wr, 32 op/s 2024-06-06T03:40:49.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:48 smithi040 ceph-mon[32220]: pgmap v68: 97 pgs: 97 active+clean; 143 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 957 KiB/s wr, 32 op/s 2024-06-06T03:40:49.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:48 smithi104 ceph-mon[36569]: pgmap v68: 97 pgs: 97 active+clean; 143 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 957 KiB/s wr, 32 op/s 2024-06-06T03:40:51.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:50 smithi131 ceph-mon[36687]: pgmap v69: 97 pgs: 97 active+clean; 146 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 51 op/s 2024-06-06T03:40:51.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:50 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:40:51.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:50 smithi040 ceph-mon[32220]: pgmap v69: 97 pgs: 97 active+clean; 146 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 51 op/s 2024-06-06T03:40:51.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:50 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:40:51.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:50 smithi104 ceph-mon[36569]: pgmap v69: 97 pgs: 97 active+clean; 146 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 51 op/s 2024-06-06T03:40:51.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:50 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:40:53.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:52 smithi131 ceph-mon[36687]: pgmap v70: 97 pgs: 97 active+clean; 146 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 749 KiB/s wr, 34 op/s 2024-06-06T03:40:53.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:52 smithi040 ceph-mon[32220]: pgmap v70: 97 pgs: 97 active+clean; 146 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 749 KiB/s wr, 34 op/s 2024-06-06T03:40:53.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:52 smithi104 ceph-mon[36569]: pgmap v70: 97 pgs: 97 active+clean; 146 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 749 KiB/s wr, 34 op/s 2024-06-06T03:40:55.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:54 smithi131 ceph-mon[36687]: pgmap v71: 97 pgs: 97 active+clean; 147 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 797 KiB/s wr, 45 op/s 2024-06-06T03:40:55.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:54 smithi040 ceph-mon[32220]: pgmap v71: 97 pgs: 97 active+clean; 147 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 797 KiB/s wr, 45 op/s 2024-06-06T03:40:55.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:54 smithi104 ceph-mon[36569]: pgmap v71: 97 pgs: 97 active+clean; 147 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 797 KiB/s wr, 45 op/s 2024-06-06T03:40:56.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:55 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:40:56.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:55 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:40:56.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:55 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:40:57.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:56 smithi131 ceph-mon[36687]: pgmap v72: 97 pgs: 97 active+clean; 173 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-06-06T03:40:57.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:56 smithi040 ceph-mon[32220]: pgmap v72: 97 pgs: 97 active+clean; 173 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-06-06T03:40:57.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:56 smithi104 ceph-mon[36569]: pgmap v72: 97 pgs: 97 active+clean; 173 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-06-06T03:40:59.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:40:58 smithi131 ceph-mon[36687]: pgmap v73: 97 pgs: 97 active+clean; 173 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-06-06T03:40:59.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:40:58 smithi040 ceph-mon[32220]: pgmap v73: 97 pgs: 97 active+clean; 173 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-06-06T03:40:59.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:40:58 smithi104 ceph-mon[36569]: pgmap v73: 97 pgs: 97 active+clean; 173 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-06-06T03:41:01.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:00 smithi131 ceph-mon[36687]: pgmap v74: 97 pgs: 97 active+clean; 209 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 715 op/s 2024-06-06T03:41:01.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:00 smithi040 ceph-mon[32220]: pgmap v74: 97 pgs: 97 active+clean; 209 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 715 op/s 2024-06-06T03:41:01.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:00 smithi104 ceph-mon[36569]: pgmap v74: 97 pgs: 97 active+clean; 209 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 715 op/s 2024-06-06T03:41:03.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:02 smithi131 ceph-mon[36687]: pgmap v75: 97 pgs: 97 active+clean; 209 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 696 op/s 2024-06-06T03:41:03.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:02 smithi040 ceph-mon[32220]: pgmap v75: 97 pgs: 97 active+clean; 209 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 696 op/s 2024-06-06T03:41:03.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:02 smithi104 ceph-mon[36569]: pgmap v75: 97 pgs: 97 active+clean; 209 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 696 op/s 2024-06-06T03:41:05.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:04 smithi131 ceph-mon[36687]: pgmap v76: 97 pgs: 97 active+clean; 209 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 696 op/s 2024-06-06T03:41:05.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:04 smithi040 ceph-mon[32220]: pgmap v76: 97 pgs: 97 active+clean; 209 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 696 op/s 2024-06-06T03:41:05.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:04 smithi104 ceph-mon[36569]: pgmap v76: 97 pgs: 97 active+clean; 209 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 696 op/s 2024-06-06T03:41:07.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:06 smithi131 ceph-mon[36687]: pgmap v77: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.4 MiB/s wr, 993 op/s 2024-06-06T03:41:07.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:06 smithi040 ceph-mon[32220]: pgmap v77: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.4 MiB/s wr, 993 op/s 2024-06-06T03:41:07.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:06 smithi104 ceph-mon[36569]: pgmap v77: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.4 MiB/s wr, 993 op/s 2024-06-06T03:41:09.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:08 smithi131 ceph-mon[36687]: pgmap v78: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 791 op/s 2024-06-06T03:41:09.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:08 smithi040 ceph-mon[32220]: pgmap v78: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 791 op/s 2024-06-06T03:41:09.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:08 smithi104 ceph-mon[36569]: pgmap v78: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 791 op/s 2024-06-06T03:41:11.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:10 smithi131 ceph-mon[36687]: pgmap v79: 97 pgs: 97 active+clean; 257 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.5 MiB/s wr, 812 op/s 2024-06-06T03:41:11.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:10 smithi040 ceph-mon[32220]: pgmap v79: 97 pgs: 97 active+clean; 257 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.5 MiB/s wr, 812 op/s 2024-06-06T03:41:11.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:10 smithi104 ceph-mon[36569]: pgmap v79: 97 pgs: 97 active+clean; 257 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.5 MiB/s wr, 812 op/s 2024-06-06T03:41:13.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:12 smithi131 ceph-mon[36687]: pgmap v80: 97 pgs: 97 active+clean; 257 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 327 op/s 2024-06-06T03:41:13.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:12 smithi040 ceph-mon[32220]: pgmap v80: 97 pgs: 97 active+clean; 257 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 327 op/s 2024-06-06T03:41:13.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:12 smithi104 ceph-mon[36569]: pgmap v80: 97 pgs: 97 active+clean; 257 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 327 op/s 2024-06-06T03:41:15.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:14 smithi131 ceph-mon[36687]: pgmap v81: 97 pgs: 97 active+clean; 257 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 327 op/s 2024-06-06T03:41:15.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:14 smithi040 ceph-mon[32220]: pgmap v81: 97 pgs: 97 active+clean; 257 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 327 op/s 2024-06-06T03:41:15.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:14 smithi104 ceph-mon[36569]: pgmap v81: 97 pgs: 97 active+clean; 257 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 327 op/s 2024-06-06T03:41:15.239 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.2 2024-06-06T03:41:15.240 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:41:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:41:16.947 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:16 smithi040 ceph-mon[32220]: pgmap v82: 97 pgs: 97 active+clean; 260 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 347 op/s 2024-06-06T03:41:16.947 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:16 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:41:17.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:16 smithi104 ceph-mon[36569]: pgmap v82: 97 pgs: 97 active+clean; 260 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 347 op/s 2024-06-06T03:41:17.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:16 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:41:17.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.207+0000 7fa21f980700 1 -- 172.21.15.40:0/1448450192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21806ab30 msgr2=0x7fa21810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:17.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.207+0000 7fa21f980700 1 --2- 172.21.15.40:0/1448450192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21806ab30 0x7fa21810f320 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa21000b0a0 tx=0x7fa21000b3b0 comp rx=0 tx=0).stop 2024-06-06T03:41:17.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.207+0000 7fa21f980700 1 -- 172.21.15.40:0/1448450192 shutdown_connections 2024-06-06T03:41:17.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.207+0000 7fa21f980700 1 --2- 172.21.15.40:0/1448450192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21806ab30 0x7fa21810f320 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:17.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.207+0000 7fa21f980700 1 --2- 172.21.15.40:0/1448450192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa218108500 0x7fa2181088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:17.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.207+0000 7fa21f980700 1 --2- 172.21.15.40:0/1448450192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa21806a170 0x7fa21806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:17.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.207+0000 7fa21f980700 1 -- 172.21.15.40:0/1448450192 >> 172.21.15.40:0/1448450192 conn(0x7fa218078340 msgr2=0x7fa218078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:17.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.207+0000 7fa21f980700 1 -- 172.21.15.40:0/1448450192 shutdown_connections 2024-06-06T03:41:17.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.207+0000 7fa21f980700 1 -- 172.21.15.40:0/1448450192 wait complete. 2024-06-06T03:41:17.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.208+0000 7fa21f980700 1 Processor -- start 2024-06-06T03:41:17.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.208+0000 7fa21f980700 1 -- start start 2024-06-06T03:41:17.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.208+0000 7fa21f980700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21806a170 0x7fa218101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:17.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.208+0000 7fa21f980700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa21806ab30 0x7fa2181019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:17.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.208+0000 7fa21f980700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa218108500 0x7fa218101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:17.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.208+0000 7fa21f980700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa218112760 con 0x7fa21806a170 2024-06-06T03:41:17.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.208+0000 7fa21f980700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa2181125e0 con 0x7fa21806ab30 2024-06-06T03:41:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.208+0000 7fa21f980700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa2181128e0 con 0x7fa218108500 2024-06-06T03:41:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.208+0000 7fa21d71c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21806a170 0x7fa218101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.208+0000 7fa21d71c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21806a170 0x7fa218101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49880/0 (socket says 172.21.15.40:49880) 2024-06-06T03:41:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.208+0000 7fa21d71c700 1 -- 172.21.15.40:0/1022171738 learned_addr learned my addr 172.21.15.40:0/1022171738 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:41:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.208+0000 7fa21cf1b700 1 --2- 172.21.15.40:0/1022171738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa21806ab30 0x7fa2181019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.208+0000 7fa21df1d700 1 --2- 172.21.15.40:0/1022171738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa218108500 0x7fa218101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.209+0000 7fa21d71c700 1 -- 172.21.15.40:0/1022171738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa218108500 msgr2=0x7fa218101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.209+0000 7fa21d71c700 1 --2- 172.21.15.40:0/1022171738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa218108500 0x7fa218101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.209+0000 7fa21d71c700 1 -- 172.21.15.40:0/1022171738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa21806ab30 msgr2=0x7fa2181019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.209+0000 7fa21d71c700 1 --2- 172.21.15.40:0/1022171738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa21806ab30 0x7fa2181019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.209+0000 7fa21d71c700 1 -- 172.21.15.40:0/1022171738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2181a70c0 con 0x7fa21806a170 2024-06-06T03:41:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.209+0000 7fa21d71c700 1 --2- 172.21.15.40:0/1022171738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21806a170 0x7fa218101460 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa21400eb30 tx=0x7fa21400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.209+0000 7fa20e7fc700 1 -- 172.21.15.40:0/1022171738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21400cbd0 con 0x7fa21806a170 2024-06-06T03:41:17.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.209+0000 7fa21f980700 1 -- 172.21.15.40:0/1022171738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2181a73b0 con 0x7fa21806a170 2024-06-06T03:41:17.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.209+0000 7fa21f980700 1 -- 172.21.15.40:0/1022171738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2181a7940 con 0x7fa21806a170 2024-06-06T03:41:17.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.209+0000 7fa20e7fc700 1 -- 172.21.15.40:0/1022171738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa214004500 con 0x7fa21806a170 2024-06-06T03:41:17.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.209+0000 7fa20e7fc700 1 -- 172.21.15.40:0/1022171738 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214010430 con 0x7fa21806a170 2024-06-06T03:41:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.211+0000 7fa21f980700 1 -- 172.21.15.40:0/1022171738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa218069180 con 0x7fa21806a170 2024-06-06T03:41:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.211+0000 7fa20e7fc700 1 -- 172.21.15.40:0/1022171738 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa214003f00 con 0x7fa21806a170 2024-06-06T03:41:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.212+0000 7fa20e7fc700 1 --2- 172.21.15.40:0/1022171738 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa2040a2e10 0x7fa2040a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.213+0000 7fa21cf1b700 1 --2- 172.21.15.40:0/1022171738 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa2040a2e10 0x7fa2040a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.213+0000 7fa20e7fc700 1 -- 172.21.15.40:0/1022171738 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fa2140cf640 con 0x7fa21806a170 2024-06-06T03:41:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.213+0000 7fa21cf1b700 1 --2- 172.21.15.40:0/1022171738 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa2040a2e10 0x7fa2040a52d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa2180788b0 tx=0x7fa208009260 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:17.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.217+0000 7fa20e7fc700 1 -- 172.21.15.40:0/1022171738 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa21409c2f0 con 0x7fa21806a170 2024-06-06T03:41:17.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:16 smithi131 ceph-mon[36687]: pgmap v82: 97 pgs: 97 active+clean; 260 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 347 op/s 2024-06-06T03:41:17.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:16 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:41:17.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.467+0000 7fa21f980700 1 -- 172.21.15.40:0/1022171738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa218102990 con 0x7fa21806a170 2024-06-06T03:41:17.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.468+0000 7fa20e7fc700 1 -- 172.21.15.40:0/1022171738 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11941 (secure 0 0 0) 0x7fa21409bbf0 con 0x7fa21806a170 2024-06-06T03:41:17.468 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:41:17.468 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:41:15.465484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24403,24398,24425,24470,24443]},"id":1}]} 2024-06-06T03:41:17.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.470+0000 7fa21f980700 1 -- 172.21.15.40:0/1022171738 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa2040a2e10 msgr2=0x7fa2040a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:17.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.470+0000 7fa21f980700 1 --2- 172.21.15.40:0/1022171738 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa2040a2e10 0x7fa2040a52d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa2180788b0 tx=0x7fa208009260 comp rx=0 tx=0).stop 2024-06-06T03:41:17.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.470+0000 7fa21f980700 1 -- 172.21.15.40:0/1022171738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21806a170 msgr2=0x7fa218101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:17.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.470+0000 7fa21f980700 1 --2- 172.21.15.40:0/1022171738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21806a170 0x7fa218101460 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa21400eb30 tx=0x7fa21400ee40 comp rx=0 tx=0).stop 2024-06-06T03:41:17.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.470+0000 7fa21f980700 1 -- 172.21.15.40:0/1022171738 shutdown_connections 2024-06-06T03:41:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.470+0000 7fa21f980700 1 --2- 172.21.15.40:0/1022171738 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fa2040a2e10 0x7fa2040a52d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.470+0000 7fa21f980700 1 --2- 172.21.15.40:0/1022171738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa218108500 0x7fa218101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.470+0000 7fa21f980700 1 --2- 172.21.15.40:0/1022171738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21806a170 0x7fa218101460 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.470+0000 7fa21f980700 1 --2- 172.21.15.40:0/1022171738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa21806ab30 0x7fa2181019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.470+0000 7fa21f980700 1 -- 172.21.15.40:0/1022171738 >> 172.21.15.40:0/1022171738 conn(0x7fa218078340 msgr2=0x7fa2180ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.470+0000 7fa21f980700 1 -- 172.21.15.40:0/1022171738 shutdown_connections 2024-06-06T03:41:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:17.470+0000 7fa21f980700 1 -- 172.21.15.40:0/1022171738 wait complete. 2024-06-06T03:41:17.471 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 34 2024-06-06T03:41:18.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:17 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1022171738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:41:18.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:17 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1022171738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:41:18.189 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0' 2024-06-06T03:41:18.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0 2024-06-06T03:41:18.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:17 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1022171738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:41:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:41:18.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:18 smithi040 ceph-mon[32220]: pgmap v83: 97 pgs: 97 active+clean; 260 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 762 KiB/s wr, 40 op/s 2024-06-06T03:41:19.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:18 smithi104 ceph-mon[36569]: pgmap v83: 97 pgs: 97 active+clean; 260 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 762 KiB/s wr, 40 op/s 2024-06-06T03:41:19.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:18 smithi131 ceph-mon[36687]: pgmap v83: 97 pgs: 97 active+clean; 260 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 762 KiB/s wr, 40 op/s 2024-06-06T03:41:20.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.184+0000 7f4e37cd0700 1 -- 172.21.15.40:0/925996120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e30102ea0 msgr2=0x7f4e3010f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:20.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.184+0000 7f4e37cd0700 1 --2- 172.21.15.40:0/925996120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e30102ea0 0x7f4e3010f380 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800b0d0 tx=0x7f4e2800b3e0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.184+0000 7f4e37cd0700 1 -- 172.21.15.40:0/925996120 shutdown_connections 2024-06-06T03:41:20.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.184+0000 7f4e37cd0700 1 --2- 172.21.15.40:0/925996120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e30102ea0 0x7f4e3010f380 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.184+0000 7f4e37cd0700 1 --2- 172.21.15.40:0/925996120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e30107480 0x7f4e30107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.184+0000 7f4e37cd0700 1 --2- 172.21.15.40:0/925996120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e301024e0 0x7f4e30102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.185+0000 7f4e37cd0700 1 -- 172.21.15.40:0/925996120 >> 172.21.15.40:0/925996120 conn(0x7f4e30079640 msgr2=0x7f4e30079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:20.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.185+0000 7f4e37cd0700 1 -- 172.21.15.40:0/925996120 shutdown_connections 2024-06-06T03:41:20.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.185+0000 7f4e37cd0700 1 -- 172.21.15.40:0/925996120 wait complete. 2024-06-06T03:41:20.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.185+0000 7f4e37cd0700 1 Processor -- start 2024-06-06T03:41:20.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.185+0000 7f4e37cd0700 1 -- start start 2024-06-06T03:41:20.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e37cd0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e301024e0 0x7f4e3019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e37cd0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e30102ea0 0x7f4e3019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e37cd0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e30107480 0x7f4e301989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e37cd0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4e301127e0 con 0x7f4e30107480 2024-06-06T03:41:20.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e37cd0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30112660 con 0x7f4e30102ea0 2024-06-06T03:41:20.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e37cd0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30112960 con 0x7f4e301024e0 2024-06-06T03:41:20.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e3626d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e30107480 0x7f4e301989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e3526b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e30102ea0 0x7f4e3019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e3626d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e30107480 0x7f4e301989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55630/0 (socket says 172.21.15.40:55630) 2024-06-06T03:41:20.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e3626d700 1 -- 172.21.15.40:0/2250342596 learned_addr learned my addr 172.21.15.40:0/2250342596 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:41:20.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e35a6c700 1 --2- 172.21.15.40:0/2250342596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e301024e0 0x7f4e3019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e3626d700 1 -- 172.21.15.40:0/2250342596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e301024e0 msgr2=0x7f4e3019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:20.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e3626d700 1 --2- 172.21.15.40:0/2250342596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e301024e0 0x7f4e3019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e3626d700 1 -- 172.21.15.40:0/2250342596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e30102ea0 msgr2=0x7f4e3019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:20.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e3626d700 1 --2- 172.21.15.40:0/2250342596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e30102ea0 0x7f4e3019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e3626d700 1 -- 172.21.15.40:0/2250342596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e30199140 con 0x7f4e30107480 2024-06-06T03:41:20.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e3526b700 1 --2- 172.21.15.40:0/2250342596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e30102ea0 0x7f4e3019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:41:20.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e3626d700 1 --2- 172.21.15.40:0/2250342596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e30107480 0x7f4e301989e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4e28006070 tx=0x7f4e28007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:20.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e26ffd700 1 -- 172.21.15.40:0/2250342596 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28003c00 con 0x7f4e30107480 2024-06-06T03:41:20.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e37cd0700 1 -- 172.21.15.40:0/2250342596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e301993d0 con 0x7f4e30107480 2024-06-06T03:41:20.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.186+0000 7f4e26ffd700 1 -- 172.21.15.40:0/2250342596 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e28003da0 con 0x7f4e30107480 2024-06-06T03:41:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.187+0000 7f4e26ffd700 1 -- 172.21.15.40:0/2250342596 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2801b960 con 0x7f4e30107480 2024-06-06T03:41:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.187+0000 7f4e37cd0700 1 -- 172.21.15.40:0/2250342596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e301a56f0 con 0x7f4e30107480 2024-06-06T03:41:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.189+0000 7f4e26ffd700 1 -- 172.21.15.40:0/2250342596 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4e28005670 con 0x7f4e30107480 2024-06-06T03:41:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.189+0000 7f4e26ffd700 1 --2- 172.21.15.40:0/2250342596 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4e1c0a2e30 0x7f4e1c0a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.190+0000 7f4e35a6c700 1 --2- 172.21.15.40:0/2250342596 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4e1c0a2e30 0x7f4e1c0a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.190+0000 7f4e26ffd700 1 -- 172.21.15.40:0/2250342596 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f4e280d0670 con 0x7f4e30107480 2024-06-06T03:41:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.190+0000 7f4e35a6c700 1 --2- 172.21.15.40:0/2250342596 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4e1c0a2e30 0x7f4e1c0a52f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4e30199b20 tx=0x7f4e2c009260 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.190+0000 7f4e42553b80 1 Processor -- start 2024-06-06T03:41:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e42553b80 1 -- start start 2024-06-06T03:41:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e42553b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923b0e0 0x556d8923b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e42553b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8923ba00 0x556d89235330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e42553b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d89235870 0x556d89235cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e42553b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556d891829a0 con 0x556d89235870 2024-06-06T03:41:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e42553b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556d8912a8b0 con 0x556d8923ba00 2024-06-06T03:41:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e42553b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556d8917e960 con 0x556d8923b0e0 2024-06-06T03:41:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e3626d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d89235870 0x556d89235cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e3526b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8923ba00 0x556d89235330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e3626d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d89235870 0x556d89235cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55640/0 (socket says 172.21.15.40:55640) 2024-06-06T03:41:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e3626d700 1 -- 172.21.15.40:0/3274392886 learned_addr learned my addr 172.21.15.40:0/3274392886 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:41:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e35a6c700 1 --2- 172.21.15.40:0/3274392886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923b0e0 0x556d8923b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e3626d700 1 -- 172.21.15.40:0/3274392886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923b0e0 msgr2=0x556d8923b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e3626d700 1 --2- 172.21.15.40:0/3274392886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923b0e0 0x556d8923b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e3626d700 1 -- 172.21.15.40:0/3274392886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8923ba00 msgr2=0x556d89235330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e3626d700 1 --2- 172.21.15.40:0/3274392886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8923ba00 0x556d89235330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.191+0000 7f4e3626d700 1 -- 172.21.15.40:0/3274392886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d891210e0 con 0x556d89235870 2024-06-06T03:41:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.192+0000 7f4e3526b700 1 --2- 172.21.15.40:0/3274392886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8923ba00 0x556d89235330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:41:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.192+0000 7f4e3626d700 1 --2- 172.21.15.40:0/3274392886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d89235870 0x556d89235cb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4e28099110 tx=0x7f4e2800b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d57f00fbb48f49 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.192+0000 7f4e1bfff700 1 -- 172.21.15.40:0/3274392886 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e280a1060 con 0x556d89235870 2024-06-06T03:41:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.192+0000 7f4e1bfff700 1 -- 172.21.15.40:0/3274392886 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e280a5890 con 0x556d89235870 2024-06-06T03:41:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.192+0000 7f4e1bfff700 1 -- 172.21.15.40:0/3274392886 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2809cf10 con 0x556d89235870 2024-06-06T03:41:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.193+0000 7f4e42553b80 1 -- 172.21.15.40:0/3274392886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d89235870 msgr2=0x556d89235cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.193+0000 7f4e42553b80 1 --2- 172.21.15.40:0/3274392886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d89235870 0x556d89235cb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4e28099110 tx=0x7f4e2800b3e0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.193+0000 7f4e42553b80 1 -- 172.21.15.40:0/3274392886 shutdown_connections 2024-06-06T03:41:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.193+0000 7f4e42553b80 1 --2- 172.21.15.40:0/3274392886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923b0e0 0x556d8923b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.193+0000 7f4e42553b80 1 --2- 172.21.15.40:0/3274392886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d89235870 0x556d89235cb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.193+0000 7f4e42553b80 1 --2- 172.21.15.40:0/3274392886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8923ba00 0x556d89235330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.193+0000 7f4e42553b80 1 -- 172.21.15.40:0/3274392886 >> 172.21.15.40:0/3274392886 conn(0x556d8915be50 msgr2=0x556d8922c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.193+0000 7f4e42553b80 1 -- 172.21.15.40:0/3274392886 shutdown_connections 2024-06-06T03:41:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.193+0000 7f4e42553b80 1 -- 172.21.15.40:0/3274392886 wait complete. 2024-06-06T03:41:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.193+0000 7f4e42553b80 1 -- start start 2024-06-06T03:41:20.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.193+0000 7f4e42553b80 1 Processor -- start 2024-06-06T03:41:20.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e42553b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d89235870 0x556d89241800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e42553b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8923b0e0 0x556d8923eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e42553b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923ba00 0x556d8923f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e42553b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x556d891829a0 con 0x556d8923ba00 2024-06-06T03:41:20.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e42553b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556d89122d80 con 0x556d8923b0e0 2024-06-06T03:41:20.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e42553b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556d89242190 con 0x556d89235870 2024-06-06T03:41:20.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e42553b80 10 client.0 _ll_register_callbacks cb 0x556d8911cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:41:20.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e3626d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923ba00 0x556d8923f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e3626d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923ba00 0x556d8923f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55644/0 (socket says 172.21.15.40:55644) 2024-06-06T03:41:20.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e3626d700 1 -- 172.21.15.40:0/1281475264 learned_addr learned my addr 172.21.15.40:0/1281475264 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:41:20.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e3526b700 1 --2- 172.21.15.40:0/1281475264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8923b0e0 0x556d8923eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e35a6c700 1 --2- 172.21.15.40:0/1281475264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d89235870 0x556d89241800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e3626d700 1 -- 172.21.15.40:0/1281475264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d89235870 msgr2=0x556d89241800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e3626d700 1 --2- 172.21.15.40:0/1281475264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d89235870 0x556d89241800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e3626d700 1 -- 172.21.15.40:0/1281475264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8923b0e0 msgr2=0x556d8923eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e3626d700 1 --2- 172.21.15.40:0/1281475264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8923b0e0 0x556d8923eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e3626d700 1 -- 172.21.15.40:0/1281475264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d890d1a60 con 0x556d8923ba00 2024-06-06T03:41:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.194+0000 7f4e35a6c700 1 --2- 172.21.15.40:0/1281475264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d89235870 0x556d89241800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:41:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e3626d700 1 --2- 172.21.15.40:0/1281475264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923ba00 0x556d8923f040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4e280a5360 tx=0x7f4e28005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55343d585cba9fe server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e197fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:41:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e197fa700 1 -- 172.21.15.40:0/1281475264 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e280b9030 con 0x556d8923ba00 2024-06-06T03:41:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e197fa700 1 -- 172.21.15.40:0/1281475264 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f4e2809afb0 con 0x556d8923ba00 2024-06-06T03:41:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e42553b80 1 -- 172.21.15.40:0/1281475264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e280a5890 con 0x556d8923ba00 2024-06-06T03:41:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e197fa700 1 -- 172.21.15.40:0/1281475264 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4e280074e0 con 0x556d8923ba00 2024-06-06T03:41:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e42553b80 10 client.15489 fetch_fsmap learned FSMap version 34 2024-06-06T03:41:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e42553b80 1 -- 172.21.15.40:0/1281475264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556d8924c9c0 con 0x556d8923ba00 2024-06-06T03:41:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e197fa700 1 -- 172.21.15.40:0/1281475264 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e280a5890 con 0x556d8923ba00 2024-06-06T03:41:20.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e197fa700 1 -- 172.21.15.40:0/1281475264 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f4e28011070 con 0x556d8923ba00 2024-06-06T03:41:20.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e42553b80 10 client.15489 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T03:41:20.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e42553b80 10 client.15489 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:41:20.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e42553b80 20 client.15489 populate_metadata read hostname 'smithi040' 2024-06-06T03:41:20.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e42553b80 1 --2- 172.21.15.40:0/1281475264 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x556d8924efb0 0x556d892513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e42553b80 4 client.15489 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:41:20.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.195+0000 7f4e42553b80 1 -- 172.21.15.40:0/1281475264 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d8918fea0 con 0x556d8924efb0 2024-06-06T03:41:20.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.196+0000 7f4e35a6c700 1 --2- 172.21.15.40:0/1281475264 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x556d8924efb0 0x556d892513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.196+0000 7f4e197fa700 10 client.15489 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:41:20.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.196+0000 7f4e35a6c700 1 --2- 172.21.15.40:0/1281475264 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x556d8924efb0 0x556d892513f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d631c2e7e1227b43 server_cookie=825c9994915db3b0 in_seq=0 out_seq=0 2024-06-06T03:41:20.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.197+0000 7f4e197fa700 1 -- 172.21.15.40:0/1281475264 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556d8918fea0 con 0x556d8924efb0 2024-06-06T03:41:20.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.197+0000 7f4e197fa700 10 client.15489 handle_command_reply: tid=0 2024-06-06T03:41:20.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.197+0000 7f4e42553b80 1 client.15489 shutdown 2024-06-06T03:41:20.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.198+0000 7f4e42553b80 1 -- 172.21.15.40:0/1281475264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923ba00 msgr2=0x556d8923f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:20.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.198+0000 7f4e42553b80 1 --2- 172.21.15.40:0/1281475264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923ba00 0x556d8923f040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4e280a5360 tx=0x7f4e28005ad0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.198+0000 7f4e42553b80 1 -- 172.21.15.40:0/1281475264 shutdown_connections 2024-06-06T03:41:20.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.198+0000 7f4e42553b80 1 --2- 172.21.15.40:0/1281475264 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x556d8924efb0 0x556d892513f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.198+0000 7f4e42553b80 1 --2- 172.21.15.40:0/1281475264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d89235870 0x556d89241800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.198+0000 7f4e42553b80 1 --2- 172.21.15.40:0/1281475264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923ba00 0x556d8923f040 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.198+0000 7f4e42553b80 1 --2- 172.21.15.40:0/1281475264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8923b0e0 0x556d8923eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.198+0000 7f4e197fa700 0 client.15489 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:41:20.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.198+0000 7f4e42553b80 1 -- 172.21.15.40:0/1281475264 >> 172.21.15.40:0/1281475264 conn(0x556d8915be50 msgr2=0x556d8922bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:20.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.198+0000 7f4e42553b80 1 -- 172.21.15.40:0/1281475264 shutdown_connections 2024-06-06T03:41:20.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.198+0000 7f4e42553b80 1 -- 172.21.15.40:0/1281475264 wait complete. 2024-06-06T03:41:20.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.198+0000 7f4e42553b80 20 client.15489 trim_cache size 0 max 16384 2024-06-06T03:41:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.220+0000 7f4e42553b80 1 Processor -- start 2024-06-06T03:41:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e42553b80 1 -- start start 2024-06-06T03:41:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e42553b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d89235870 0x556d8924b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e42553b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923b0e0 0x556d89256cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e42553b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923ba00 0x556d89257200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e42553b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4e301aa9a0 con 0x556d8923b0e0 2024-06-06T03:41:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e42553b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4e301ac5d0 con 0x556d89235870 2024-06-06T03:41:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e42553b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4e301ab070 con 0x556d8923ba00 2024-06-06T03:41:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e3526b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923b0e0 0x556d89256cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e35a6c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d89235870 0x556d8924b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e35a6c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d89235870 0x556d8924b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:59442/0 (socket says 172.21.15.40:59442) 2024-06-06T03:41:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e3526b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923b0e0 0x556d89256cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55660/0 (socket says 172.21.15.40:55660) 2024-06-06T03:41:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e35a6c700 1 -- 172.21.15.40:0/697100385 learned_addr learned my addr 172.21.15.40:0/697100385 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:41:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e3626d700 1 --2- 172.21.15.40:0/697100385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923ba00 0x556d89257200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e35a6c700 1 -- 172.21.15.40:0/697100385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923ba00 msgr2=0x556d89257200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e35a6c700 1 --2- 172.21.15.40:0/697100385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923ba00 0x556d89257200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e35a6c700 1 -- 172.21.15.40:0/697100385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923b0e0 msgr2=0x556d89256cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e35a6c700 1 --2- 172.21.15.40:0/697100385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923b0e0 0x556d89256cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.221+0000 7f4e35a6c700 1 -- 172.21.15.40:0/697100385 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d89241340 con 0x556d89235870 2024-06-06T03:41:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.222+0000 7f4e35a6c700 1 --2- 172.21.15.40:0/697100385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d89235870 0x556d8924b2b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0097d0 tx=0x7f4e2c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63527639e857473e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.222+0000 7f4e1b7fe700 1 -- 172.21.15.40:0/697100385 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c011c00 con 0x556d89235870 2024-06-06T03:41:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.222+0000 7f4e1b7fe700 1 -- 172.21.15.40:0/697100385 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e2c011da0 con 0x556d89235870 2024-06-06T03:41:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.222+0000 7f4e1b7fe700 1 -- 172.21.15.40:0/697100385 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c0275c0 con 0x556d89235870 2024-06-06T03:41:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.223+0000 7f4e42553b80 1 -- 172.21.15.40:0/697100385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d89235870 msgr2=0x556d8924b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.223+0000 7f4e42553b80 1 --2- 172.21.15.40:0/697100385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d89235870 0x556d8924b2b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0097d0 tx=0x7f4e2c013000 comp rx=0 tx=0).stop 2024-06-06T03:41:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.223+0000 7f4e42553b80 1 -- 172.21.15.40:0/697100385 shutdown_connections 2024-06-06T03:41:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.223+0000 7f4e42553b80 1 --2- 172.21.15.40:0/697100385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923ba00 0x556d89257200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.223+0000 7f4e42553b80 1 --2- 172.21.15.40:0/697100385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d89235870 0x556d8924b2b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.223+0000 7f4e42553b80 1 --2- 172.21.15.40:0/697100385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923b0e0 0x556d89256cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.223+0000 7f4e42553b80 1 -- 172.21.15.40:0/697100385 >> 172.21.15.40:0/697100385 conn(0x556d8915be50 msgr2=0x556d8922ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.223+0000 7f4e42553b80 1 -- 172.21.15.40:0/697100385 shutdown_connections 2024-06-06T03:41:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.223+0000 7f4e42553b80 1 -- 172.21.15.40:0/697100385 wait complete. 2024-06-06T03:41:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.223+0000 7f4e42553b80 1 -- start start 2024-06-06T03:41:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.223+0000 7f4e42553b80 1 Processor -- start 2024-06-06T03:41:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e42553b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d89235870 0x556d892443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e42553b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923b0e0 0x556d89244930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e42553b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923ba00 0x556d89244e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e42553b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4e301abe40 con 0x556d8923ba00 2024-06-06T03:41:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e42553b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4e301ab740 con 0x556d89235870 2024-06-06T03:41:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e42553b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4e301a89a0 con 0x556d8923b0e0 2024-06-06T03:41:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e42553b80 10 client.0 _ll_register_callbacks cb 0x556d8911cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:41:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e35a6c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d89235870 0x556d892443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e3626d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923ba00 0x556d89244e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e35a6c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d89235870 0x556d892443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:59444/0 (socket says 172.21.15.40:59444) 2024-06-06T03:41:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e35a6c700 1 -- 172.21.15.40:0/3219069536 learned_addr learned my addr 172.21.15.40:0/3219069536 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:41:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e3526b700 1 --2- 172.21.15.40:0/3219069536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923b0e0 0x556d89244930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e3626d700 1 -- 172.21.15.40:0/3219069536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923b0e0 msgr2=0x556d89244930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e3626d700 1 --2- 172.21.15.40:0/3219069536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923b0e0 0x556d89244930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e3626d700 1 -- 172.21.15.40:0/3219069536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d89235870 msgr2=0x556d892443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e3626d700 1 --2- 172.21.15.40:0/3219069536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d89235870 0x556d892443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e3626d700 1 -- 172.21.15.40:0/3219069536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d892b9d50 con 0x556d8923ba00 2024-06-06T03:41:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e35a6c700 1 --2- 172.21.15.40:0/3219069536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d89235870 0x556d892443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:41:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e3626d700 1 --2- 172.21.15.40:0/3219069536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923ba00 0x556d89244e70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4e28099680 tx=0x7f4e28005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc6ff90044d3385 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.224+0000 7f4e197fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:41:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.225+0000 7f4e197fa700 1 -- 172.21.15.40:0/3219069536 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28005e60 con 0x556d8923ba00 2024-06-06T03:41:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.225+0000 7f4e42553b80 1 -- 172.21.15.40:0/3219069536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e2c011da0 con 0x556d8923ba00 2024-06-06T03:41:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.225+0000 7f4e197fa700 1 -- 172.21.15.40:0/3219069536 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f4e2809cf10 con 0x556d8923ba00 2024-06-06T03:41:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.225+0000 7f4e197fa700 1 -- 172.21.15.40:0/3219069536 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4e280a5890 con 0x556d8923ba00 2024-06-06T03:41:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.225+0000 7f4e42553b80 10 client.15495 fetch_fsmap learned FSMap version 34 2024-06-06T03:41:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.225+0000 7f4e42553b80 1 -- 172.21.15.40:0/3219069536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556d892b9d50 con 0x556d8923ba00 2024-06-06T03:41:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.225+0000 7f4e197fa700 1 -- 172.21.15.40:0/3219069536 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e2c011da0 con 0x556d8923ba00 2024-06-06T03:41:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.225+0000 7f4e197fa700 1 -- 172.21.15.40:0/3219069536 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f4e28011070 con 0x556d8923ba00 2024-06-06T03:41:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.225+0000 7f4e42553b80 10 client.15495 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T03:41:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.225+0000 7f4e42553b80 10 client.15495 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:41:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.225+0000 7f4e42553b80 20 client.15495 populate_metadata read hostname 'smithi040' 2024-06-06T03:41:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.225+0000 7f4e42553b80 1 --2- 172.21.15.40:0/3219069536 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x556d892bb340 0x556d892bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.225+0000 7f4e42553b80 4 client.15495 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}] 2024-06-06T03:41:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.225+0000 7f4e42553b80 1 -- 172.21.15.40:0/3219069536 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}) -- 0x556d8918a420 con 0x556d892bb340 2024-06-06T03:41:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.226+0000 7f4e35a6c700 1 --2- 172.21.15.40:0/3219069536 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x556d892bb340 0x556d892bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.226+0000 7f4e35a6c700 1 --2- 172.21.15.40:0/3219069536 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x556d892bb340 0x556d892bd760 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc42d2569cf7270b server_cookie=e5d8d570e5ea01eb in_seq=0 out_seq=0 2024-06-06T03:41:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:20.226+0000 7f4e197fa700 10 client.15495 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:41:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:20 smithi040 ceph-mon[32220]: pgmap v84: 97 pgs: 97 active+clean; 266 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 56 op/s 2024-06-06T03:41:21.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:20 smithi104 ceph-mon[36569]: pgmap v84: 97 pgs: 97 active+clean; 266 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 56 op/s 2024-06-06T03:41:21.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:20 smithi131 ceph-mon[36687]: pgmap v84: 97 pgs: 97 active+clean; 266 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 56 op/s 2024-06-06T03:41:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:21 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:41:22.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:21 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:41:22.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:21 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:41:23.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:22 smithi040 ceph-mon[32220]: pgmap v85: 97 pgs: 97 active+clean; 266 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 912 KiB/s wr, 35 op/s 2024-06-06T03:41:23.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:22 smithi104 ceph-mon[36569]: pgmap v85: 97 pgs: 97 active+clean; 266 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 912 KiB/s wr, 35 op/s 2024-06-06T03:41:23.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:22 smithi131 ceph-mon[36687]: pgmap v85: 97 pgs: 97 active+clean; 266 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 912 KiB/s wr, 35 op/s 2024-06-06T03:41:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:24 smithi040 ceph-mon[32220]: pgmap v86: 97 pgs: 97 active+clean; 266 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 912 KiB/s wr, 35 op/s 2024-06-06T03:41:25.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:24 smithi104 ceph-mon[36569]: pgmap v86: 97 pgs: 97 active+clean; 266 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 912 KiB/s wr, 35 op/s 2024-06-06T03:41:25.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:24 smithi131 ceph-mon[36687]: pgmap v86: 97 pgs: 97 active+clean; 266 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 912 KiB/s wr, 35 op/s 2024-06-06T03:41:27.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:26 smithi040 ceph-mon[32220]: pgmap v87: 97 pgs: 97 active+clean; 290 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 546 op/s 2024-06-06T03:41:27.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:26 smithi104 ceph-mon[36569]: pgmap v87: 97 pgs: 97 active+clean; 290 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 546 op/s 2024-06-06T03:41:27.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:26 smithi131 ceph-mon[36687]: pgmap v87: 97 pgs: 97 active+clean; 290 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 546 op/s 2024-06-06T03:41:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:28 smithi040 ceph-mon[32220]: pgmap v88: 97 pgs: 97 active+clean; 290 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 526 op/s 2024-06-06T03:41:29.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:28 smithi104 ceph-mon[36569]: pgmap v88: 97 pgs: 97 active+clean; 290 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 526 op/s 2024-06-06T03:41:29.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:28 smithi131 ceph-mon[36687]: pgmap v88: 97 pgs: 97 active+clean; 290 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 526 op/s 2024-06-06T03:41:31.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:30 smithi040 ceph-mon[32220]: pgmap v89: 97 pgs: 97 active+clean; 319 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.7 MiB/s wr, 973 op/s 2024-06-06T03:41:31.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:30 smithi104 ceph-mon[36569]: pgmap v89: 97 pgs: 97 active+clean; 319 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.7 MiB/s wr, 973 op/s 2024-06-06T03:41:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:30 smithi131 ceph-mon[36687]: pgmap v89: 97 pgs: 97 active+clean; 319 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.7 MiB/s wr, 973 op/s 2024-06-06T03:41:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:32 smithi040 ceph-mon[32220]: pgmap v90: 97 pgs: 97 active+clean; 319 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 957 op/s 2024-06-06T03:41:33.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:32 smithi131 ceph-mon[36687]: pgmap v90: 97 pgs: 97 active+clean; 319 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 957 op/s 2024-06-06T03:41:33.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:32 smithi104 ceph-mon[36569]: pgmap v90: 97 pgs: 97 active+clean; 319 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 957 op/s 2024-06-06T03:41:33.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e197fa700 1 -- 172.21.15.40:0/3219069536 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556d8918a420 con 0x556d892bb340 2024-06-06T03:41:33.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e197fa700 10 client.15495 handle_command_reply: tid=0 2024-06-06T03:41:33.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e42553b80 1 client.15495 shutdown 2024-06-06T03:41:33.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e42553b80 1 -- 172.21.15.40:0/3219069536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923ba00 msgr2=0x556d89244e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:33.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e42553b80 1 --2- 172.21.15.40:0/3219069536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923ba00 0x556d89244e70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4e28099680 tx=0x7f4e28005980 comp rx=0 tx=0).stop 2024-06-06T03:41:33.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e42553b80 1 -- 172.21.15.40:0/3219069536 shutdown_connections 2024-06-06T03:41:33.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e42553b80 1 --2- 172.21.15.40:0/3219069536 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x556d892bb340 0x556d892bd760 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:33.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e42553b80 1 --2- 172.21.15.40:0/3219069536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556d8923ba00 0x556d89244e70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:33.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e42553b80 1 --2- 172.21.15.40:0/3219069536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d89235870 0x556d892443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:33.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e42553b80 1 --2- 172.21.15.40:0/3219069536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d8923b0e0 0x556d89244930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:33.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e42553b80 1 -- 172.21.15.40:0/3219069536 >> 172.21.15.40:0/3219069536 conn(0x556d8915be50 msgr2=0x556d8923d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:33.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e197fa700 0 client.15495 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:41:33.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e42553b80 1 -- 172.21.15.40:0/3219069536 shutdown_connections 2024-06-06T03:41:33.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e42553b80 1 -- 172.21.15.40:0/3219069536 wait complete. 2024-06-06T03:41:33.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.094+0000 7f4e42553b80 20 client.15495 trim_cache size 0 max 16384 2024-06-06T03:41:33.096 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:41:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 35, 2024-06-06T03:41:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:41:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 6, 2024-06-06T03:41:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": { 2024-06-06T03:41:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: "4a80a44d": { 2024-06-06T03:41:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 6, 2024-06-06T03:41:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: "age_ref": 0.0, 2024-06-06T03:41:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:41:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "QUIESCED", 2024-06-06T03:41:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:41:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:41:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "timeout": 90.0, 2024-06-06T03:41:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "expiration": 145.2, 2024-06-06T03:41:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "members": { 2024-06-06T03:41:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "file:/": { 2024-06-06T03:41:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "excluded": false, 2024-06-06T03:41:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:41:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "QUIESCED", 2024-06-06T03:41:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:41:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:41:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:41:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:41:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:41:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:41:33.098 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:41:33.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.095+0000 7f4e37cd0700 1 -- 172.21.15.40:0/2250342596 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4e1c0a2e30 msgr2=0x7f4e1c0a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:33.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.095+0000 7f4e37cd0700 1 --2- 172.21.15.40:0/2250342596 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4e1c0a2e30 0x7f4e1c0a52f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4e30199b20 tx=0x7f4e2c009260 comp rx=0 tx=0).stop 2024-06-06T03:41:33.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.095+0000 7f4e37cd0700 1 -- 172.21.15.40:0/2250342596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e30107480 msgr2=0x7f4e301989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:33.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.095+0000 7f4e37cd0700 1 --2- 172.21.15.40:0/2250342596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e30107480 0x7f4e301989e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4e28006070 tx=0x7f4e28007a20 comp rx=0 tx=0).stop 2024-06-06T03:41:33.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.095+0000 7f4e37cd0700 1 -- 172.21.15.40:0/2250342596 shutdown_connections 2024-06-06T03:41:33.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.095+0000 7f4e37cd0700 1 --2- 172.21.15.40:0/2250342596 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4e1c0a2e30 0x7f4e1c0a52f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:33.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.095+0000 7f4e37cd0700 1 --2- 172.21.15.40:0/2250342596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e301024e0 0x7f4e3019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:33.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.095+0000 7f4e37cd0700 1 --2- 172.21.15.40:0/2250342596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e30107480 0x7f4e301989e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:33.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.095+0000 7f4e37cd0700 1 --2- 172.21.15.40:0/2250342596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e30102ea0 0x7f4e3019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:33.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.095+0000 7f4e37cd0700 1 -- 172.21.15.40:0/2250342596 >> 172.21.15.40:0/2250342596 conn(0x7f4e30079640 msgr2=0x7f4e301004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:33.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.095+0000 7f4e37cd0700 1 -- 172.21.15.40:0/2250342596 shutdown_connections 2024-06-06T03:41:33.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:33.095+0000 7f4e37cd0700 1 -- 172.21.15.40:0/2250342596 wait complete. 2024-06-06T03:41:33.980 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4a80a44d', quiesce took 18.7 seconds. Will release after: 6.5 2024-06-06T03:41:35.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:34 smithi040 ceph-mon[32220]: pgmap v91: 97 pgs: 97 active+clean; 319 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 957 op/s 2024-06-06T03:41:35.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:34 smithi104 ceph-mon[36569]: pgmap v91: 97 pgs: 97 active+clean; 319 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 957 op/s 2024-06-06T03:41:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:34 smithi131 ceph-mon[36687]: pgmap v91: 97 pgs: 97 active+clean; 319 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 957 op/s 2024-06-06T03:41:37.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:36 smithi040 ceph-mon[32220]: pgmap v92: 97 pgs: 97 active+clean; 349 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.8 MiB/s wr, 1.28k op/s 2024-06-06T03:41:37.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:36 smithi104 ceph-mon[36569]: pgmap v92: 97 pgs: 97 active+clean; 349 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.8 MiB/s wr, 1.28k op/s 2024-06-06T03:41:37.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:36 smithi131 ceph-mon[36687]: pgmap v92: 97 pgs: 97 active+clean; 349 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.8 MiB/s wr, 1.28k op/s 2024-06-06T03:41:39.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:38 smithi040 ceph-mon[32220]: pgmap v93: 97 pgs: 97 active+clean; 349 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.5 MiB/s wr, 773 op/s 2024-06-06T03:41:39.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:38 smithi104 ceph-mon[36569]: pgmap v93: 97 pgs: 97 active+clean; 349 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.5 MiB/s wr, 773 op/s 2024-06-06T03:41:39.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:38 smithi131 ceph-mon[36687]: pgmap v93: 97 pgs: 97 active+clean; 349 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.5 MiB/s wr, 773 op/s 2024-06-06T03:41:40.482 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:41:40.747 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:41:41.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:40 smithi040 ceph-mon[32220]: pgmap v94: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 784 op/s 2024-06-06T03:41:41.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:40 smithi104 ceph-mon[36569]: pgmap v94: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 784 op/s 2024-06-06T03:41:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:40 smithi131 ceph-mon[36687]: pgmap v94: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 784 op/s 2024-06-06T03:41:42.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.550+0000 7f09d5b87700 1 -- 172.21.15.40:0/3617067412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09d0107870 msgr2=0x7f09d010f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:42.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.550+0000 7f09d5b87700 1 --2- 172.21.15.40:0/3617067412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09d0107870 0x7f09d010f3f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f09c800b0d0 tx=0x7f09c800b3e0 comp rx=0 tx=0).stop 2024-06-06T03:41:42.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.550+0000 7f09d5b87700 1 -- 172.21.15.40:0/3617067412 shutdown_connections 2024-06-06T03:41:42.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.550+0000 7f09d5b87700 1 --2- 172.21.15.40:0/3617067412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09d01027a0 0x7f09d0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.550+0000 7f09d5b87700 1 --2- 172.21.15.40:0/3617067412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09d0107870 0x7f09d010f3f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.550+0000 7f09d5b87700 1 --2- 172.21.15.40:0/3617067412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09d0103150 0x7f09d0107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.550+0000 7f09d5b87700 1 -- 172.21.15.40:0/3617067412 >> 172.21.15.40:0/3617067412 conn(0x7f09d0079570 msgr2=0x7f09d0079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.551+0000 7f09d5b87700 1 -- 172.21.15.40:0/3617067412 shutdown_connections 2024-06-06T03:41:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.551+0000 7f09d5b87700 1 -- 172.21.15.40:0/3617067412 wait complete. 2024-06-06T03:41:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.551+0000 7f09d5b87700 1 Processor -- start 2024-06-06T03:41:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.551+0000 7f09d5b87700 1 -- start start 2024-06-06T03:41:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09d5b87700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09d01027a0 0x7f09d0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09d5b87700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09d0103150 0x7f09d0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09d5b87700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09d0107870 0x7f09d0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09d5b87700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f09d01127b0 con 0x7f09d0103150 2024-06-06T03:41:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09d5b87700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0112630 con 0x7f09d01027a0 2024-06-06T03:41:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09d5b87700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0112930 con 0x7f09d0107870 2024-06-06T03:41:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09c6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09d0103150 0x7f09d0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09c6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09d0103150 0x7f09d0074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39048/0 (socket says 172.21.15.40:39048) 2024-06-06T03:41:42.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09c6ffd700 1 -- 172.21.15.40:0/2578209644 learned_addr learned my addr 172.21.15.40:0/2578209644 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:41:42.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09cf7fe700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09d01027a0 0x7f09d0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:42.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09cffff700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09d0107870 0x7f09d0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:42.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09c6ffd700 1 -- 172.21.15.40:0/2578209644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09d0107870 msgr2=0x7f09d0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:42.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09c6ffd700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09d0107870 0x7f09d0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:42.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09c6ffd700 1 -- 172.21.15.40:0/2578209644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09d01027a0 msgr2=0x7f09d0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:42.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09c6ffd700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09d01027a0 0x7f09d0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:42.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09c6ffd700 1 -- 172.21.15.40:0/2578209644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d01a7070 con 0x7f09d0103150 2024-06-06T03:41:42.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09cf7fe700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09d01027a0 0x7f09d0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:41:42.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09cffff700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09d0107870 0x7f09d0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:41:42.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.552+0000 7f09c6ffd700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09d0103150 0x7f09d0074cd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f09b800c920 tx=0x7f09b800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:42.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.553+0000 7f09cd7fa700 1 -- 172.21.15.40:0/2578209644 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8007b00 con 0x7f09d0103150 2024-06-06T03:41:42.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.553+0000 7f09cd7fa700 1 -- 172.21.15.40:0/2578209644 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b8004d10 con 0x7f09d0103150 2024-06-06T03:41:42.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.553+0000 7f09d5b87700 1 -- 172.21.15.40:0/2578209644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d01a7360 con 0x7f09d0103150 2024-06-06T03:41:42.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.553+0000 7f09cd7fa700 1 -- 172.21.15.40:0/2578209644 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b80056e0 con 0x7f09d0103150 2024-06-06T03:41:42.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.553+0000 7f09d5b87700 1 -- 172.21.15.40:0/2578209644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d01a78a0 con 0x7f09d0103150 2024-06-06T03:41:42.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.554+0000 7f09d5b87700 1 -- 172.21.15.40:0/2578209644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d0069180 con 0x7f09d0103150 2024-06-06T03:41:42.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.555+0000 7f09cd7fa700 1 -- 172.21.15.40:0/2578209644 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f09b8007ca0 con 0x7f09d0103150 2024-06-06T03:41:42.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.556+0000 7f09cd7fa700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f09bc0a2d90 0x7f09bc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:42.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.556+0000 7f09cf7fe700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f09bc0a2d90 0x7f09bc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:42.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.556+0000 7f09cd7fa700 1 -- 172.21.15.40:0/2578209644 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f09b80cf3e0 con 0x7f09d0103150 2024-06-06T03:41:42.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.556+0000 7f09cf7fe700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f09bc0a2d90 0x7f09bc0a5250 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f09c000bcf0 tx=0x7f09c000b2c0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:42.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.559+0000 7f09cd7fa700 1 -- 172.21.15.40:0/2578209644 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f09b8014030 con 0x7f09d0103150 2024-06-06T03:41:42.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.812+0000 7f09d5b87700 1 -- 172.21.15.40:0/2578209644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09d0075cc0 con 0x7f09d0103150 2024-06-06T03:41:42.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.813+0000 7f09cd7fa700 1 -- 172.21.15.40:0/2578209644 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11945 (secure 0 0 0) 0x7f09d0075cc0 con 0x7f09d0103150 2024-06-06T03:41:42.812 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:41:42.812 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:41:20.463469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24443,24470,24425,24403,24398]},"id":1}]} 2024-06-06T03:41:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.815+0000 7f09d5b87700 1 -- 172.21.15.40:0/2578209644 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f09bc0a2d90 msgr2=0x7f09bc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.815+0000 7f09d5b87700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f09bc0a2d90 0x7f09bc0a5250 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f09c000bcf0 tx=0x7f09c000b2c0 comp rx=0 tx=0).stop 2024-06-06T03:41:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.815+0000 7f09d5b87700 1 -- 172.21.15.40:0/2578209644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09d0103150 msgr2=0x7f09d0074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.815+0000 7f09d5b87700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09d0103150 0x7f09d0074cd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f09b800c920 tx=0x7f09b800cce0 comp rx=0 tx=0).stop 2024-06-06T03:41:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.815+0000 7f09d5b87700 1 -- 172.21.15.40:0/2578209644 shutdown_connections 2024-06-06T03:41:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.815+0000 7f09d5b87700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f09bc0a2d90 0x7f09bc0a5250 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.815+0000 7f09d5b87700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09d0107870 0x7f09d0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.815+0000 7f09d5b87700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09d01027a0 0x7f09d0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.815+0000 7f09d5b87700 1 --2- 172.21.15.40:0/2578209644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09d0103150 0x7f09d0074cd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:42.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.815+0000 7f09d5b87700 1 -- 172.21.15.40:0/2578209644 >> 172.21.15.40:0/2578209644 conn(0x7f09d0079570 msgr2=0x7f09d01004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:42.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.816+0000 7f09d5b87700 1 -- 172.21.15.40:0/2578209644 shutdown_connections 2024-06-06T03:41:42.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:42.816+0000 7f09d5b87700 1 -- 172.21.15.40:0/2578209644 wait complete. 2024-06-06T03:41:42.816 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 35 2024-06-06T03:41:43.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:42 smithi040 ceph-mon[32220]: pgmap v95: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-06-06T03:41:43.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:42 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2578209644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:41:43.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:42 smithi104 ceph-mon[36569]: pgmap v95: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-06-06T03:41:43.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:42 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2578209644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:41:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:42 smithi131 ceph-mon[36687]: pgmap v95: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-06-06T03:41:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:42 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2578209644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:41:43.524 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 4a80a44d --release --await' 2024-06-06T03:41:43.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 4a80a44d --release --await 2024-06-06T03:41:43.779 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:41:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:44 smithi040 ceph-mon[32220]: pgmap v96: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-06-06T03:41:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:44 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:41:45.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:44 smithi104 ceph-mon[36569]: pgmap v96: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-06-06T03:41:45.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:44 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:41:45.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:44 smithi131 ceph-mon[36687]: pgmap v96: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-06-06T03:41:45.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:44 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:41:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.778+0000 7f8ea5d27700 1 -- 172.21.15.40:0/3137440654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea01084f0 msgr2=0x7f8ea01088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.778+0000 7f8ea5d27700 1 --2- 172.21.15.40:0/3137440654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea01084f0 0x7f8ea01088d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009990 tx=0x7f8e90009ca0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.778+0000 7f8ea5d27700 1 -- 172.21.15.40:0/3137440654 shutdown_connections 2024-06-06T03:41:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.778+0000 7f8ea5d27700 1 --2- 172.21.15.40:0/3137440654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea0102f10 0x7f8ea010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.778+0000 7f8ea5d27700 1 --2- 172.21.15.40:0/3137440654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea01084f0 0x7f8ea01088d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.778+0000 7f8ea5d27700 1 --2- 172.21.15.40:0/3137440654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ea0102550 0x7f8ea01029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.778+0000 7f8ea5d27700 1 -- 172.21.15.40:0/3137440654 >> 172.21.15.40:0/3137440654 conn(0x7f8ea0079650 msgr2=0x7f8ea0079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.778+0000 7f8ea5d27700 1 -- 172.21.15.40:0/3137440654 shutdown_connections 2024-06-06T03:41:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.778+0000 7f8ea5d27700 1 -- 172.21.15.40:0/3137440654 wait complete. 2024-06-06T03:41:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.779+0000 7f8ea5d27700 1 Processor -- start 2024-06-06T03:41:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.779+0000 7f8ea5d27700 1 -- start start 2024-06-06T03:41:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.779+0000 7f8ea5d27700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea0102550 0x7f8ea0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.779+0000 7f8ea5d27700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ea0102f10 0x7f8ea0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.779+0000 7f8ea5d27700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea01084f0 0x7f8ea0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.779+0000 7f8ea5d27700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea0112750 con 0x7f8ea0102550 2024-06-06T03:41:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.779+0000 7f8ea5d27700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01125d0 con 0x7f8ea0102f10 2024-06-06T03:41:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.779+0000 7f8ea5d27700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01128d0 con 0x7f8ea01084f0 2024-06-06T03:41:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea0102550 0x7f8ea0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea0102550 0x7f8ea0074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39066/0 (socket says 172.21.15.40:39066) 2024-06-06T03:41:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9f7fe700 1 -- 172.21.15.40:0/2333884004 learned_addr learned my addr 172.21.15.40:0/2333884004 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:41:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9ffff700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea01084f0 0x7f8ea0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9effd700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ea0102f10 0x7f8ea0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9f7fe700 1 -- 172.21.15.40:0/2333884004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea01084f0 msgr2=0x7f8ea0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9f7fe700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea01084f0 0x7f8ea0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9f7fe700 1 -- 172.21.15.40:0/2333884004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ea0102f10 msgr2=0x7f8ea0074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9f7fe700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ea0102f10 0x7f8ea0074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9f7fe700 1 -- 172.21.15.40:0/2333884004 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea01a71a0 con 0x7f8ea0102550 2024-06-06T03:41:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9effd700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ea0102f10 0x7f8ea0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:41:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9f7fe700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea0102550 0x7f8ea0074790 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8e90000c00 tx=0x7f8e900047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9cff9700 1 -- 172.21.15.40:0/2333884004 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90027070 con 0x7f8ea0102550 2024-06-06T03:41:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9cff9700 1 -- 172.21.15.40:0/2333884004 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e90003780 con 0x7f8ea0102550 2024-06-06T03:41:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8ea5d27700 1 -- 172.21.15.40:0/2333884004 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea01a7430 con 0x7f8ea0102550 2024-06-06T03:41:45.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.780+0000 7f8e9cff9700 1 -- 172.21.15.40:0/2333884004 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90022dd0 con 0x7f8ea0102550 2024-06-06T03:41:45.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.781+0000 7f8ea5d27700 1 -- 172.21.15.40:0/2333884004 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea01a7880 con 0x7f8ea0102550 2024-06-06T03:41:45.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.783+0000 7f8e9cff9700 1 -- 172.21.15.40:0/2333884004 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8e9002a070 con 0x7f8ea0102550 2024-06-06T03:41:45.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.783+0000 7f8e9cff9700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f8e880a2d90 0x7f8e880a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.784+0000 7f8e9cff9700 1 -- 172.21.15.40:0/2333884004 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f8e900d0100 con 0x7f8ea0102550 2024-06-06T03:41:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.784+0000 7f8e9effd700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f8e880a2d90 0x7f8e880a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.784+0000 7f8e9effd700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f8e880a2d90 0x7f8e880a5250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0079ce0 tx=0x7f8e9400b5f0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.785+0000 7f8eb05aab80 1 Processor -- start 2024-06-06T03:41:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.785+0000 7f8eb05aab80 1 -- start start 2024-06-06T03:41:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.785+0000 7f8eb05aab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668500 0x5559b16688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.785+0000 7f8eb05aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 0x5559b1662710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.785+0000 7f8eb05aab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1662c50 0x5559b1663070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.785+0000 7f8eb05aab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5559b1555060 con 0x5559b1668500 2024-06-06T03:41:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.785+0000 7f8eb05aab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5559b15b3950 con 0x5559b1662c50 2024-06-06T03:41:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.785+0000 7f8eb05aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5559b151b560 con 0x5559b1668e20 2024-06-06T03:41:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.785+0000 7f8e9f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668500 0x5559b16688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.785+0000 7f8e9f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668500 0x5559b16688e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39072/0 (socket says 172.21.15.40:39072) 2024-06-06T03:41:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.785+0000 7f8e9f7fe700 1 -- 172.21.15.40:0/2214653087 learned_addr learned my addr 172.21.15.40:0/2214653087 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:41:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.786+0000 7f8e9effd700 1 --2- 172.21.15.40:0/2214653087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 0x5559b1662710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.786+0000 7f8e9ffff700 1 --2- 172.21.15.40:0/2214653087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1662c50 0x5559b1663070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.786+0000 7f8e9f7fe700 1 -- 172.21.15.40:0/2214653087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 msgr2=0x5559b1662710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.786+0000 7f8e9f7fe700 1 --2- 172.21.15.40:0/2214653087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 0x5559b1662710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.786+0000 7f8e9f7fe700 1 -- 172.21.15.40:0/2214653087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1662c50 msgr2=0x5559b1663070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.786+0000 7f8e9f7fe700 1 --2- 172.21.15.40:0/2214653087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1662c50 0x5559b1663070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.786+0000 7f8e9f7fe700 1 -- 172.21.15.40:0/2214653087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559b15029d0 con 0x5559b1668500 2024-06-06T03:41:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.786+0000 7f8e9f7fe700 1 --2- 172.21.15.40:0/2214653087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668500 0x5559b16688e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8e900a4e20 tx=0x7f8e900a30d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71888d15b1d3800 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.786+0000 7f8e85ffb700 1 -- 172.21.15.40:0/2214653087 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e900a0b20 con 0x5559b1668500 2024-06-06T03:41:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.786+0000 7f8e85ffb700 1 -- 172.21.15.40:0/2214653087 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e900a5350 con 0x5559b1668500 2024-06-06T03:41:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.786+0000 7f8e85ffb700 1 -- 172.21.15.40:0/2214653087 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e900a5120 con 0x5559b1668500 2024-06-06T03:41:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.787+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2214653087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668500 msgr2=0x5559b16688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.787+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/2214653087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668500 0x5559b16688e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8e900a4e20 tx=0x7f8e900a30d0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.787+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2214653087 shutdown_connections 2024-06-06T03:41:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.787+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/2214653087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1662c50 0x5559b1663070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.787+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/2214653087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668500 0x5559b16688e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.787+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/2214653087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 0x5559b1662710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.787+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2214653087 >> 172.21.15.40:0/2214653087 conn(0x5559b158f180 msgr2=0x5559b165da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.787+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2214653087 shutdown_connections 2024-06-06T03:41:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.787+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2214653087 wait complete. 2024-06-06T03:41:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.787+0000 7f8eb05aab80 1 -- start start 2024-06-06T03:41:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.787+0000 7f8eb05aab80 1 Processor -- start 2024-06-06T03:41:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8eb05aab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1662c50 0x5559b166eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8eb05aab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668500 0x5559b166bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8eb05aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 0x5559b166c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8eb05aab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5559b15540c0 con 0x5559b1668500 2024-06-06T03:41:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8eb05aab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5559b15af910 con 0x5559b1662c50 2024-06-06T03:41:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8eb05aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5559b166f5b0 con 0x5559b1668e20 2024-06-06T03:41:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8e9effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668500 0x5559b166bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8e9effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668500 0x5559b166bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39084/0 (socket says 172.21.15.40:39084) 2024-06-06T03:41:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8e9effd700 1 -- 172.21.15.40:0/2332261715 learned_addr learned my addr 172.21.15.40:0/2332261715 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:41:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8eb05aab80 10 client.0 _ll_register_callbacks cb 0x5559b15496b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:41:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8e9ffff700 1 --2- 172.21.15.40:0/2332261715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 0x5559b166c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8e9f7fe700 1 --2- 172.21.15.40:0/2332261715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1662c50 0x5559b166eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8e9effd700 1 -- 172.21.15.40:0/2332261715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 msgr2=0x5559b166c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8e9effd700 1 --2- 172.21.15.40:0/2332261715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 0x5559b166c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8e9effd700 1 -- 172.21.15.40:0/2332261715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1662c50 msgr2=0x5559b166eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8e9effd700 1 --2- 172.21.15.40:0/2332261715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1662c50 0x5559b166eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8e9effd700 1 -- 172.21.15.40:0/2332261715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559b1579850 con 0x5559b1668500 2024-06-06T03:41:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8e9effd700 1 --2- 172.21.15.40:0/2332261715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668500 0x5559b166bef0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400ffe0 tx=0x7f8e94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f901a2e3d0d6ce9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.788+0000 7f8e7effd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:41:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.789+0000 7f8e7effd700 1 -- 172.21.15.40:0/2332261715 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94028670 con 0x5559b1668500 2024-06-06T03:41:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.789+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2332261715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e900a5350 con 0x5559b1668500 2024-06-06T03:41:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.789+0000 7f8e7effd700 1 -- 172.21.15.40:0/2332261715 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f8e9401f430 con 0x5559b1668500 2024-06-06T03:41:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.789+0000 7f8e7effd700 1 -- 172.21.15.40:0/2332261715 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8e94011e40 con 0x5559b1668500 2024-06-06T03:41:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.789+0000 7f8eb05aab80 10 client.15513 fetch_fsmap learned FSMap version 35 2024-06-06T03:41:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.789+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2332261715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5559b1679d70 con 0x5559b1668500 2024-06-06T03:41:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.789+0000 7f8e7effd700 1 -- 172.21.15.40:0/2332261715 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e900a5350 con 0x5559b1668500 2024-06-06T03:41:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.789+0000 7f8e7effd700 1 -- 172.21.15.40:0/2332261715 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f8e9401fc40 con 0x5559b1668500 2024-06-06T03:41:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.789+0000 7f8eb05aab80 10 client.15513 fetch_fsmap finished waiting for FSMap version 35 2024-06-06T03:41:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.789+0000 7f8eb05aab80 10 client.15513 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:41:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.789+0000 7f8eb05aab80 20 client.15513 populate_metadata read hostname 'smithi040' 2024-06-06T03:41:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.789+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/2332261715 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x5559b167c350 0x5559b167e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.789+0000 7f8eb05aab80 4 client.15513 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:41:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.789+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2332261715 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559b15c0e50 con 0x5559b167c350 2024-06-06T03:41:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.790+0000 7f8e9f7fe700 1 --2- 172.21.15.40:0/2332261715 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x5559b167c350 0x5559b167e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.790+0000 7f8e9f7fe700 1 --2- 172.21.15.40:0/2332261715 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x5559b167c350 0x5559b167e770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15449917402b9ec3 server_cookie=c93eab7bbde1231d in_seq=0 out_seq=0 2024-06-06T03:41:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.790+0000 7f8e7effd700 10 client.15513 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:41:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8e7effd700 1 -- 172.21.15.40:0/2332261715 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5559b15c0e50 con 0x5559b167c350 2024-06-06T03:41:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8e7effd700 10 client.15513 handle_command_reply: tid=0 2024-06-06T03:41:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8eb05aab80 1 client.15513 shutdown 2024-06-06T03:41:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2332261715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668500 msgr2=0x5559b166bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/2332261715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668500 0x5559b166bef0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400ffe0 tx=0x7f8e94004750 comp rx=0 tx=0).stop 2024-06-06T03:41:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2332261715 shutdown_connections 2024-06-06T03:41:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/2332261715 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x5559b167c350 0x5559b167e770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/2332261715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 0x5559b166c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/2332261715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1662c50 0x5559b166eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8e7effd700 0 client.15513 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:41:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/2332261715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668500 0x5559b166bef0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2332261715 >> 172.21.15.40:0/2332261715 conn(0x5559b158f180 msgr2=0x5559b165c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2332261715 shutdown_connections 2024-06-06T03:41:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2332261715 wait complete. 2024-06-06T03:41:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.791+0000 7f8eb05aab80 20 client.15513 trim_cache size 0 max 16384 2024-06-06T03:41:45.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8eb05aab80 1 Processor -- start 2024-06-06T03:41:45.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8eb05aab80 1 -- start start 2024-06-06T03:41:45.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8eb05aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1662c50 0x5559b1662710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8eb05aab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1668500 0x5559b16665f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8eb05aab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668e20 0x5559b1669a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8eb05aab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01aec80 con 0x5559b1668e20 2024-06-06T03:41:45.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8eb05aab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01b0910 con 0x5559b1668500 2024-06-06T03:41:45.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8eb05aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01af350 con 0x5559b1662c50 2024-06-06T03:41:45.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8e9effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1668500 0x5559b16665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8e9ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668e20 0x5559b1669a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8e9effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1668500 0x5559b16665f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:54572/0 (socket says 172.21.15.40:54572) 2024-06-06T03:41:45.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8e9effd700 1 -- 172.21.15.40:0/2730817362 learned_addr learned my addr 172.21.15.40:0/2730817362 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:41:45.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8e9f7fe700 1 --2- 172.21.15.40:0/2730817362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1662c50 0x5559b1662710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8e9ffff700 1 -- 172.21.15.40:0/2730817362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1662c50 msgr2=0x5559b1662710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:45.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8e9ffff700 1 --2- 172.21.15.40:0/2730817362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1662c50 0x5559b1662710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8e9ffff700 1 -- 172.21.15.40:0/2730817362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1668500 msgr2=0x5559b16665f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:45.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8e9ffff700 1 --2- 172.21.15.40:0/2730817362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1668500 0x5559b16665f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8e9ffff700 1 -- 172.21.15.40:0/2730817362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559b1679b80 con 0x5559b1668e20 2024-06-06T03:41:45.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.812+0000 7f8e9f7fe700 1 --2- 172.21.15.40:0/2730817362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1662c50 0x5559b1662710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:41:45.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.813+0000 7f8e9ffff700 1 --2- 172.21.15.40:0/2730817362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668e20 0x5559b1669a90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009e80 tx=0x7f8e98009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7c60a4ff6a29b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:45.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.813+0000 7f8e857fa700 1 -- 172.21.15.40:0/2730817362 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9801b030 con 0x5559b1668e20 2024-06-06T03:41:45.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.813+0000 7f8e857fa700 1 -- 172.21.15.40:0/2730817362 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e98004bb0 con 0x5559b1668e20 2024-06-06T03:41:45.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.813+0000 7f8e857fa700 1 -- 172.21.15.40:0/2730817362 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98005260 con 0x5559b1668e20 2024-06-06T03:41:45.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.813+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2730817362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668e20 msgr2=0x5559b1669a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:45.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.813+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/2730817362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668e20 0x5559b1669a90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009e80 tx=0x7f8e98009100 comp rx=0 tx=0).stop 2024-06-06T03:41:45.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2730817362 shutdown_connections 2024-06-06T03:41:45.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/2730817362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1662c50 0x5559b1662710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/2730817362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1668e20 0x5559b1669a90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/2730817362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1668500 0x5559b16665f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2730817362 >> 172.21.15.40:0/2730817362 conn(0x5559b158f180 msgr2=0x5559b165c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:45.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2730817362 shutdown_connections 2024-06-06T03:41:45.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 -- 172.21.15.40:0/2730817362 wait complete. 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 -- start start 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 Processor -- start 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1662c50 0x5559b1674ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1668500 0x5559b1675020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 0x5559b1671870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01b0180 con 0x5559b1662c50 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01afa20 con 0x5559b1668500 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.814+0000 7f8eb05aab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01acc80 con 0x5559b1668e20 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8eb05aab80 10 client.0 _ll_register_callbacks cb 0x5559b166ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e9f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1662c50 0x5559b1674ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e9f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1662c50 0x5559b1674ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39098/0 (socket says 172.21.15.40:39098) 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e9f7fe700 1 -- 172.21.15.40:0/292420173 learned_addr learned my addr 172.21.15.40:0/292420173 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e9effd700 1 --2- 172.21.15.40:0/292420173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1668500 0x5559b1675020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e9ffff700 1 --2- 172.21.15.40:0/292420173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 0x5559b1671870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e9f7fe700 1 -- 172.21.15.40:0/292420173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 msgr2=0x5559b1671870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:45.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e9f7fe700 1 --2- 172.21.15.40:0/292420173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 0x5559b1671870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e9f7fe700 1 -- 172.21.15.40:0/292420173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1668500 msgr2=0x5559b1675020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e9f7fe700 1 --2- 172.21.15.40:0/292420173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1668500 0x5559b1675020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e9f7fe700 1 -- 172.21.15.40:0/292420173 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559b16720b0 con 0x5559b1662c50 2024-06-06T03:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e9f7fe700 1 --2- 172.21.15.40:0/292420173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1662c50 0x5559b1674ae0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8e9009ca40 tx=0x7f8e90009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432e2fcc0dd0807a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e7effd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e7effd700 1 -- 172.21.15.40:0/292420173 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e900a5350 con 0x5559b1662c50 2024-06-06T03:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8eb05aab80 1 -- 172.21.15.40:0/292420173 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e98004bb0 con 0x5559b1662c50 2024-06-06T03:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e7effd700 1 -- 172.21.15.40:0/292420173 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f8e9009aa70 con 0x5559b1662c50 2024-06-06T03:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8e7effd700 1 -- 172.21.15.40:0/292420173 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8e9009cae0 con 0x5559b1662c50 2024-06-06T03:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8eb05aab80 10 client.15519 fetch_fsmap learned FSMap version 35 2024-06-06T03:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.815+0000 7f8eb05aab80 1 -- 172.21.15.40:0/292420173 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5559b16f0450 con 0x5559b1662c50 2024-06-06T03:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.816+0000 7f8e7effd700 1 -- 172.21.15.40:0/292420173 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e98004bb0 con 0x5559b1662c50 2024-06-06T03:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.816+0000 7f8e7effd700 1 -- 172.21.15.40:0/292420173 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f8e900a4ec0 con 0x5559b1662c50 2024-06-06T03:41:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.816+0000 7f8eb05aab80 10 client.15519 fetch_fsmap finished waiting for FSMap version 35 2024-06-06T03:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.816+0000 7f8eb05aab80 10 client.15519 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.816+0000 7f8eb05aab80 20 client.15519 populate_metadata read hostname 'smithi040' 2024-06-06T03:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.816+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/292420173 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x5559b167c350 0x5559b16f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.816+0000 7f8eb05aab80 4 client.15519 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4a80a44d", "release": true, "await": true}] 2024-06-06T03:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.816+0000 7f8eb05aab80 1 -- 172.21.15.40:0/292420173 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4a80a44d", "release": true, "await": true}) -- 0x5559b15bb3d0 con 0x5559b167c350 2024-06-06T03:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.816+0000 7f8e9effd700 1 --2- 172.21.15.40:0/292420173 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x5559b167c350 0x5559b16f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.817+0000 7f8e9effd700 1 --2- 172.21.15.40:0/292420173 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x5559b167c350 0x5559b16f2be0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b3b4691143fecae server_cookie=d244fde215df7180 in_seq=0 out_seq=0 2024-06-06T03:41:45.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:45.817+0000 7f8e7effd700 10 client.15519 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:41:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:45 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:41:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:45 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:41:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:45 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:41:46.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:45 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:41:46.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:45 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:41:46.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:45 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:41:46.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.269+0000 7f8e7effd700 1 -- 172.21.15.40:0/292420173 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5559b15bb3d0 con 0x5559b167c350 2024-06-06T03:41:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.269+0000 7f8e7effd700 10 client.15519 handle_command_reply: tid=0 2024-06-06T03:41:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.269+0000 7f8eb05aab80 1 client.15519 shutdown 2024-06-06T03:41:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.270+0000 7f8eb05aab80 1 -- 172.21.15.40:0/292420173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1662c50 msgr2=0x5559b1674ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.270+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/292420173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1662c50 0x5559b1674ae0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8e9009ca40 tx=0x7f8e90009ca0 comp rx=0 tx=0).stop 2024-06-06T03:41:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.270+0000 7f8eb05aab80 1 -- 172.21.15.40:0/292420173 shutdown_connections 2024-06-06T03:41:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.270+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/292420173 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x5559b167c350 0x5559b16f2be0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.270+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/292420173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5559b1668e20 0x5559b1671870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.270+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/292420173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5559b1662c50 0x5559b1674ae0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.270+0000 7f8eb05aab80 1 --2- 172.21.15.40:0/292420173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559b1668500 0x5559b1675020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.270+0000 7f8eb05aab80 1 -- 172.21.15.40:0/292420173 >> 172.21.15.40:0/292420173 conn(0x5559b158f180 msgr2=0x5559b1669a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.270+0000 7f8e7effd700 0 client.15519 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:41:46.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.270+0000 7f8eb05aab80 1 -- 172.21.15.40:0/292420173 shutdown_connections 2024-06-06T03:41:46.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.270+0000 7f8eb05aab80 1 -- 172.21.15.40:0/292420173 wait complete. 2024-06-06T03:41:46.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.270+0000 7f8eb05aab80 20 client.15519 trim_cache size 0 max 16384 2024-06-06T03:41:46.271 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:41:46.271 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 35, 2024-06-06T03:41:46.271 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:41:46.271 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 8, 2024-06-06T03:41:46.271 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": { 2024-06-06T03:41:46.271 INFO:teuthology.orchestra.run.smithi040.stdout: "4a80a44d": { 2024-06-06T03:41:46.271 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 8, 2024-06-06T03:41:46.272 INFO:teuthology.orchestra.run.smithi040.stdout: "age_ref": 0.0, 2024-06-06T03:41:46.272 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:41:46.272 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RELEASED", 2024-06-06T03:41:46.272 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:41:46.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:41:46.272 INFO:teuthology.orchestra.run.smithi040.stdout: "timeout": 90.0, 2024-06-06T03:41:46.272 INFO:teuthology.orchestra.run.smithi040.stdout: "expiration": 145.2, 2024-06-06T03:41:46.272 INFO:teuthology.orchestra.run.smithi040.stdout: "members": { 2024-06-06T03:41:46.272 INFO:teuthology.orchestra.run.smithi040.stdout: "file:/": { 2024-06-06T03:41:46.273 INFO:teuthology.orchestra.run.smithi040.stdout: "excluded": false, 2024-06-06T03:41:46.273 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:41:46.273 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RELEASED", 2024-06-06T03:41:46.273 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:41:46.273 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:41:46.273 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:41:46.273 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:41:46.273 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:41:46.273 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:41:46.273 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:41:46.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.271+0000 7f8ea5d27700 1 -- 172.21.15.40:0/2333884004 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f8e880a2d90 msgr2=0x7f8e880a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:46.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.271+0000 7f8ea5d27700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f8e880a2d90 0x7f8e880a5250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0079ce0 tx=0x7f8e9400b5f0 comp rx=0 tx=0).stop 2024-06-06T03:41:46.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.271+0000 7f8ea5d27700 1 -- 172.21.15.40:0/2333884004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea0102550 msgr2=0x7f8ea0074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:41:46.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.271+0000 7f8ea5d27700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea0102550 0x7f8ea0074790 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8e90000c00 tx=0x7f8e900047b0 comp rx=0 tx=0).stop 2024-06-06T03:41:46.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.271+0000 7f8ea5d27700 1 -- 172.21.15.40:0/2333884004 shutdown_connections 2024-06-06T03:41:46.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.271+0000 7f8ea5d27700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f8e880a2d90 0x7f8e880a5250 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:46.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.271+0000 7f8ea5d27700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea01084f0 0x7f8ea0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:46.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.271+0000 7f8ea5d27700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea0102550 0x7f8ea0074790 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:46.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.271+0000 7f8ea5d27700 1 --2- 172.21.15.40:0/2333884004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ea0102f10 0x7f8ea0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:41:46.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.271+0000 7f8ea5d27700 1 -- 172.21.15.40:0/2333884004 >> 172.21.15.40:0/2333884004 conn(0x7f8ea0079650 msgr2=0x7f8ea0100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:41:46.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.271+0000 7f8ea5d27700 1 -- 172.21.15.40:0/2333884004 shutdown_connections 2024-06-06T03:41:46.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:41:46.271+0000 7f8ea5d27700 1 -- 172.21.15.40:0/2333884004 wait complete. 2024-06-06T03:41:46.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:45 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:41:46.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:45 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:41:46.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:45 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:41:47.062 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:46 smithi040 ceph-mon[32220]: pgmap v97: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 346 op/s 2024-06-06T03:41:47.090 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4a80a44d', total seconds elapsed: 31.9 2024-06-06T03:41:47.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:46 smithi104 ceph-mon[36569]: pgmap v97: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 346 op/s 2024-06-06T03:41:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:46 smithi131 ceph-mon[36687]: pgmap v97: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 346 op/s 2024-06-06T03:41:49.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:48 smithi104 ceph-mon[36569]: pgmap v98: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 81 KiB/s wr, 18 op/s 2024-06-06T03:41:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:48 smithi131 ceph-mon[36687]: pgmap v98: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 81 KiB/s wr, 18 op/s 2024-06-06T03:41:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:48 smithi040 ceph-mon[32220]: pgmap v98: 97 pgs: 97 active+clean; 350 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 81 KiB/s wr, 18 op/s 2024-06-06T03:41:51.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:50 smithi131 ceph-mon[36687]: pgmap v99: 97 pgs: 97 active+clean; 353 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 361 KiB/s wr, 27 op/s 2024-06-06T03:41:51.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:50 smithi040 ceph-mon[32220]: pgmap v99: 97 pgs: 97 active+clean; 353 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 361 KiB/s wr, 27 op/s 2024-06-06T03:41:51.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:50 smithi104 ceph-mon[36569]: pgmap v99: 97 pgs: 97 active+clean; 353 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 361 KiB/s wr, 27 op/s 2024-06-06T03:41:53.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:52 smithi131 ceph-mon[36687]: pgmap v100: 97 pgs: 97 active+clean; 353 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 289 KiB/s wr, 16 op/s 2024-06-06T03:41:53.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:52 smithi040 ceph-mon[32220]: pgmap v100: 97 pgs: 97 active+clean; 353 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 289 KiB/s wr, 16 op/s 2024-06-06T03:41:53.362 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:52 smithi104 ceph-mon[36569]: pgmap v100: 97 pgs: 97 active+clean; 353 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 289 KiB/s wr, 16 op/s 2024-06-06T03:41:55.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:54 smithi131 ceph-mon[36687]: pgmap v101: 97 pgs: 97 active+clean; 353 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 289 KiB/s wr, 16 op/s 2024-06-06T03:41:55.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:54 smithi040 ceph-mon[32220]: pgmap v101: 97 pgs: 97 active+clean; 353 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 289 KiB/s wr, 16 op/s 2024-06-06T03:41:55.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:54 smithi104 ceph-mon[36569]: pgmap v101: 97 pgs: 97 active+clean; 353 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 289 KiB/s wr, 16 op/s 2024-06-06T03:41:57.131 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:56 smithi040 ceph-mon[32220]: pgmap v102: 97 pgs: 97 active+clean; 356 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 547 KiB/s wr, 28 op/s 2024-06-06T03:41:57.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:56 smithi131 ceph-mon[36687]: pgmap v102: 97 pgs: 97 active+clean; 356 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 547 KiB/s wr, 28 op/s 2024-06-06T03:41:57.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:56 smithi104 ceph-mon[36569]: pgmap v102: 97 pgs: 97 active+clean; 356 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 547 KiB/s wr, 28 op/s 2024-06-06T03:41:59.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:41:58 smithi131 ceph-mon[36687]: pgmap v103: 97 pgs: 97 active+clean; 356 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 538 KiB/s wr, 21 op/s 2024-06-06T03:41:59.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:41:58 smithi040 ceph-mon[32220]: pgmap v103: 97 pgs: 97 active+clean; 356 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 538 KiB/s wr, 21 op/s 2024-06-06T03:41:59.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:41:58 smithi104 ceph-mon[36569]: pgmap v103: 97 pgs: 97 active+clean; 356 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 538 KiB/s wr, 21 op/s 2024-06-06T03:42:01.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:00 smithi131 ceph-mon[36687]: pgmap v104: 97 pgs: 97 active+clean; 365 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-06-06T03:42:01.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:00 smithi040 ceph-mon[32220]: pgmap v104: 97 pgs: 97 active+clean; 365 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-06-06T03:42:01.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:00 smithi104 ceph-mon[36569]: pgmap v104: 97 pgs: 97 active+clean; 365 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-06-06T03:42:03.142 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:02 smithi104 ceph-mon[36569]: pgmap v105: 97 pgs: 97 active+clean; 365 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-06-06T03:42:03.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:02 smithi131 ceph-mon[36687]: pgmap v105: 97 pgs: 97 active+clean; 365 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-06-06T03:42:03.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:02 smithi040 ceph-mon[32220]: pgmap v105: 97 pgs: 97 active+clean; 365 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-06-06T03:42:05.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:04 smithi131 ceph-mon[36687]: pgmap v106: 97 pgs: 97 active+clean; 365 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-06-06T03:42:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:04 smithi040 ceph-mon[32220]: pgmap v106: 97 pgs: 97 active+clean; 365 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-06-06T03:42:05.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:04 smithi104 ceph-mon[36569]: pgmap v106: 97 pgs: 97 active+clean; 365 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-06-06T03:42:07.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:06 smithi131 ceph-mon[36687]: pgmap v107: 97 pgs: 97 active+clean; 370 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-06-06T03:42:07.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:06 smithi040 ceph-mon[32220]: pgmap v107: 97 pgs: 97 active+clean; 370 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-06-06T03:42:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:06 smithi104 ceph-mon[36569]: pgmap v107: 97 pgs: 97 active+clean; 370 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-06-06T03:42:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:08 smithi131 ceph-mon[36687]: pgmap v108: 97 pgs: 97 active+clean; 370 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-06-06T03:42:09.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:08 smithi040 ceph-mon[32220]: pgmap v108: 97 pgs: 97 active+clean; 370 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-06-06T03:42:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:08 smithi104 ceph-mon[36569]: pgmap v108: 97 pgs: 97 active+clean; 370 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-06-06T03:42:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:10 smithi131 ceph-mon[36687]: pgmap v109: 97 pgs: 97 active+clean; 374 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-06-06T03:42:11.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:10 smithi040 ceph-mon[32220]: pgmap v109: 97 pgs: 97 active+clean; 374 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-06-06T03:42:11.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:10 smithi104 ceph-mon[36569]: pgmap v109: 97 pgs: 97 active+clean; 374 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-06-06T03:42:13.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:12 smithi131 ceph-mon[36687]: pgmap v110: 97 pgs: 97 active+clean; 374 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 861 KiB/s wr, 43 op/s 2024-06-06T03:42:13.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:12 smithi040 ceph-mon[32220]: pgmap v110: 97 pgs: 97 active+clean; 374 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 861 KiB/s wr, 43 op/s 2024-06-06T03:42:13.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:12 smithi104 ceph-mon[36569]: pgmap v110: 97 pgs: 97 active+clean; 374 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 861 KiB/s wr, 43 op/s 2024-06-06T03:42:15.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:14 smithi131 ceph-mon[36687]: pgmap v111: 97 pgs: 97 active+clean; 374 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 861 KiB/s wr, 43 op/s 2024-06-06T03:42:15.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:14 smithi040 ceph-mon[32220]: pgmap v111: 97 pgs: 97 active+clean; 374 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 861 KiB/s wr, 43 op/s 2024-06-06T03:42:15.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:14 smithi104 ceph-mon[36569]: pgmap v111: 97 pgs: 97 active+clean; 374 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 861 KiB/s wr, 43 op/s 2024-06-06T03:42:17.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:16 smithi131 ceph-mon[36687]: pgmap v112: 97 pgs: 97 active+clean; 421 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 393 op/s 2024-06-06T03:42:17.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:16 smithi040 ceph-mon[32220]: pgmap v112: 97 pgs: 97 active+clean; 421 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 393 op/s 2024-06-06T03:42:17.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:16 smithi104 ceph-mon[36569]: pgmap v112: 97 pgs: 97 active+clean; 421 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 393 op/s 2024-06-06T03:42:19.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:18 smithi131 ceph-mon[36687]: pgmap v113: 97 pgs: 97 active+clean; 421 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 366 op/s 2024-06-06T03:42:19.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:18 smithi040 ceph-mon[32220]: pgmap v113: 97 pgs: 97 active+clean; 421 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 366 op/s 2024-06-06T03:42:19.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:18 smithi104 ceph-mon[36569]: pgmap v113: 97 pgs: 97 active+clean; 421 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 366 op/s 2024-06-06T03:42:21.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:20 smithi131 ceph-mon[36687]: pgmap v114: 97 pgs: 97 active+clean; 452 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 648 op/s 2024-06-06T03:42:21.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:20 smithi040 ceph-mon[32220]: pgmap v114: 97 pgs: 97 active+clean; 452 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 648 op/s 2024-06-06T03:42:21.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:20 smithi104 ceph-mon[36569]: pgmap v114: 97 pgs: 97 active+clean; 452 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 648 op/s 2024-06-06T03:42:23.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:22 smithi131 ceph-mon[36687]: pgmap v115: 97 pgs: 97 active+clean; 452 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 633 op/s 2024-06-06T03:42:23.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:22 smithi040 ceph-mon[32220]: pgmap v115: 97 pgs: 97 active+clean; 452 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 633 op/s 2024-06-06T03:42:23.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:22 smithi104 ceph-mon[36569]: pgmap v115: 97 pgs: 97 active+clean; 452 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 633 op/s 2024-06-06T03:42:25.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:24 smithi131 ceph-mon[36687]: pgmap v116: 97 pgs: 97 active+clean; 452 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 633 op/s 2024-06-06T03:42:25.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:24 smithi040 ceph-mon[32220]: pgmap v116: 97 pgs: 97 active+clean; 452 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 633 op/s 2024-06-06T03:42:25.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:24 smithi104 ceph-mon[36569]: pgmap v116: 97 pgs: 97 active+clean; 452 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 633 op/s 2024-06-06T03:42:27.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:26 smithi131 ceph-mon[36687]: pgmap v117: 97 pgs: 97 active+clean; 461 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.8 MiB/s wr, 672 op/s 2024-06-06T03:42:27.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:26 smithi040 ceph-mon[32220]: pgmap v117: 97 pgs: 97 active+clean; 461 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.8 MiB/s wr, 672 op/s 2024-06-06T03:42:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:26 smithi104 ceph-mon[36569]: pgmap v117: 97 pgs: 97 active+clean; 461 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.8 MiB/s wr, 672 op/s 2024-06-06T03:42:29.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:28 smithi131 ceph-mon[36687]: pgmap v118: 97 pgs: 97 active+clean; 461 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 321 op/s 2024-06-06T03:42:29.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:28 smithi040 ceph-mon[32220]: pgmap v118: 97 pgs: 97 active+clean; 461 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 321 op/s 2024-06-06T03:42:29.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:28 smithi104 ceph-mon[36569]: pgmap v118: 97 pgs: 97 active+clean; 461 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 321 op/s 2024-06-06T03:42:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:30 smithi131 ceph-mon[36687]: pgmap v119: 97 pgs: 97 active+clean; 467 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 344 op/s 2024-06-06T03:42:31.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:30 smithi040 ceph-mon[32220]: pgmap v119: 97 pgs: 97 active+clean; 467 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 344 op/s 2024-06-06T03:42:31.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:30 smithi104 ceph-mon[36569]: pgmap v119: 97 pgs: 97 active+clean; 467 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 344 op/s 2024-06-06T03:42:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:32 smithi131 ceph-mon[36687]: pgmap v120: 97 pgs: 97 active+clean; 467 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 61 op/s 2024-06-06T03:42:33.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:32 smithi040 ceph-mon[32220]: pgmap v120: 97 pgs: 97 active+clean; 467 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 61 op/s 2024-06-06T03:42:33.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:32 smithi104 ceph-mon[36569]: pgmap v120: 97 pgs: 97 active+clean; 467 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 61 op/s 2024-06-06T03:42:33.891 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-06T03:42:33.891 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.25 2024-06-06T03:42:33.892 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:42:34.161 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:42:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:34 smithi131 ceph-mon[36687]: pgmap v121: 97 pgs: 97 active+clean; 467 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 61 op/s 2024-06-06T03:42:35.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:34 smithi040 ceph-mon[32220]: pgmap v121: 97 pgs: 97 active+clean; 467 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 61 op/s 2024-06-06T03:42:35.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:34 smithi104 ceph-mon[36569]: pgmap v121: 97 pgs: 97 active+clean; 467 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 61 op/s 2024-06-06T03:42:35.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.939+0000 7f988590e700 1 -- 172.21.15.40:0/315990663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880102740 msgr2=0x7f9880102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:42:35.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.939+0000 7f988590e700 1 --2- 172.21.15.40:0/315990663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880102740 0x7f9880102b20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f98700099c0 tx=0x7f9870009cd0 comp rx=0 tx=0).stop 2024-06-06T03:42:35.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.939+0000 7f988590e700 1 -- 172.21.15.40:0/315990663 shutdown_connections 2024-06-06T03:42:35.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.939+0000 7f988590e700 1 --2- 172.21.15.40:0/315990663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9880107840 0x7f988010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.939+0000 7f988590e700 1 --2- 172.21.15.40:0/315990663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880102740 0x7f9880102b20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.939+0000 7f988590e700 1 --2- 172.21.15.40:0/315990663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98801030f0 0x7f9880107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.939+0000 7f988590e700 1 -- 172.21.15.40:0/315990663 >> 172.21.15.40:0/315990663 conn(0x7f98800795f0 msgr2=0x7f9880079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:42:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.940+0000 7f988590e700 1 -- 172.21.15.40:0/315990663 shutdown_connections 2024-06-06T03:42:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.940+0000 7f988590e700 1 -- 172.21.15.40:0/315990663 wait complete. 2024-06-06T03:42:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.940+0000 7f988590e700 1 Processor -- start 2024-06-06T03:42:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.940+0000 7f988590e700 1 -- start start 2024-06-06T03:42:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f988590e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9880102740 0x7f988019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f988590e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98801030f0 0x7f988019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f988590e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880107840 0x7f9880198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f988590e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9880112780 con 0x7f9880107840 2024-06-06T03:42:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f988590e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9880112600 con 0x7f98801030f0 2024-06-06T03:42:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f988590e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9880112900 con 0x7f9880102740 2024-06-06T03:42:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f987f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880107840 0x7f9880198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f987f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880107840 0x7f9880198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34690/0 (socket says 172.21.15.40:34690) 2024-06-06T03:42:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f98767fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98801030f0 0x7f988019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f987f7fe700 1 -- 172.21.15.40:0/2037340415 learned_addr learned my addr 172.21.15.40:0/2037340415 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:42:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f987effd700 1 --2- 172.21.15.40:0/2037340415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9880102740 0x7f988019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:35.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f987f7fe700 1 -- 172.21.15.40:0/2037340415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9880102740 msgr2=0x7f988019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:42:35.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f987f7fe700 1 --2- 172.21.15.40:0/2037340415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9880102740 0x7f988019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:35.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f987f7fe700 1 -- 172.21.15.40:0/2037340415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98801030f0 msgr2=0x7f988019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:42:35.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f987f7fe700 1 --2- 172.21.15.40:0/2037340415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98801030f0 0x7f988019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:35.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f987f7fe700 1 -- 172.21.15.40:0/2037340415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98801991a0 con 0x7f9880107840 2024-06-06T03:42:35.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f98767fc700 1 --2- 172.21.15.40:0/2037340415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98801030f0 0x7f988019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:42:35.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f987f7fe700 1 --2- 172.21.15.40:0/2037340415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880107840 0x7f9880198a40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f987800ef30 tx=0x7f987800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:42:35.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.941+0000 7f988490c700 1 -- 172.21.15.40:0/2037340415 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98780092e0 con 0x7f9880107840 2024-06-06T03:42:35.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.942+0000 7f988590e700 1 -- 172.21.15.40:0/2037340415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9880199430 con 0x7f9880107840 2024-06-06T03:42:35.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.942+0000 7f988490c700 1 -- 172.21.15.40:0/2037340415 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9878004bb0 con 0x7f9880107840 2024-06-06T03:42:35.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.942+0000 7f988490c700 1 -- 172.21.15.40:0/2037340415 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878004280 con 0x7f9880107840 2024-06-06T03:42:35.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.942+0000 7f988590e700 1 -- 172.21.15.40:0/2037340415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98801a5700 con 0x7f9880107840 2024-06-06T03:42:35.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.943+0000 7f988590e700 1 -- 172.21.15.40:0/2037340415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f988010aa80 con 0x7f9880107840 2024-06-06T03:42:35.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.944+0000 7f988490c700 1 -- 172.21.15.40:0/2037340415 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f987801b030 con 0x7f9880107840 2024-06-06T03:42:35.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.944+0000 7f988490c700 1 --2- 172.21.15.40:0/2037340415 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f986c0a2d90 0x7f986c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.944+0000 7f987effd700 1 --2- 172.21.15.40:0/2037340415 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f986c0a2d90 0x7f986c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.944+0000 7f988490c700 1 -- 172.21.15.40:0/2037340415 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f98780ceea0 con 0x7f9880107840 2024-06-06T03:42:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.945+0000 7f987effd700 1 --2- 172.21.15.40:0/2037340415 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f986c0a2d90 0x7f986c0a5250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9870000c00 tx=0x7f9870005fb0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:42:35.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:35.949+0000 7f988490c700 1 -- 172.21.15.40:0/2037340415 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9878022020 con 0x7f9880107840 2024-06-06T03:42:36.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:36.200+0000 7f988590e700 1 -- 172.21.15.40:0/2037340415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9880199f20 con 0x7f9880107840 2024-06-06T03:42:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:36.201+0000 7f988490c700 1 -- 172.21.15.40:0/2037340415 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11945 (secure 0 0 0) 0x7f9878010040 con 0x7f9880107840 2024-06-06T03:42:36.201 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:42:36.201 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:41:20.463469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24443,24470,24425,24403,24398]},"id":1}]} 2024-06-06T03:42:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:36.203+0000 7f988590e700 1 -- 172.21.15.40:0/2037340415 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f986c0a2d90 msgr2=0x7f986c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:42:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:36.203+0000 7f988590e700 1 --2- 172.21.15.40:0/2037340415 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f986c0a2d90 0x7f986c0a5250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9870000c00 tx=0x7f9870005fb0 comp rx=0 tx=0).stop 2024-06-06T03:42:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:36.203+0000 7f988590e700 1 -- 172.21.15.40:0/2037340415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880107840 msgr2=0x7f9880198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:42:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:36.203+0000 7f988590e700 1 --2- 172.21.15.40:0/2037340415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880107840 0x7f9880198a40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f987800ef30 tx=0x7f987800c3a0 comp rx=0 tx=0).stop 2024-06-06T03:42:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:36.203+0000 7f988590e700 1 -- 172.21.15.40:0/2037340415 shutdown_connections 2024-06-06T03:42:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:36.203+0000 7f988590e700 1 --2- 172.21.15.40:0/2037340415 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f986c0a2d90 0x7f986c0a5250 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:36.203+0000 7f988590e700 1 --2- 172.21.15.40:0/2037340415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9880102740 0x7f988019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:36.203+0000 7f988590e700 1 --2- 172.21.15.40:0/2037340415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880107840 0x7f9880198a40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:36.203+0000 7f988590e700 1 --2- 172.21.15.40:0/2037340415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98801030f0 0x7f988019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:36.203+0000 7f988590e700 1 -- 172.21.15.40:0/2037340415 >> 172.21.15.40:0/2037340415 conn(0x7f98800795f0 msgr2=0x7f9880108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:42:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:36.203+0000 7f988590e700 1 -- 172.21.15.40:0/2037340415 shutdown_connections 2024-06-06T03:42:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:36.203+0000 7f988590e700 1 -- 172.21.15.40:0/2037340415 wait complete. 2024-06-06T03:42:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 35 2024-06-06T03:42:36.961 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0' 2024-06-06T03:42:36.962 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0 2024-06-06T03:42:37.237 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:42:37.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:36 smithi131 ceph-mon[36687]: pgmap v122: 97 pgs: 97 active+clean; 642 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 700 op/s 2024-06-06T03:42:37.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:36 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2037340415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:42:37.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:36 smithi040 ceph-mon[32220]: pgmap v122: 97 pgs: 97 active+clean; 642 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 700 op/s 2024-06-06T03:42:37.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:36 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2037340415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:42:37.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:36 smithi104 ceph-mon[36569]: pgmap v122: 97 pgs: 97 active+clean; 642 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 700 op/s 2024-06-06T03:42:37.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:36 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2037340415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:42:39.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:38 smithi131 ceph-mon[36687]: pgmap v123: 97 pgs: 97 active+clean; 642 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 15 MiB/s wr, 661 op/s 2024-06-06T03:42:39.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:38 smithi040 ceph-mon[32220]: pgmap v123: 97 pgs: 97 active+clean; 642 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 15 MiB/s wr, 661 op/s 2024-06-06T03:42:39.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:38 smithi104 ceph-mon[36569]: pgmap v123: 97 pgs: 97 active+clean; 642 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 15 MiB/s wr, 661 op/s 2024-06-06T03:42:40.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.090+0000 7f18a2723700 1 -- 172.21.15.40:0/3382876231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c1027a0 msgr2=0x7f189c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:42:40.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.090+0000 7f18a2723700 1 --2- 172.21.15.40:0/3382876231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c1027a0 0x7f189c102b80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f188c0099c0 tx=0x7f188c009cd0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.090+0000 7f18a2723700 1 -- 172.21.15.40:0/3382876231 shutdown_connections 2024-06-06T03:42:40.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.090+0000 7f18a2723700 1 --2- 172.21.15.40:0/3382876231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c107790 0x7f189c10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.090+0000 7f18a2723700 1 --2- 172.21.15.40:0/3382876231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c1027a0 0x7f189c102b80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.090+0000 7f18a2723700 1 --2- 172.21.15.40:0/3382876231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f189c103150 0x7f189c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.090+0000 7f18a2723700 1 -- 172.21.15.40:0/3382876231 >> 172.21.15.40:0/3382876231 conn(0x7f189c079560 msgr2=0x7f189c079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:42:40.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.090+0000 7f18a2723700 1 -- 172.21.15.40:0/3382876231 shutdown_connections 2024-06-06T03:42:40.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.090+0000 7f18a2723700 1 -- 172.21.15.40:0/3382876231 wait complete. 2024-06-06T03:42:40.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.091+0000 7f18a2723700 1 Processor -- start 2024-06-06T03:42:40.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.091+0000 7f18a2723700 1 -- start start 2024-06-06T03:42:40.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.091+0000 7f18a2723700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c1027a0 0x7f189c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.091+0000 7f18a2723700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c103150 0x7f189c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.091+0000 7f18a2723700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f189c107790 0x7f189c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.091+0000 7f18a2723700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1127b0 con 0x7f189c103150 2024-06-06T03:42:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.091+0000 7f18a2723700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f189c112630 con 0x7f189c1027a0 2024-06-06T03:42:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.091+0000 7f18a2723700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f189c112930 con 0x7f189c107790 2024-06-06T03:42:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.091+0000 7f189bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c1027a0 0x7f189c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.091+0000 7f189bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c1027a0 0x7f189c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:43114/0 (socket says 172.21.15.40:43114) 2024-06-06T03:42:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.092+0000 7f189bfff700 1 -- 172.21.15.40:0/2440818791 learned_addr learned my addr 172.21.15.40:0/2440818791 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:42:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.091+0000 7f189b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c103150 0x7f189c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.092+0000 7f18a0cc0700 1 --2- 172.21.15.40:0/2440818791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f189c107790 0x7f189c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.092+0000 7f189bfff700 1 -- 172.21.15.40:0/2440818791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f189c107790 msgr2=0x7f189c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:42:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.092+0000 7f189bfff700 1 --2- 172.21.15.40:0/2440818791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f189c107790 0x7f189c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.092+0000 7f189bfff700 1 -- 172.21.15.40:0/2440818791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c103150 msgr2=0x7f189c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:42:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.092+0000 7f189bfff700 1 --2- 172.21.15.40:0/2440818791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c103150 0x7f189c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.092+0000 7f189bfff700 1 -- 172.21.15.40:0/2440818791 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c1a70f0 con 0x7f189c1027a0 2024-06-06T03:42:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.092+0000 7f189bfff700 1 --2- 172.21.15.40:0/2440818791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c1027a0 0x7f189c074790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f188c006010 tx=0x7f188c009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:42:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.092+0000 7f18997fa700 1 -- 172.21.15.40:0/2440818791 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c026070 con 0x7f189c1027a0 2024-06-06T03:42:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.092+0000 7f18997fa700 1 -- 172.21.15.40:0/2440818791 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f188c004cd0 con 0x7f189c1027a0 2024-06-06T03:42:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.092+0000 7f18a2723700 1 -- 172.21.15.40:0/2440818791 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c1a7380 con 0x7f189c1027a0 2024-06-06T03:42:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.092+0000 7f18997fa700 1 -- 172.21.15.40:0/2440818791 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c003ba0 con 0x7f189c1027a0 2024-06-06T03:42:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.092+0000 7f18a2723700 1 -- 172.21.15.40:0/2440818791 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c1a77d0 con 0x7f189c1027a0 2024-06-06T03:42:40.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.095+0000 7f18997fa700 1 -- 172.21.15.40:0/2440818791 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f188c029070 con 0x7f189c1027a0 2024-06-06T03:42:40.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.095+0000 7f18997fa700 1 --2- 172.21.15.40:0/2440818791 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f18840a2e30 0x7f18840a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.096+0000 7f189b7fe700 1 --2- 172.21.15.40:0/2440818791 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f18840a2e30 0x7f18840a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.096+0000 7f18997fa700 1 -- 172.21.15.40:0/2440818791 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f188c0cfd00 con 0x7f189c1027a0 2024-06-06T03:42:40.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.096+0000 7f189b7fe700 1 --2- 172.21.15.40:0/2440818791 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f18840a2e30 0x7f18840a52f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f189c100d20 tx=0x7f189000b410 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:42:40.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.096+0000 7f18acfa6b80 1 Processor -- start 2024-06-06T03:42:40.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.096+0000 7f18acfa6b80 1 -- start start 2024-06-06T03:42:40.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.096+0000 7f18acfa6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 0x562c9c3cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.096+0000 7f18acfa6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.096+0000 7f18acfa6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3c7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.096+0000 7f18acfa6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562c9c2b9100 con 0x562c9c3c6cf0 2024-06-06T03:42:40.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.096+0000 7f18acfa6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562c9c3179f0 con 0x562c9c3ccec0 2024-06-06T03:42:40.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.096+0000 7f18acfa6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562c9c27f600 con 0x562c9c3cc5a0 2024-06-06T03:42:40.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.096+0000 7f18a0cc0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.096+0000 7f189b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.097+0000 7f18a0cc0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3c7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46520/0 (socket says 172.21.15.40:46520) 2024-06-06T03:42:40.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.097+0000 7f189b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3c67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:43120/0 (socket says 172.21.15.40:43120) 2024-06-06T03:42:40.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.097+0000 7f18a0cc0700 1 -- 172.21.15.40:0/259444275 learned_addr learned my addr 172.21.15.40:0/259444275 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:42:40.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.097+0000 7f189bfff700 1 --2- 172.21.15.40:0/259444275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 0x562c9c3cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.097+0000 7f189b7fe700 1 -- 172.21.15.40:0/259444275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 msgr2=0x562c9c3cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:42:40.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.097+0000 7f189b7fe700 1 --2- 172.21.15.40:0/259444275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 0x562c9c3cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.097+0000 7f189b7fe700 1 -- 172.21.15.40:0/259444275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 msgr2=0x562c9c3c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:42:40.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.097+0000 7f189b7fe700 1 --2- 172.21.15.40:0/259444275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.097+0000 7f189b7fe700 1 -- 172.21.15.40:0/259444275 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c9c266a70 con 0x562c9c3ccec0 2024-06-06T03:42:40.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.097+0000 7f18a0cc0700 1 --2- 172.21.15.40:0/259444275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:42:40.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.097+0000 7f189b7fe700 1 --2- 172.21.15.40:0/259444275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3c67b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1890005d90 tx=0x7f1890013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5637841d54e56ec server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:42:40.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.097+0000 7f18827fc700 1 -- 172.21.15.40:0/259444275 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890011b80 con 0x562c9c3ccec0 2024-06-06T03:42:40.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.097+0000 7f18827fc700 1 -- 172.21.15.40:0/259444275 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1890011d20 con 0x562c9c3ccec0 2024-06-06T03:42:40.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.097+0000 7f18827fc700 1 -- 172.21.15.40:0/259444275 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189001e420 con 0x562c9c3ccec0 2024-06-06T03:42:40.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.098+0000 7f18acfa6b80 1 -- 172.21.15.40:0/259444275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 msgr2=0x562c9c3c67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:42:40.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.098+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/259444275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3c67b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1890005d90 tx=0x7f1890013000 comp rx=0 tx=0).stop 2024-06-06T03:42:40.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.098+0000 7f18acfa6b80 1 -- 172.21.15.40:0/259444275 shutdown_connections 2024-06-06T03:42:40.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.098+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/259444275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 0x562c9c3cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.098+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/259444275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.098+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/259444275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3c67b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.098+0000 7f18acfa6b80 1 -- 172.21.15.40:0/259444275 >> 172.21.15.40:0/259444275 conn(0x562c9c2f3220 msgr2=0x562c9c3c1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:42:40.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.098+0000 7f18acfa6b80 1 -- 172.21.15.40:0/259444275 shutdown_connections 2024-06-06T03:42:40.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.098+0000 7f18acfa6b80 1 -- 172.21.15.40:0/259444275 wait complete. 2024-06-06T03:42:40.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.098+0000 7f18acfa6b80 1 -- start start 2024-06-06T03:42:40.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f18acfa6b80 1 Processor -- start 2024-06-06T03:42:40.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f18acfa6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f18acfa6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 0x562c9c3cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f18acfa6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f18acfa6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562c9c2b8160 con 0x562c9c3c6cf0 2024-06-06T03:42:40.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f18acfa6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562c9c3139b0 con 0x562c9c3ccec0 2024-06-06T03:42:40.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f18acfa6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562c9c3d38a0 con 0x562c9c3cc5a0 2024-06-06T03:42:40.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f18acfa6b80 10 client.0 _ll_register_callbacks cb 0x562c9c2ad750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:42:40.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f189bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f18a0cc0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f189bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46536/0 (socket says 172.21.15.40:46536) 2024-06-06T03:42:40.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f189bfff700 1 -- 172.21.15.40:0/2630678630 learned_addr learned my addr 172.21.15.40:0/2630678630 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:42:40.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f189b7fe700 1 --2- 172.21.15.40:0/2630678630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 0x562c9c3cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f18a0cc0700 1 -- 172.21.15.40:0/2630678630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 msgr2=0x562c9c3cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:42:40.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f18a0cc0700 1 --2- 172.21.15.40:0/2630678630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 0x562c9c3cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.099+0000 7f18a0cc0700 1 -- 172.21.15.40:0/2630678630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 msgr2=0x562c9c3d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:42:40.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.100+0000 7f18a0cc0700 1 --2- 172.21.15.40:0/2630678630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.100+0000 7f18a0cc0700 1 -- 172.21.15.40:0/2630678630 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c9c2dd8f0 con 0x562c9c3ccec0 2024-06-06T03:42:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.100+0000 7f189bfff700 1 --2- 172.21.15.40:0/2630678630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:42:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.100+0000 7f18a0cc0700 1 --2- 172.21.15.40:0/2630678630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3d04b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1894004070 tx=0x7f18940040a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da01e8617e7a80cb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:42:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.100+0000 7f187b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-06-06T03:42:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.100+0000 7f187b7fe700 1 -- 172.21.15.40:0/2630678630 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189401a070 con 0x562c9c3ccec0 2024-06-06T03:42:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.100+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2630678630 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1890011d20 con 0x562c9c3ccec0 2024-06-06T03:42:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.100+0000 7f187b7fe700 1 -- 172.21.15.40:0/2630678630 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f1894004db0 con 0x562c9c3ccec0 2024-06-06T03:42:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.100+0000 7f187b7fe700 1 -- 172.21.15.40:0/2630678630 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1894025050 con 0x562c9c3ccec0 2024-06-06T03:42:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.100+0000 7f18acfa6b80 10 client.25223 fetch_fsmap learned FSMap version 35 2024-06-06T03:42:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.100+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2630678630 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562c9c3de040 con 0x562c9c3ccec0 2024-06-06T03:42:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.100+0000 7f187b7fe700 1 -- 172.21.15.40:0/2630678630 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1890011d20 con 0x562c9c3ccec0 2024-06-06T03:42:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.100+0000 7f187b7fe700 1 -- 172.21.15.40:0/2630678630 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f18940165f0 con 0x562c9c3ccec0 2024-06-06T03:42:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.101+0000 7f18acfa6b80 10 client.25223 fetch_fsmap finished waiting for FSMap version 35 2024-06-06T03:42:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.101+0000 7f18acfa6b80 10 client.25223 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:42:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.101+0000 7f18acfa6b80 20 client.25223 populate_metadata read hostname 'smithi040' 2024-06-06T03:42:40.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.101+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/2630678630 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x562c9c3e06d0 0x562c9c3e2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.101+0000 7f18acfa6b80 4 client.25223 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:42:40.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.101+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2630678630 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c9c324ef0 con 0x562c9c3e06d0 2024-06-06T03:42:40.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.101+0000 7f189bfff700 1 --2- 172.21.15.40:0/2630678630 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x562c9c3e06d0 0x562c9c3e2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.102+0000 7f189bfff700 1 --2- 172.21.15.40:0/2630678630 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x562c9c3e06d0 0x562c9c3e2b10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2229dfadcd9116fe server_cookie=9614e5cd13a4b9bf in_seq=0 out_seq=0 2024-06-06T03:42:40.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.102+0000 7f187b7fe700 10 client.25223 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:42:40.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.104+0000 7f187b7fe700 1 -- 172.21.15.40:0/2630678630 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562c9c324ef0 con 0x562c9c3e06d0 2024-06-06T03:42:40.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.104+0000 7f187b7fe700 10 client.25223 handle_command_reply: tid=0 2024-06-06T03:42:40.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.104+0000 7f18acfa6b80 1 client.25223 shutdown 2024-06-06T03:42:40.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.104+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2630678630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 msgr2=0x562c9c3d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:42:40.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.104+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/2630678630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3d04b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1894004070 tx=0x7f18940040a0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.104+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2630678630 shutdown_connections 2024-06-06T03:42:40.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.104+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/2630678630 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x562c9c3e06d0 0x562c9c3e2b10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.104+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/2630678630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3d04b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.104+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/2630678630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.104+0000 7f187b7fe700 0 client.25223 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:42:40.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.104+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/2630678630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 0x562c9c3cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.104+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2630678630 >> 172.21.15.40:0/2630678630 conn(0x562c9c2f3220 msgr2=0x562c9c3c0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:42:40.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.105+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2630678630 shutdown_connections 2024-06-06T03:42:40.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.105+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2630678630 wait complete. 2024-06-06T03:42:40.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.105+0000 7f18acfa6b80 20 client.25223 trim_cache size 0 max 16384 2024-06-06T03:42:40.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.125+0000 7f18acfa6b80 1 Processor -- start 2024-06-06T03:42:40.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.125+0000 7f18acfa6b80 1 -- start start 2024-06-06T03:42:40.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.125+0000 7f18acfa6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.125+0000 7f18acfa6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3cc5a0 0x562c9c3c13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.125+0000 7f18acfa6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3ccec0 0x562c9c3c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.125+0000 7f18acfa6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1aed10 con 0x562c9c3c6cf0 2024-06-06T03:42:40.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.125+0000 7f18acfa6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b09a0 con 0x562c9c3cc5a0 2024-06-06T03:42:40.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.125+0000 7f18acfa6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1af3e0 con 0x562c9c3ccec0 2024-06-06T03:42:40.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.126+0000 7f189b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3cc5a0 0x562c9c3c13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.126+0000 7f189b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3cc5a0 0x562c9c3c13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:43142/0 (socket says 172.21.15.40:43142) 2024-06-06T03:42:40.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.126+0000 7f189b7fe700 1 -- 172.21.15.40:0/551227659 learned_addr learned my addr 172.21.15.40:0/551227659 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:42:40.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.126+0000 7f189bfff700 1 --2- 172.21.15.40:0/551227659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.126+0000 7f18a0cc0700 1 --2- 172.21.15.40:0/551227659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3ccec0 0x562c9c3c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.126+0000 7f189b7fe700 1 -- 172.21.15.40:0/551227659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3ccec0 msgr2=0x562c9c3c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:42:40.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.126+0000 7f189b7fe700 1 --2- 172.21.15.40:0/551227659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3ccec0 0x562c9c3c1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.126+0000 7f189b7fe700 1 -- 172.21.15.40:0/551227659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 msgr2=0x562c9c3d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:42:40.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.126+0000 7f189b7fe700 1 --2- 172.21.15.40:0/551227659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3d27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.126+0000 7f189b7fe700 1 -- 172.21.15.40:0/551227659 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c9c3ce4e0 con 0x562c9c3cc5a0 2024-06-06T03:42:40.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.126+0000 7f189b7fe700 1 --2- 172.21.15.40:0/551227659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3cc5a0 0x562c9c3c13d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1890005d90 tx=0x7f189000bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=263a44215db53f0c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:42:40.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.126+0000 7f1881ffb700 1 -- 172.21.15.40:0/551227659 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890011e40 con 0x562c9c3cc5a0 2024-06-06T03:42:40.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.126+0000 7f1881ffb700 1 -- 172.21.15.40:0/551227659 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1890035070 con 0x562c9c3cc5a0 2024-06-06T03:42:40.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.126+0000 7f1881ffb700 1 -- 172.21.15.40:0/551227659 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890026db0 con 0x562c9c3cc5a0 2024-06-06T03:42:40.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.127+0000 7f18acfa6b80 1 -- 172.21.15.40:0/551227659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3cc5a0 msgr2=0x562c9c3c13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:42:40.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.127+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/551227659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3cc5a0 0x562c9c3c13d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1890005d90 tx=0x7f189000bb10 comp rx=0 tx=0).stop 2024-06-06T03:42:40.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.127+0000 7f18acfa6b80 1 -- 172.21.15.40:0/551227659 shutdown_connections 2024-06-06T03:42:40.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.127+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/551227659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3ccec0 0x562c9c3c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.127+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/551227659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3d27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.127+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/551227659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3cc5a0 0x562c9c3c13d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.127+0000 7f18acfa6b80 1 -- 172.21.15.40:0/551227659 >> 172.21.15.40:0/551227659 conn(0x562c9c2f3220 msgr2=0x562c9c3cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:42:40.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.127+0000 7f18acfa6b80 1 -- 172.21.15.40:0/551227659 shutdown_connections 2024-06-06T03:42:40.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.127+0000 7f18acfa6b80 1 -- 172.21.15.40:0/551227659 wait complete. 2024-06-06T03:42:40.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.127+0000 7f18acfa6b80 1 -- start start 2024-06-06T03:42:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.128+0000 7f18acfa6b80 1 Processor -- start 2024-06-06T03:42:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.128+0000 7f18acfa6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3d8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.128+0000 7f18acfa6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 0x562c9c3d90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.128+0000 7f18acfa6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3d5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.128+0000 7f18acfa6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b0210 con 0x562c9c3c6cf0 2024-06-06T03:42:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.128+0000 7f18acfa6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1afab0 con 0x562c9c3ccec0 2024-06-06T03:42:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.128+0000 7f18acfa6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1acd10 con 0x562c9c3cc5a0 2024-06-06T03:42:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.128+0000 7f18acfa6b80 10 client.0 _ll_register_callbacks cb 0x562c9c3caba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:42:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.128+0000 7f18a0cc0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.128+0000 7f189bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3d8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.128+0000 7f18a0cc0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3d5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:43144/0 (socket says 172.21.15.40:43144) 2024-06-06T03:42:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.128+0000 7f18a0cc0700 1 -- 172.21.15.40:0/2417859551 learned_addr learned my addr 172.21.15.40:0/2417859551 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:42:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.128+0000 7f189b7fe700 1 --2- 172.21.15.40:0/2417859551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 0x562c9c3d90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.129+0000 7f18a0cc0700 1 -- 172.21.15.40:0/2417859551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 msgr2=0x562c9c3d90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.129+0000 7f18a0cc0700 1 --2- 172.21.15.40:0/2417859551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 0x562c9c3d90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.129+0000 7f18a0cc0700 1 -- 172.21.15.40:0/2417859551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 msgr2=0x562c9c3d8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.129+0000 7f18a0cc0700 1 --2- 172.21.15.40:0/2417859551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3d8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.129+0000 7f18a0cc0700 1 -- 172.21.15.40:0/2417859551 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c9c3d61a0 con 0x562c9c3ccec0 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.129+0000 7f18a0cc0700 1 --2- 172.21.15.40:0/2417859551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3d5b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1894016bb0 tx=0x7f1894005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d23a0a947839d73 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.129+0000 7f187b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.129+0000 7f187b7fe700 1 -- 172.21.15.40:0/2417859551 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189401a070 con 0x562c9c3ccec0 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.129+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2417859551 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1890035070 con 0x562c9c3ccec0 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.129+0000 7f187b7fe700 1 -- 172.21.15.40:0/2417859551 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f1894004920 con 0x562c9c3ccec0 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.129+0000 7f187b7fe700 1 -- 172.21.15.40:0/2417859551 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1894004b20 con 0x562c9c3ccec0 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.129+0000 7f18acfa6b80 10 client.25229 fetch_fsmap learned FSMap version 35 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.129+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2417859551 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562c9c44bce0 con 0x562c9c3ccec0 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.129+0000 7f187b7fe700 1 -- 172.21.15.40:0/2417859551 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1890035070 con 0x562c9c3ccec0 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.130+0000 7f187b7fe700 1 -- 172.21.15.40:0/2417859551 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f1894005040 con 0x562c9c3ccec0 2024-06-06T03:42:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.130+0000 7f18acfa6b80 10 client.25229 fetch_fsmap finished waiting for FSMap version 35 2024-06-06T03:42:40.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.130+0000 7f18acfa6b80 10 client.25229 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:42:40.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.130+0000 7f18acfa6b80 20 client.25229 populate_metadata read hostname 'smithi040' 2024-06-06T03:42:40.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.130+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/2417859551 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x562c9c44c510 0x562c9c44e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:42:40.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.130+0000 7f18acfa6b80 4 client.25229 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}] 2024-06-06T03:42:40.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.130+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2417859551 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}) -- 0x562c9c31f470 con 0x562c9c44c510 2024-06-06T03:42:40.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.130+0000 7f189bfff700 1 --2- 172.21.15.40:0/2417859551 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x562c9c44c510 0x562c9c44e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:42:40.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.130+0000 7f189bfff700 1 --2- 172.21.15.40:0/2417859551 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x562c9c44c510 0x562c9c44e930 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9df1e89c722f246 server_cookie=55c2a49fe482d4d1 in_seq=0 out_seq=0 2024-06-06T03:42:40.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:40.130+0000 7f187b7fe700 10 client.25229 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:42:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:40 smithi131 ceph-mon[36687]: pgmap v124: 97 pgs: 97 active+clean; 658 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 17 MiB/s wr, 692 op/s 2024-06-06T03:42:41.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:40 smithi040 ceph-mon[32220]: pgmap v124: 97 pgs: 97 active+clean; 658 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 17 MiB/s wr, 692 op/s 2024-06-06T03:42:41.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:40 smithi104 ceph-mon[36569]: pgmap v124: 97 pgs: 97 active+clean; 658 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 17 MiB/s wr, 692 op/s 2024-06-06T03:42:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:42 smithi131 ceph-mon[36687]: pgmap v125: 97 pgs: 97 active+clean; 658 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 669 op/s 2024-06-06T03:42:43.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:42 smithi040 ceph-mon[32220]: pgmap v125: 97 pgs: 97 active+clean; 658 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 669 op/s 2024-06-06T03:42:43.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:42 smithi104 ceph-mon[36569]: pgmap v125: 97 pgs: 97 active+clean; 658 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 669 op/s 2024-06-06T03:42:45.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:44 smithi131 ceph-mon[36687]: pgmap v126: 97 pgs: 97 active+clean; 658 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 669 op/s 2024-06-06T03:42:45.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:44 smithi040 ceph-mon[32220]: pgmap v126: 97 pgs: 97 active+clean; 658 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 669 op/s 2024-06-06T03:42:45.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:44 smithi104 ceph-mon[36569]: pgmap v126: 97 pgs: 97 active+clean; 658 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 669 op/s 2024-06-06T03:42:46.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:45 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:42:46.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:45 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:42:46.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:45 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:42:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:46 smithi131 ceph-mon[36687]: pgmap v127: 97 pgs: 97 active+clean; 663 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 17 MiB/s wr, 736 op/s 2024-06-06T03:42:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:46 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:42:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:46 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:42:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:46 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:42:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:46 smithi040 ceph-mon[32220]: pgmap v127: 97 pgs: 97 active+clean; 663 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 17 MiB/s wr, 736 op/s 2024-06-06T03:42:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:46 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:42:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:46 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:42:47.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:46 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:42:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:46 smithi104 ceph-mon[36569]: pgmap v127: 97 pgs: 97 active+clean; 663 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 17 MiB/s wr, 736 op/s 2024-06-06T03:42:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:46 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:42:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:46 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:42:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:46 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:42:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:48 smithi131 ceph-mon[36687]: pgmap v128: 97 pgs: 97 active+clean; 663 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-06-06T03:42:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:48 smithi040 ceph-mon[32220]: pgmap v128: 97 pgs: 97 active+clean; 663 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-06-06T03:42:49.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:48 smithi104 ceph-mon[36569]: pgmap v128: 97 pgs: 97 active+clean; 663 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-06-06T03:42:51.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:50 smithi131 ceph-mon[36687]: pgmap v129: 97 pgs: 97 active+clean; 685 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 347 op/s 2024-06-06T03:42:51.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:50 smithi040 ceph-mon[32220]: pgmap v129: 97 pgs: 97 active+clean; 685 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 347 op/s 2024-06-06T03:42:51.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:50 smithi104 ceph-mon[36569]: pgmap v129: 97 pgs: 97 active+clean; 685 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 347 op/s 2024-06-06T03:42:53.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:52 smithi131 ceph-mon[36687]: pgmap v130: 97 pgs: 97 active+clean; 685 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-06-06T03:42:53.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:52 smithi040 ceph-mon[32220]: pgmap v130: 97 pgs: 97 active+clean; 685 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-06-06T03:42:53.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:52 smithi104 ceph-mon[36569]: pgmap v130: 97 pgs: 97 active+clean; 685 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-06-06T03:42:55.120 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:42:55.120 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 35, 2024-06-06T03:42:55.120 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:42:55.120 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 10, 2024-06-06T03:42:55.120 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": { 2024-06-06T03:42:55.120 INFO:teuthology.orchestra.run.smithi040.stdout: "699b8c5b": { 2024-06-06T03:42:55.120 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 10, 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: "age_ref": 0.0, 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "QUIESCED", 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: "timeout": 90.0, 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: "expiration": 145.2, 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: "members": { 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: "file:/": { 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: "excluded": false, 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "QUIESCED", 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:42:55.121 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:42:55.122 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:42:55.122 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:42:55.122 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:42:55.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f187b7fe700 1 -- 172.21.15.40:0/2417859551 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562c9c31f470 con 0x562c9c44c510 2024-06-06T03:42:55.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f187b7fe700 10 client.25229 handle_command_reply: tid=0 2024-06-06T03:42:55.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f18acfa6b80 1 client.25229 shutdown 2024-06-06T03:42:55.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2417859551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 msgr2=0x562c9c3d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:42:55.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/2417859551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3d5b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1894016bb0 tx=0x7f1894005f10 comp rx=0 tx=0).stop 2024-06-06T03:42:55.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2417859551 shutdown_connections 2024-06-06T03:42:55.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/2417859551 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x562c9c44c510 0x562c9c44e930 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:55.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/2417859551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562c9c3ccec0 0x562c9c3d5b30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:55.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/2417859551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562c9c3c6cf0 0x562c9c3d8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:55.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f187b7fe700 0 client.25229 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:42:55.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f18acfa6b80 1 --2- 172.21.15.40:0/2417859551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562c9c3cc5a0 0x562c9c3d90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:55.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2417859551 >> 172.21.15.40:0/2417859551 conn(0x562c9c2f3220 msgr2=0x562c9c3c57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:42:55.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2417859551 shutdown_connections 2024-06-06T03:42:55.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f18acfa6b80 1 -- 172.21.15.40:0/2417859551 wait complete. 2024-06-06T03:42:55.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.119+0000 7f18acfa6b80 20 client.25229 trim_cache size 0 max 16384 2024-06-06T03:42:55.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.120+0000 7f18a2723700 1 -- 172.21.15.40:0/2440818791 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f18840a2e30 msgr2=0x7f18840a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:42:55.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.120+0000 7f18a2723700 1 --2- 172.21.15.40:0/2440818791 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f18840a2e30 0x7f18840a52f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f189c100d20 tx=0x7f189000b410 comp rx=0 tx=0).stop 2024-06-06T03:42:55.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.120+0000 7f18a2723700 1 -- 172.21.15.40:0/2440818791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c1027a0 msgr2=0x7f189c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:42:55.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.120+0000 7f18a2723700 1 --2- 172.21.15.40:0/2440818791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c1027a0 0x7f189c074790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f188c006010 tx=0x7f188c009f80 comp rx=0 tx=0).stop 2024-06-06T03:42:55.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.120+0000 7f18a2723700 1 -- 172.21.15.40:0/2440818791 shutdown_connections 2024-06-06T03:42:55.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.120+0000 7f18a2723700 1 --2- 172.21.15.40:0/2440818791 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f18840a2e30 0x7f18840a52f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:55.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.120+0000 7f18a2723700 1 --2- 172.21.15.40:0/2440818791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f189c107790 0x7f189c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:55.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.120+0000 7f18a2723700 1 --2- 172.21.15.40:0/2440818791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c1027a0 0x7f189c074790 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:55.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.120+0000 7f18a2723700 1 --2- 172.21.15.40:0/2440818791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f189c103150 0x7f189c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:42:55.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.120+0000 7f18a2723700 1 -- 172.21.15.40:0/2440818791 >> 172.21.15.40:0/2440818791 conn(0x7f189c079560 msgr2=0x7f189c101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:42:55.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.120+0000 7f18a2723700 1 -- 172.21.15.40:0/2440818791 shutdown_connections 2024-06-06T03:42:55.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:42:55.120+0000 7f18a2723700 1 -- 172.21.15.40:0/2440818791 wait complete. 2024-06-06T03:42:55.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:54 smithi131 ceph-mon[36687]: pgmap v131: 97 pgs: 97 active+clean; 685 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-06-06T03:42:55.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:54 smithi040 ceph-mon[32220]: pgmap v131: 97 pgs: 97 active+clean; 685 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-06-06T03:42:55.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:54 smithi104 ceph-mon[36569]: pgmap v131: 97 pgs: 97 active+clean; 685 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-06-06T03:42:56.118 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '699b8c5b', quiesce took 22.2 seconds. Will release after: 3.0500000000000007 2024-06-06T03:42:57.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:56 smithi131 ceph-mon[36687]: pgmap v132: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 33 MiB/s wr, 654 op/s 2024-06-06T03:42:57.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:56 smithi040 ceph-mon[32220]: pgmap v132: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 33 MiB/s wr, 654 op/s 2024-06-06T03:42:57.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:56 smithi104 ceph-mon[36569]: pgmap v132: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 33 MiB/s wr, 654 op/s 2024-06-06T03:42:59.170 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:42:59.199 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:42:58 smithi040 ceph-mon[32220]: pgmap v133: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 32 MiB/s wr, 587 op/s 2024-06-06T03:42:59.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:42:58 smithi131 ceph-mon[36687]: pgmap v133: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 32 MiB/s wr, 587 op/s 2024-06-06T03:42:59.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:42:58 smithi104 ceph-mon[36569]: pgmap v133: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 32 MiB/s wr, 587 op/s 2024-06-06T03:42:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:43:01.254 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:00 smithi040 ceph-mon[32220]: pgmap v134: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 33 MiB/s wr, 636 op/s 2024-06-06T03:43:01.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:00 smithi131 ceph-mon[36687]: pgmap v134: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 33 MiB/s wr, 636 op/s 2024-06-06T03:43:01.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:00 smithi104 ceph-mon[36569]: pgmap v134: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 33 MiB/s wr, 636 op/s 2024-06-06T03:43:01.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.620+0000 7ff945d18700 1 -- 172.21.15.40:0/3970049939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff940107840 msgr2=0x7ff94010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:01.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.620+0000 7ff945d18700 1 --2- 172.21.15.40:0/3970049939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff940107840 0x7ff94010f3c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff93800b0d0 tx=0x7ff93800b3e0 comp rx=0 tx=0).stop 2024-06-06T03:43:01.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.620+0000 7ff945d18700 1 -- 172.21.15.40:0/3970049939 shutdown_connections 2024-06-06T03:43:01.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.620+0000 7ff945d18700 1 --2- 172.21.15.40:0/3970049939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff940102740 0x7ff940102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:01.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.620+0000 7ff945d18700 1 --2- 172.21.15.40:0/3970049939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff940107840 0x7ff94010f3c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:01.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.620+0000 7ff945d18700 1 --2- 172.21.15.40:0/3970049939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9401030f0 0x7ff940107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.620+0000 7ff945d18700 1 -- 172.21.15.40:0/3970049939 >> 172.21.15.40:0/3970049939 conn(0x7ff9400795f0 msgr2=0x7ff940079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.620+0000 7ff945d18700 1 -- 172.21.15.40:0/3970049939 shutdown_connections 2024-06-06T03:43:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.620+0000 7ff945d18700 1 -- 172.21.15.40:0/3970049939 wait complete. 2024-06-06T03:43:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.621+0000 7ff945d18700 1 Processor -- start 2024-06-06T03:43:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.621+0000 7ff945d18700 1 -- start start 2024-06-06T03:43:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.621+0000 7ff945d18700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff940102740 0x7ff94019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.621+0000 7ff945d18700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9401030f0 0x7ff94019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff945d18700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff940107840 0x7ff9401988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff945d18700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff940112780 con 0x7ff940107840 2024-06-06T03:43:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff945d18700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff940112600 con 0x7ff9401030f0 2024-06-06T03:43:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff945d18700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff940112900 con 0x7ff940102740 2024-06-06T03:43:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff940107840 0x7ff9401988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff940107840 0x7ff9401988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50502/0 (socket says 172.21.15.40:50502) 2024-06-06T03:43:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93ffff700 1 -- 172.21.15.40:0/4293313207 learned_addr learned my addr 172.21.15.40:0/4293313207 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93effd700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9401030f0 0x7ff94019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93f7fe700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff940102740 0x7ff94019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93ffff700 1 -- 172.21.15.40:0/4293313207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff940102740 msgr2=0x7ff94019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93ffff700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff940102740 0x7ff94019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93ffff700 1 -- 172.21.15.40:0/4293313207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9401030f0 msgr2=0x7ff94019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93ffff700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9401030f0 0x7ff94019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93ffff700 1 -- 172.21.15.40:0/4293313207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff940199020 con 0x7ff940107840 2024-06-06T03:43:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93effd700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9401030f0 0x7ff94019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:43:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93f7fe700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff940102740 0x7ff94019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:43:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93ffff700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff940107840 0x7ff9401988c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff938006070 tx=0x7ff938004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93cff9700 1 -- 172.21.15.40:0/4293313207 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938021070 con 0x7ff940107840 2024-06-06T03:43:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff945d18700 1 -- 172.21.15.40:0/4293313207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9401992b0 con 0x7ff940107840 2024-06-06T03:43:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93cff9700 1 -- 172.21.15.40:0/4293313207 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff938007df0 con 0x7ff940107840 2024-06-06T03:43:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff93cff9700 1 -- 172.21.15.40:0/4293313207 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938005470 con 0x7ff940107840 2024-06-06T03:43:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.622+0000 7ff945d18700 1 -- 172.21.15.40:0/4293313207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9401a55d0 con 0x7ff940107840 2024-06-06T03:43:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.623+0000 7ff945d18700 1 -- 172.21.15.40:0/4293313207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff940069180 con 0x7ff940107840 2024-06-06T03:43:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.625+0000 7ff93cff9700 1 -- 172.21.15.40:0/4293313207 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff938024070 con 0x7ff940107840 2024-06-06T03:43:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.626+0000 7ff93cff9700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7ff9280a2d90 0x7ff9280a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.626+0000 7ff93f7fe700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7ff9280a2d90 0x7ff9280a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.626+0000 7ff93cff9700 1 -- 172.21.15.40:0/4293313207 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7ff93800f020 con 0x7ff940107840 2024-06-06T03:43:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.626+0000 7ff93f7fe700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7ff9280a2d90 0x7ff9280a5250 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff930005fd0 tx=0x7ff930005de0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.630+0000 7ff93cff9700 1 -- 172.21.15.40:0/4293313207 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff93802b020 con 0x7ff940107840 2024-06-06T03:43:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.885+0000 7ff945d18700 1 -- 172.21.15.40:0/4293313207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9401a57a0 con 0x7ff940107840 2024-06-06T03:43:01.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.886+0000 7ff93cff9700 1 -- 172.21.15.40:0/4293313207 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11945 (secure 0 0 0) 0x7ff93809dcd0 con 0x7ff940107840 2024-06-06T03:43:01.886 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:43:01.887 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:41:20.463469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24443,24470,24425,24403,24398]},"id":1}]} 2024-06-06T03:43:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.888+0000 7ff945d18700 1 -- 172.21.15.40:0/4293313207 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7ff9280a2d90 msgr2=0x7ff9280a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.888+0000 7ff945d18700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7ff9280a2d90 0x7ff9280a5250 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff930005fd0 tx=0x7ff930005de0 comp rx=0 tx=0).stop 2024-06-06T03:43:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.888+0000 7ff945d18700 1 -- 172.21.15.40:0/4293313207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff940107840 msgr2=0x7ff9401988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.888+0000 7ff945d18700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff940107840 0x7ff9401988c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff938006070 tx=0x7ff938004ef0 comp rx=0 tx=0).stop 2024-06-06T03:43:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.888+0000 7ff945d18700 1 -- 172.21.15.40:0/4293313207 shutdown_connections 2024-06-06T03:43:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.888+0000 7ff945d18700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7ff9280a2d90 0x7ff9280a5250 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.888+0000 7ff945d18700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff940102740 0x7ff94019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.888+0000 7ff945d18700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff940107840 0x7ff9401988c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.888+0000 7ff945d18700 1 --2- 172.21.15.40:0/4293313207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9401030f0 0x7ff94019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.888+0000 7ff945d18700 1 -- 172.21.15.40:0/4293313207 >> 172.21.15.40:0/4293313207 conn(0x7ff9400795f0 msgr2=0x7ff940100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.888+0000 7ff945d18700 1 -- 172.21.15.40:0/4293313207 shutdown_connections 2024-06-06T03:43:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:01.888+0000 7ff945d18700 1 -- 172.21.15.40:0/4293313207 wait complete. 2024-06-06T03:43:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 35 2024-06-06T03:43:02.253 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:01 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/4293313207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:43:02.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:01 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/4293313207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:43:02.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:01 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/4293313207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:43:02.702 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 699b8c5b --release --await' 2024-06-06T03:43:02.703 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 699b8c5b --release --await 2024-06-06T03:43:03.000 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:43:03.242 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:02 smithi104 ceph-mon[36569]: pgmap v135: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 31 MiB/s wr, 386 op/s 2024-06-06T03:43:03.262 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:02 smithi040 ceph-mon[32220]: pgmap v135: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 31 MiB/s wr, 386 op/s 2024-06-06T03:43:03.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:02 smithi131 ceph-mon[36687]: pgmap v135: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 31 MiB/s wr, 386 op/s 2024-06-06T03:43:04.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.732+0000 7fc84f52c700 1 -- 172.21.15.40:0/543315009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc848105f50 msgr2=0x7fc84810da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:04.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.732+0000 7fc84f52c700 1 --2- 172.21.15.40:0/543315009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc848105f50 0x7fc84810da90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc84000c210 tx=0x7fc84000c520 comp rx=0 tx=0).stop 2024-06-06T03:43:04.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.732+0000 7fc84f52c700 1 -- 172.21.15.40:0/543315009 shutdown_connections 2024-06-06T03:43:04.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.732+0000 7fc84f52c700 1 --2- 172.21.15.40:0/543315009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc848105f50 0x7fc84810da90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.732+0000 7fc84f52c700 1 --2- 172.21.15.40:0/543315009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8481010b0 0x7fc848101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.732+0000 7fc84f52c700 1 --2- 172.21.15.40:0/543315009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc848101a60 0x7fc848105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.732+0000 7fc84f52c700 1 -- 172.21.15.40:0/543315009 >> 172.21.15.40:0/543315009 conn(0x7fc8480766b0 msgr2=0x7fc848076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:04.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.732+0000 7fc84f52c700 1 -- 172.21.15.40:0/543315009 shutdown_connections 2024-06-06T03:43:04.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.732+0000 7fc84f52c700 1 -- 172.21.15.40:0/543315009 wait complete. 2024-06-06T03:43:04.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.732+0000 7fc84f52c700 1 Processor -- start 2024-06-06T03:43:04.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84f52c700 1 -- start start 2024-06-06T03:43:04.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84f52c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8481010b0 0x7fc84819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84f52c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc848101a60 0x7fc84819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84f52c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc848105f50 0x7fc8481a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84f52c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc848110660 con 0x7fc8481010b0 2024-06-06T03:43:04.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84d2c8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8481010b0 0x7fc84819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84d2c8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8481010b0 0x7fc84819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50530/0 (socket says 172.21.15.40:50530) 2024-06-06T03:43:04.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84d2c8700 1 -- 172.21.15.40:0/1922449801 learned_addr learned my addr 172.21.15.40:0/1922449801 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:04.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84f52c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481104e0 con 0x7fc848105f50 2024-06-06T03:43:04.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84f52c700 1 -- 172.21.15.40:0/1922449801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481107e0 con 0x7fc848101a60 2024-06-06T03:43:04.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84cac7700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc848101a60 0x7fc84819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84dac9700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc848105f50 0x7fc8481a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84d2c8700 1 -- 172.21.15.40:0/1922449801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc848101a60 msgr2=0x7fc84819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:04.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84d2c8700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc848101a60 0x7fc84819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84d2c8700 1 -- 172.21.15.40:0/1922449801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc848105f50 msgr2=0x7fc8481a39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:04.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84d2c8700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc848105f50 0x7fc8481a39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84d2c8700 1 -- 172.21.15.40:0/1922449801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8481a4120 con 0x7fc8481010b0 2024-06-06T03:43:04.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.733+0000 7fc84dac9700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc848105f50 0x7fc8481a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:43:04.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.734+0000 7fc84d2c8700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8481010b0 0x7fc84819f160 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc84400c9b0 tx=0x7fc84400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:04.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.734+0000 7fc83e7fc700 1 -- 172.21.15.40:0/1922449801 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844007b00 con 0x7fc8481010b0 2024-06-06T03:43:04.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.734+0000 7fc83e7fc700 1 -- 172.21.15.40:0/1922449801 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc844007ca0 con 0x7fc8481010b0 2024-06-06T03:43:04.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.734+0000 7fc84f52c700 1 -- 172.21.15.40:0/1922449801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8481a4410 con 0x7fc8481010b0 2024-06-06T03:43:04.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.734+0000 7fc83e7fc700 1 -- 172.21.15.40:0/1922449801 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844018730 con 0x7fc8481010b0 2024-06-06T03:43:04.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.734+0000 7fc84f52c700 1 -- 172.21.15.40:0/1922449801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8481abc90 con 0x7fc8481010b0 2024-06-06T03:43:04.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.736+0000 7fc83e7fc700 1 -- 172.21.15.40:0/1922449801 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc8440188d0 con 0x7fc8481010b0 2024-06-06T03:43:04.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.737+0000 7fc83e7fc700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fc8340a2d90 0x7fc8340a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.737+0000 7fc83e7fc700 1 -- 172.21.15.40:0/1922449801 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fc8440cf3a0 con 0x7fc8481010b0 2024-06-06T03:43:04.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.737+0000 7fc84cac7700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fc8340a2d90 0x7fc8340a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.737+0000 7fc84cac7700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fc8340a2d90 0x7fc8340a5250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc848107620 tx=0x7fc83800a300 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:04.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc859dafb80 1 Processor -- start 2024-06-06T03:43:04.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc859dafb80 1 -- start start 2024-06-06T03:43:04.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc859dafb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 0x55a049b86020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc859dafb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b86560 0x55a049b7fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc859dafb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b803d0 0x55a049b80810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc859dafb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a049ad1910 con 0x55a049b86560 2024-06-06T03:43:04.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc859dafb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a049a79820 con 0x55a049b803d0 2024-06-06T03:43:04.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc859dafb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a049acd8d0 con 0x55a049b85c40 2024-06-06T03:43:04.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc84cac7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b86560 0x55a049b7fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc84cac7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b86560 0x55a049b7fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50536/0 (socket says 172.21.15.40:50536) 2024-06-06T03:43:04.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc84cac7700 1 -- 172.21.15.40:0/4144642327 learned_addr learned my addr 172.21.15.40:0/4144642327 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:04.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc84d2c8700 1 --2- 172.21.15.40:0/4144642327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 0x55a049b86020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc84dac9700 1 --2- 172.21.15.40:0/4144642327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b803d0 0x55a049b80810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc84cac7700 1 -- 172.21.15.40:0/4144642327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 msgr2=0x55a049b86020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:04.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc84cac7700 1 --2- 172.21.15.40:0/4144642327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 0x55a049b86020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc84cac7700 1 -- 172.21.15.40:0/4144642327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b803d0 msgr2=0x55a049b80810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:04.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc84cac7700 1 --2- 172.21.15.40:0/4144642327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b803d0 0x55a049b80810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc84cac7700 1 -- 172.21.15.40:0/4144642327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a049a70050 con 0x55a049b86560 2024-06-06T03:43:04.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.738+0000 7fc84cac7700 1 --2- 172.21.15.40:0/4144642327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b86560 0x55a049b7fe90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc83800fd10 tx=0x7fc838012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8040140edccb86c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:04.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.739+0000 7fc8337fe700 1 -- 172.21.15.40:0/4144642327 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838016070 con 0x55a049b86560 2024-06-06T03:43:04.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.739+0000 7fc8337fe700 1 -- 172.21.15.40:0/4144642327 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc838010cc0 con 0x55a049b86560 2024-06-06T03:43:04.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.739+0000 7fc8337fe700 1 -- 172.21.15.40:0/4144642327 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83801b420 con 0x55a049b86560 2024-06-06T03:43:04.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.739+0000 7fc859dafb80 1 -- 172.21.15.40:0/4144642327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b86560 msgr2=0x55a049b7fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:04.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.739+0000 7fc859dafb80 1 --2- 172.21.15.40:0/4144642327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b86560 0x55a049b7fe90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc83800fd10 tx=0x7fc838012000 comp rx=0 tx=0).stop 2024-06-06T03:43:04.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.739+0000 7fc859dafb80 1 -- 172.21.15.40:0/4144642327 shutdown_connections 2024-06-06T03:43:04.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.739+0000 7fc859dafb80 1 --2- 172.21.15.40:0/4144642327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 0x55a049b86020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.739+0000 7fc859dafb80 1 --2- 172.21.15.40:0/4144642327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b803d0 0x55a049b80810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.739+0000 7fc859dafb80 1 --2- 172.21.15.40:0/4144642327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b86560 0x55a049b7fe90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.739+0000 7fc859dafb80 1 -- 172.21.15.40:0/4144642327 >> 172.21.15.40:0/4144642327 conn(0x55a049aaadc0 msgr2=0x55a049b7b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:04.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.740+0000 7fc859dafb80 1 -- 172.21.15.40:0/4144642327 shutdown_connections 2024-06-06T03:43:04.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.740+0000 7fc859dafb80 1 -- 172.21.15.40:0/4144642327 wait complete. 2024-06-06T03:43:04.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.740+0000 7fc859dafb80 1 -- start start 2024-06-06T03:43:04.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.740+0000 7fc859dafb80 1 Processor -- start 2024-06-06T03:43:04.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.740+0000 7fc859dafb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b803d0 0x55a049b8c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.740+0000 7fc859dafb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 0x55a049b89590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.740+0000 7fc859dafb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b86560 0x55a049b89ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.740+0000 7fc859dafb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a049b8cfd0 con 0x55a049b803d0 2024-06-06T03:43:04.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.740+0000 7fc859dafb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a049a71cf0 con 0x55a049b86560 2024-06-06T03:43:04.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.740+0000 7fc859dafb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a049b8d150 con 0x55a049b85c40 2024-06-06T03:43:04.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.740+0000 7fc859dafb80 10 client.0 _ll_register_callbacks cb 0x55a049a6ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:43:04.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.740+0000 7fc84d2c8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b803d0 0x55a049b8c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc84d2c8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b803d0 0x55a049b8c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50550/0 (socket says 172.21.15.40:50550) 2024-06-06T03:43:04.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc84d2c8700 1 -- 172.21.15.40:0/2882729079 learned_addr learned my addr 172.21.15.40:0/2882729079 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:04.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc84cac7700 1 --2- 172.21.15.40:0/2882729079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 0x55a049b89590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc84dac9700 1 --2- 172.21.15.40:0/2882729079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b86560 0x55a049b89ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc84d2c8700 1 -- 172.21.15.40:0/2882729079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 msgr2=0x55a049b89590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:04.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc84d2c8700 1 --2- 172.21.15.40:0/2882729079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 0x55a049b89590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc84d2c8700 1 -- 172.21.15.40:0/2882729079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b86560 msgr2=0x55a049b89ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:04.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc84d2c8700 1 --2- 172.21.15.40:0/2882729079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b86560 0x55a049b89ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc84d2c8700 1 -- 172.21.15.40:0/2882729079 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a049a209d0 con 0x55a049b803d0 2024-06-06T03:43:04.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc84dac9700 1 --2- 172.21.15.40:0/2882729079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b86560 0x55a049b89ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:43:04.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc84d2c8700 1 --2- 172.21.15.40:0/2882729079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b803d0 0x55a049b8c290 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc84400afd0 tx=0x7fc84409b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54669f74e97c3b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:04.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc830ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:43:04.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc830ff9700 1 -- 172.21.15.40:0/2882729079 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8440b4030 con 0x55a049b803d0 2024-06-06T03:43:04.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc830ff9700 1 -- 172.21.15.40:0/2882729079 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fc84409bdb0 con 0x55a049b803d0 2024-06-06T03:43:04.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc859dafb80 1 -- 172.21.15.40:0/2882729079 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc838010cc0 con 0x55a049b803d0 2024-06-06T03:43:04.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc830ff9700 1 -- 172.21.15.40:0/2882729079 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc8440ba270 con 0x55a049b803d0 2024-06-06T03:43:04.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc859dafb80 10 client.15561 fetch_fsmap learned FSMap version 35 2024-06-06T03:43:04.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc859dafb80 1 -- 172.21.15.40:0/2882729079 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a049b978f0 con 0x55a049b803d0 2024-06-06T03:43:04.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.741+0000 7fc830ff9700 1 -- 172.21.15.40:0/2882729079 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc838010cc0 con 0x55a049b803d0 2024-06-06T03:43:04.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.742+0000 7fc830ff9700 1 -- 172.21.15.40:0/2882729079 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fc844018bc0 con 0x55a049b803d0 2024-06-06T03:43:04.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.742+0000 7fc859dafb80 10 client.15561 fetch_fsmap finished waiting for FSMap version 35 2024-06-06T03:43:04.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.742+0000 7fc859dafb80 10 client.15561 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:43:04.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.742+0000 7fc859dafb80 20 client.15561 populate_metadata read hostname 'smithi040' 2024-06-06T03:43:04.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.742+0000 7fc859dafb80 1 --2- 172.21.15.40:0/2882729079 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a049b9a3b0 0x55a049b9c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.742+0000 7fc859dafb80 4 client.15561 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:43:04.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.742+0000 7fc859dafb80 1 -- 172.21.15.40:0/2882729079 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a049adee10 con 0x55a049b9a3b0 2024-06-06T03:43:04.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.742+0000 7fc84cac7700 1 --2- 172.21.15.40:0/2882729079 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a049b9a3b0 0x55a049b9c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.742+0000 7fc84cac7700 1 --2- 172.21.15.40:0/2882729079 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a049b9a3b0 0x55a049b9c830 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12a8ea32d4ac2089 server_cookie=a93570dd423671a3 in_seq=0 out_seq=0 2024-06-06T03:43:04.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.742+0000 7fc830ff9700 10 client.15561 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:43:04.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.743+0000 7fc830ff9700 1 -- 172.21.15.40:0/2882729079 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a049adee10 con 0x55a049b9a3b0 2024-06-06T03:43:04.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.743+0000 7fc830ff9700 10 client.15561 handle_command_reply: tid=0 2024-06-06T03:43:04.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.743+0000 7fc859dafb80 1 client.15561 shutdown 2024-06-06T03:43:04.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.743+0000 7fc859dafb80 1 -- 172.21.15.40:0/2882729079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b803d0 msgr2=0x55a049b8c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:04.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.743+0000 7fc859dafb80 1 --2- 172.21.15.40:0/2882729079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b803d0 0x55a049b8c290 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc84400afd0 tx=0x7fc84409b950 comp rx=0 tx=0).stop 2024-06-06T03:43:04.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.743+0000 7fc859dafb80 1 -- 172.21.15.40:0/2882729079 shutdown_connections 2024-06-06T03:43:04.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.743+0000 7fc859dafb80 1 --2- 172.21.15.40:0/2882729079 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a049b9a3b0 0x55a049b9c830 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.743+0000 7fc859dafb80 1 --2- 172.21.15.40:0/2882729079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b86560 0x55a049b89ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.743+0000 7fc859dafb80 1 --2- 172.21.15.40:0/2882729079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b803d0 0x55a049b8c290 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.743+0000 7fc830ff9700 0 client.15561 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:43:04.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.743+0000 7fc859dafb80 1 --2- 172.21.15.40:0/2882729079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 0x55a049b89590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.743+0000 7fc859dafb80 1 -- 172.21.15.40:0/2882729079 >> 172.21.15.40:0/2882729079 conn(0x55a049aaadc0 msgr2=0x55a049b7ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:04.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.744+0000 7fc859dafb80 1 -- 172.21.15.40:0/2882729079 shutdown_connections 2024-06-06T03:43:04.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.744+0000 7fc859dafb80 1 -- 172.21.15.40:0/2882729079 wait complete. 2024-06-06T03:43:04.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.744+0000 7fc859dafb80 20 client.15561 trim_cache size 0 max 16384 2024-06-06T03:43:04.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.764+0000 7fc859dafb80 1 Processor -- start 2024-06-06T03:43:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.764+0000 7fc859dafb80 1 -- start start 2024-06-06T03:43:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.764+0000 7fc859dafb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b803d0 0x55a049b871d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.764+0000 7fc859dafb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b85c40 0x55a049b884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.764+0000 7fc859dafb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b86560 0x55a049b89b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.764+0000 7fc859dafb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b3210 con 0x55a049b85c40 2024-06-06T03:43:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.764+0000 7fc859dafb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b4ea0 con 0x55a049b86560 2024-06-06T03:43:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.764+0000 7fc859dafb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b38e0 con 0x55a049b803d0 2024-06-06T03:43:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.765+0000 7fc84cac7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b85c40 0x55a049b884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.765+0000 7fc84cac7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b85c40 0x55a049b884f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50552/0 (socket says 172.21.15.40:50552) 2024-06-06T03:43:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.765+0000 7fc84cac7700 1 -- 172.21.15.40:0/1645224403 learned_addr learned my addr 172.21.15.40:0/1645224403 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.765+0000 7fc84d2c8700 1 --2- 172.21.15.40:0/1645224403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b803d0 0x55a049b871d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.765+0000 7fc84dac9700 1 --2- 172.21.15.40:0/1645224403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b86560 0x55a049b89b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.765+0000 7fc84cac7700 1 -- 172.21.15.40:0/1645224403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b803d0 msgr2=0x55a049b871d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.765+0000 7fc84cac7700 1 --2- 172.21.15.40:0/1645224403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b803d0 0x55a049b871d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.765+0000 7fc84cac7700 1 -- 172.21.15.40:0/1645224403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b86560 msgr2=0x55a049b89b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.765+0000 7fc84cac7700 1 --2- 172.21.15.40:0/1645224403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b86560 0x55a049b89b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.765+0000 7fc84cac7700 1 -- 172.21.15.40:0/1645224403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a049b85720 con 0x55a049b85c40 2024-06-06T03:43:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.765+0000 7fc84cac7700 1 --2- 172.21.15.40:0/1645224403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b85c40 0x55a049b884f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc83800a9d0 tx=0x7fc838005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1c646b7d9b49a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.765+0000 7fc832ffd700 1 -- 172.21.15.40:0/1645224403 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838035070 con 0x55a049b85c40 2024-06-06T03:43:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.765+0000 7fc832ffd700 1 -- 172.21.15.40:0/1645224403 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc838025470 con 0x55a049b85c40 2024-06-06T03:43:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.765+0000 7fc832ffd700 1 -- 172.21.15.40:0/1645224403 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838016030 con 0x55a049b85c40 2024-06-06T03:43:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.766+0000 7fc859dafb80 1 -- 172.21.15.40:0/1645224403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b85c40 msgr2=0x55a049b884f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.766+0000 7fc859dafb80 1 --2- 172.21.15.40:0/1645224403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b85c40 0x55a049b884f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc83800a9d0 tx=0x7fc838005e30 comp rx=0 tx=0).stop 2024-06-06T03:43:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.766+0000 7fc859dafb80 1 -- 172.21.15.40:0/1645224403 shutdown_connections 2024-06-06T03:43:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.766+0000 7fc859dafb80 1 --2- 172.21.15.40:0/1645224403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b803d0 0x55a049b871d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.766+0000 7fc859dafb80 1 --2- 172.21.15.40:0/1645224403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b86560 0x55a049b89b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.766+0000 7fc859dafb80 1 --2- 172.21.15.40:0/1645224403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b85c40 0x55a049b884f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.766+0000 7fc859dafb80 1 -- 172.21.15.40:0/1645224403 >> 172.21.15.40:0/1645224403 conn(0x55a049aaadc0 msgr2=0x55a049b7a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.766+0000 7fc859dafb80 1 -- 172.21.15.40:0/1645224403 shutdown_connections 2024-06-06T03:43:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.766+0000 7fc859dafb80 1 -- 172.21.15.40:0/1645224403 wait complete. 2024-06-06T03:43:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.766+0000 7fc859dafb80 1 -- start start 2024-06-06T03:43:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.766+0000 7fc859dafb80 1 Processor -- start 2024-06-06T03:43:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc859dafb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b803d0 0x55a049b92ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc859dafb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 0x55a049b8f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc859dafb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b86560 0x55a049b8f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc859dafb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b4710 con 0x55a049b86560 2024-06-06T03:43:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc859dafb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b3fb0 con 0x55a049b803d0 2024-06-06T03:43:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc859dafb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b11f0 con 0x55a049b85c40 2024-06-06T03:43:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc859dafb80 10 client.0 _ll_register_callbacks cb 0x55a049b75980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:43:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc84dac9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b86560 0x55a049b8f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc84dac9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b86560 0x55a049b8f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50564/0 (socket says 172.21.15.40:50564) 2024-06-06T03:43:04.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc84dac9700 1 -- 172.21.15.40:0/3341254718 learned_addr learned my addr 172.21.15.40:0/3341254718 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:04.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc84cac7700 1 --2- 172.21.15.40:0/3341254718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 0x55a049b8f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc84d2c8700 1 --2- 172.21.15.40:0/3341254718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b803d0 0x55a049b92ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc84dac9700 1 -- 172.21.15.40:0/3341254718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 msgr2=0x55a049b8f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:04.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc84dac9700 1 --2- 172.21.15.40:0/3341254718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 0x55a049b8f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc84dac9700 1 -- 172.21.15.40:0/3341254718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b803d0 msgr2=0x55a049b92ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:04.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc84dac9700 1 --2- 172.21.15.40:0/3341254718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b803d0 0x55a049b92ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:04.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc84dac9700 1 -- 172.21.15.40:0/3341254718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a049b90160 con 0x55a049b86560 2024-06-06T03:43:04.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc84dac9700 1 --2- 172.21.15.40:0/3341254718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b86560 0x55a049b8f920 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc840004cb0 tx=0x7fc840004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc498f3091ca961 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:04.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.767+0000 7fc830ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:43:04.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc830ff9700 1 -- 172.21.15.40:0/3341254718 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8400074a0 con 0x55a049b86560 2024-06-06T03:43:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc830ff9700 1 -- 172.21.15.40:0/3341254718 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fc840007640 con 0x55a049b86560 2024-06-06T03:43:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc859dafb80 1 -- 172.21.15.40:0/3341254718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a049ad9390 con 0x55a049b86560 2024-06-06T03:43:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc830ff9700 1 -- 172.21.15.40:0/3341254718 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc840007840 con 0x55a049b86560 2024-06-06T03:43:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc859dafb80 10 client.15567 fetch_fsmap learned FSMap version 35 2024-06-06T03:43:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc859dafb80 1 -- 172.21.15.40:0/3341254718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a049c06190 con 0x55a049b86560 2024-06-06T03:43:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc830ff9700 1 -- 172.21.15.40:0/3341254718 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a049ad9390 con 0x55a049b86560 2024-06-06T03:43:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc830ff9700 1 -- 172.21.15.40:0/3341254718 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fc840012070 con 0x55a049b86560 2024-06-06T03:43:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc859dafb80 10 client.15567 fetch_fsmap finished waiting for FSMap version 35 2024-06-06T03:43:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc859dafb80 10 client.15567 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:43:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc859dafb80 20 client.15567 populate_metadata read hostname 'smithi040' 2024-06-06T03:43:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc859dafb80 1 --2- 172.21.15.40:0/3341254718 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a049c06990 0x55a049c08db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc859dafb80 4 client.15567 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "699b8c5b", "release": true, "await": true}] 2024-06-06T03:43:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc859dafb80 1 -- 172.21.15.40:0/3341254718 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "set_id": "699b8c5b", "release": true, "await": true}) -- 0x55a049953350 con 0x55a049c06990 2024-06-06T03:43:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.768+0000 7fc84d2c8700 1 --2- 172.21.15.40:0/3341254718 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a049c06990 0x55a049c08db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.769+0000 7fc84d2c8700 1 --2- 172.21.15.40:0/3341254718 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a049c06990 0x55a049c08db0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f84f8019f58c265b server_cookie=e6d5be419a284259 in_seq=0 out_seq=0 2024-06-06T03:43:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:04.769+0000 7fc830ff9700 10 client.15567 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:43:05.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:04 smithi040 ceph-mon[32220]: pgmap v136: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 31 MiB/s wr, 386 op/s 2024-06-06T03:43:05.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:04 smithi131 ceph-mon[36687]: pgmap v136: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 31 MiB/s wr, 386 op/s 2024-06-06T03:43:05.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:04 smithi104 ceph-mon[36569]: pgmap v136: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 31 MiB/s wr, 386 op/s 2024-06-06T03:43:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc830ff9700 1 -- 172.21.15.40:0/3341254718 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a049953350 con 0x55a049c06990 2024-06-06T03:43:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc830ff9700 10 client.15567 handle_command_reply: tid=0 2024-06-06T03:43:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc859dafb80 1 client.15567 shutdown 2024-06-06T03:43:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc859dafb80 1 -- 172.21.15.40:0/3341254718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b86560 msgr2=0x55a049b8f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc859dafb80 1 --2- 172.21.15.40:0/3341254718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b86560 0x55a049b8f920 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc840004cb0 tx=0x7fc840004ce0 comp rx=0 tx=0).stop 2024-06-06T03:43:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc859dafb80 1 -- 172.21.15.40:0/3341254718 shutdown_connections 2024-06-06T03:43:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc859dafb80 1 --2- 172.21.15.40:0/3341254718 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a049c06990 0x55a049c08db0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc859dafb80 1 --2- 172.21.15.40:0/3341254718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a049b86560 0x55a049b8f920 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc859dafb80 1 --2- 172.21.15.40:0/3341254718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a049b803d0 0x55a049b92ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc859dafb80 1 --2- 172.21.15.40:0/3341254718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a049b85c40 0x55a049b8f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc830ff9700 0 client.15567 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:43:05.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc859dafb80 1 -- 172.21.15.40:0/3341254718 >> 172.21.15.40:0/3341254718 conn(0x55a049aaadc0 msgr2=0x55a049b7b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:05.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc859dafb80 1 -- 172.21.15.40:0/3341254718 shutdown_connections 2024-06-06T03:43:05.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc859dafb80 1 -- 172.21.15.40:0/3341254718 wait complete. 2024-06-06T03:43:05.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc859dafb80 20 client.15567 trim_cache size 0 max 16384 2024-06-06T03:43:05.631 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:43:05.631 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 35, 2024-06-06T03:43:05.631 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:43:05.631 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 12, 2024-06-06T03:43:05.631 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": { 2024-06-06T03:43:05.631 INFO:teuthology.orchestra.run.smithi040.stdout: "699b8c5b": { 2024-06-06T03:43:05.631 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 12, 2024-06-06T03:43:05.631 INFO:teuthology.orchestra.run.smithi040.stdout: "age_ref": 0.0, 2024-06-06T03:43:05.631 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:43:05.631 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RELEASED", 2024-06-06T03:43:05.631 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:43:05.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:43:05.631 INFO:teuthology.orchestra.run.smithi040.stdout: "timeout": 90.0, 2024-06-06T03:43:05.631 INFO:teuthology.orchestra.run.smithi040.stdout: "expiration": 145.2, 2024-06-06T03:43:05.632 INFO:teuthology.orchestra.run.smithi040.stdout: "members": { 2024-06-06T03:43:05.632 INFO:teuthology.orchestra.run.smithi040.stdout: "file:/": { 2024-06-06T03:43:05.632 INFO:teuthology.orchestra.run.smithi040.stdout: "excluded": false, 2024-06-06T03:43:05.632 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:43:05.632 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RELEASED", 2024-06-06T03:43:05.632 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:43:05.632 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:05.632 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:05.632 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:05.632 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:05.632 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:05.632 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:43:05.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc84f52c700 1 -- 172.21.15.40:0/1922449801 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fc8340a2d90 msgr2=0x7fc8340a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:05.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc84f52c700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fc8340a2d90 0x7fc8340a5250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc848107620 tx=0x7fc83800a300 comp rx=0 tx=0).stop 2024-06-06T03:43:05.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc84f52c700 1 -- 172.21.15.40:0/1922449801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8481010b0 msgr2=0x7fc84819f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:05.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc84f52c700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8481010b0 0x7fc84819f160 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc84400c9b0 tx=0x7fc84400ccc0 comp rx=0 tx=0).stop 2024-06-06T03:43:05.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc84f52c700 1 -- 172.21.15.40:0/1922449801 shutdown_connections 2024-06-06T03:43:05.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc84f52c700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fc8340a2d90 0x7fc8340a5250 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:05.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc84f52c700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc848105f50 0x7fc8481a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:05.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc84f52c700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8481010b0 0x7fc84819f160 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:05.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc84f52c700 1 --2- 172.21.15.40:0/1922449801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc848101a60 0x7fc84819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:05.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc84f52c700 1 -- 172.21.15.40:0/1922449801 >> 172.21.15.40:0/1922449801 conn(0x7fc8480766b0 msgr2=0x7fc8480ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:05.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc84f52c700 1 -- 172.21.15.40:0/1922449801 shutdown_connections 2024-06-06T03:43:05.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:05.629+0000 7fc84f52c700 1 -- 172.21.15.40:0/1922449801 wait complete. 2024-06-06T03:43:06.720 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '699b8c5b', total seconds elapsed: 32.8 2024-06-06T03:43:06.720 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.25 2024-06-06T03:43:06.720 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:43:06.981 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:43:07.243 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:06 smithi040 ceph-mon[32220]: pgmap v137: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 31 MiB/s wr, 410 op/s 2024-06-06T03:43:07.243 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:06 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:43:07.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:06 smithi131 ceph-mon[36687]: pgmap v137: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 31 MiB/s wr, 410 op/s 2024-06-06T03:43:07.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:06 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:43:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:06 smithi104 ceph-mon[36569]: pgmap v137: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 31 MiB/s wr, 410 op/s 2024-06-06T03:43:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:06 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:43:08.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.913+0000 7fe8789fa700 1 -- 172.21.15.40:0/3978347163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8741073c0 msgr2=0x7fe8741077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:08.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.913+0000 7fe8789fa700 1 --2- 172.21.15.40:0/3978347163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8741073c0 0x7fe8741077a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe86400b0a0 tx=0x7fe86400b3b0 comp rx=0 tx=0).stop 2024-06-06T03:43:08.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.913+0000 7fe8789fa700 1 -- 172.21.15.40:0/3978347163 shutdown_connections 2024-06-06T03:43:08.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.913+0000 7fe8789fa700 1 --2- 172.21.15.40:0/3978347163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe874101d40 0x7fe87410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:08.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.913+0000 7fe8789fa700 1 --2- 172.21.15.40:0/3978347163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8741073c0 0x7fe8741077a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:08.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.913+0000 7fe8789fa700 1 --2- 172.21.15.40:0/3978347163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe874101380 0x7fe874101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:08.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.913+0000 7fe8789fa700 1 -- 172.21.15.40:0/3978347163 >> 172.21.15.40:0/3978347163 conn(0x7fe874078330 msgr2=0x7fe874078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:08.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.913+0000 7fe8789fa700 1 -- 172.21.15.40:0/3978347163 shutdown_connections 2024-06-06T03:43:08.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.913+0000 7fe8789fa700 1 -- 172.21.15.40:0/3978347163 wait complete. 2024-06-06T03:43:08.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.914+0000 7fe8789fa700 1 Processor -- start 2024-06-06T03:43:08.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.914+0000 7fe8789fa700 1 -- start start 2024-06-06T03:43:08.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe8789fa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe874101380 0x7fe874103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:08.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe8789fa700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe874101d40 0x7fe874103570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:08.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe8789fa700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8741073c0 0x7fe874103ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:08.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe8789fa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe874111660 con 0x7fe874101380 2024-06-06T03:43:08.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe8789fa700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741114e0 con 0x7fe874101d40 2024-06-06T03:43:08.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe8789fa700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741117e0 con 0x7fe8741073c0 2024-06-06T03:43:08.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe871d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe874101d40 0x7fe874103570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:08.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe871d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe874101d40 0x7fe874103570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:40278/0 (socket says 172.21.15.40:40278) 2024-06-06T03:43:08.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe871d9b700 1 -- 172.21.15.40:0/3988883876 learned_addr learned my addr 172.21.15.40:0/3988883876 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe87259c700 1 --2- 172.21.15.40:0/3988883876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe874101380 0x7fe874103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe872d9d700 1 --2- 172.21.15.40:0/3988883876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8741073c0 0x7fe874103ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe871d9b700 1 -- 172.21.15.40:0/3988883876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8741073c0 msgr2=0x7fe874103ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe871d9b700 1 --2- 172.21.15.40:0/3988883876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8741073c0 0x7fe874103ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe871d9b700 1 -- 172.21.15.40:0/3988883876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe874101380 msgr2=0x7fe874103030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe871d9b700 1 --2- 172.21.15.40:0/3988883876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe874101380 0x7fe874103030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe871d9b700 1 -- 172.21.15.40:0/3988883876 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8741a7100 con 0x7fe874101d40 2024-06-06T03:43:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.915+0000 7fe871d9b700 1 --2- 172.21.15.40:0/3988883876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe874101d40 0x7fe874103570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe85c00d980 tx=0x7fe85c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.916+0000 7fe86b7fe700 1 -- 172.21.15.40:0/3988883876 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c017070 con 0x7fe874101d40 2024-06-06T03:43:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.916+0000 7fe8789fa700 1 -- 172.21.15.40:0/3988883876 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8741a73f0 con 0x7fe874101d40 2024-06-06T03:43:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.916+0000 7fe86b7fe700 1 -- 172.21.15.40:0/3988883876 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe85c005020 con 0x7fe874101d40 2024-06-06T03:43:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.916+0000 7fe86b7fe700 1 -- 172.21.15.40:0/3988883876 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c009bc0 con 0x7fe874101d40 2024-06-06T03:43:08.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.916+0000 7fe8789fa700 1 -- 172.21.15.40:0/3988883876 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8741a78e0 con 0x7fe874101d40 2024-06-06T03:43:08.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.917+0000 7fe8789fa700 1 -- 172.21.15.40:0/3988883876 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe874102a70 con 0x7fe874101d40 2024-06-06T03:43:08.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.919+0000 7fe86b7fe700 1 -- 172.21.15.40:0/3988883876 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe85c0051d0 con 0x7fe874101d40 2024-06-06T03:43:08.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.919+0000 7fe86b7fe700 1 --2- 172.21.15.40:0/3988883876 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fe8600a2d90 0x7fe8600a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:08.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.919+0000 7fe87259c700 1 --2- 172.21.15.40:0/3988883876 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fe8600a2d90 0x7fe8600a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:08.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.919+0000 7fe86b7fe700 1 -- 172.21.15.40:0/3988883876 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7fe85c0cf470 con 0x7fe874101d40 2024-06-06T03:43:08.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.920+0000 7fe87259c700 1 --2- 172.21.15.40:0/3988883876 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fe8600a2d90 0x7fe8600a5250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe864009e60 tx=0x7fe86400b700 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:08.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:08.922+0000 7fe86b7fe700 1 -- 172.21.15.40:0/3988883876 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe85c020020 con 0x7fe874101d40 2024-06-06T03:43:09.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:09.185+0000 7fe8789fa700 1 -- 172.21.15.40:0/3988883876 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe874069180 con 0x7fe874101d40 2024-06-06T03:43:09.185 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:08 smithi040 ceph-mon[32220]: pgmap v138: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 525 KiB/s wr, 72 op/s 2024-06-06T03:43:09.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:09.186+0000 7fe86b7fe700 1 -- 172.21.15.40:0/3988883876 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11941 (secure 0 0 0) 0x7fe85c09c120 con 0x7fe874101d40 2024-06-06T03:43:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:43:09.186 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:43:05.692670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24398,24403,24425,24443,24470]},"id":1}]} 2024-06-06T03:43:09.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:09.188+0000 7fe8789fa700 1 -- 172.21.15.40:0/3988883876 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fe8600a2d90 msgr2=0x7fe8600a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:09.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:09.188+0000 7fe8789fa700 1 --2- 172.21.15.40:0/3988883876 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fe8600a2d90 0x7fe8600a5250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe864009e60 tx=0x7fe86400b700 comp rx=0 tx=0).stop 2024-06-06T03:43:09.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:09.188+0000 7fe8789fa700 1 -- 172.21.15.40:0/3988883876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe874101d40 msgr2=0x7fe874103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:09.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:09.188+0000 7fe8789fa700 1 --2- 172.21.15.40:0/3988883876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe874101d40 0x7fe874103570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe85c00d980 tx=0x7fe85c00dc90 comp rx=0 tx=0).stop 2024-06-06T03:43:09.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:09.188+0000 7fe8789fa700 1 -- 172.21.15.40:0/3988883876 shutdown_connections 2024-06-06T03:43:09.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:09.188+0000 7fe8789fa700 1 --2- 172.21.15.40:0/3988883876 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fe8600a2d90 0x7fe8600a5250 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:09.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:09.188+0000 7fe8789fa700 1 --2- 172.21.15.40:0/3988883876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8741073c0 0x7fe874103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:09.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:09.188+0000 7fe8789fa700 1 --2- 172.21.15.40:0/3988883876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe874101380 0x7fe874103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:09.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:09.188+0000 7fe8789fa700 1 --2- 172.21.15.40:0/3988883876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe874101d40 0x7fe874103570 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:09.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:09.188+0000 7fe8789fa700 1 -- 172.21.15.40:0/3988883876 >> 172.21.15.40:0/3988883876 conn(0x7fe874078330 msgr2=0x7fe874100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:09.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:09.188+0000 7fe8789fa700 1 -- 172.21.15.40:0/3988883876 shutdown_connections 2024-06-06T03:43:09.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:09.189+0000 7fe8789fa700 1 -- 172.21.15.40:0/3988883876 wait complete. 2024-06-06T03:43:09.189 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 36 2024-06-06T03:43:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:08 smithi131 ceph-mon[36687]: pgmap v138: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 525 KiB/s wr, 72 op/s 2024-06-06T03:43:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:08 smithi104 ceph-mon[36569]: pgmap v138: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 525 KiB/s wr, 72 op/s 2024-06-06T03:43:10.010 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0' 2024-06-06T03:43:10.010 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0 2024-06-06T03:43:10.279 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:43:10.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:09 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3988883876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:43:10.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:09 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3988883876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:43:10.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:09 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3988883876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:43:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:10 smithi131 ceph-mon[36687]: pgmap v139: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 958 KiB/s wr, 84 op/s 2024-06-06T03:43:11.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:10 smithi040 ceph-mon[32220]: pgmap v139: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 958 KiB/s wr, 84 op/s 2024-06-06T03:43:11.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:10 smithi104 ceph-mon[36569]: pgmap v139: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 958 KiB/s wr, 84 op/s 2024-06-06T03:43:12.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.291+0000 7f2010ed4700 1 -- 172.21.15.40:0/2185607619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f200c107790 msgr2=0x7f200c10f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:12.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.291+0000 7f2010ed4700 1 --2- 172.21.15.40:0/2185607619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f200c107790 0x7f200c10f3a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f200400b0d0 tx=0x7f200400b3e0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.291+0000 7f2010ed4700 1 -- 172.21.15.40:0/2185607619 shutdown_connections 2024-06-06T03:43:12.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.291+0000 7f2010ed4700 1 --2- 172.21.15.40:0/2185607619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f200c107790 0x7f200c10f3a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.291+0000 7f2010ed4700 1 --2- 172.21.15.40:0/2185607619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f200c1027a0 0x7f200c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.291+0000 7f2010ed4700 1 --2- 172.21.15.40:0/2185607619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200c103150 0x7f200c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.291+0000 7f2010ed4700 1 -- 172.21.15.40:0/2185607619 >> 172.21.15.40:0/2185607619 conn(0x7f200c079560 msgr2=0x7f200c079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:12.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.291+0000 7f2010ed4700 1 -- 172.21.15.40:0/2185607619 shutdown_connections 2024-06-06T03:43:12.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.291+0000 7f2010ed4700 1 -- 172.21.15.40:0/2185607619 wait complete. 2024-06-06T03:43:12.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.291+0000 7f2010ed4700 1 Processor -- start 2024-06-06T03:43:12.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f2010ed4700 1 -- start start 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f2010ed4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200c1027a0 0x7f200c19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f2010ed4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f200c103150 0x7f200c19b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f2010ed4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f200c107790 0x7f200c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f2010ed4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1127c0 con 0x7f200c107790 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f2010ed4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f200c112640 con 0x7f200c103150 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f2010ed4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f200c112940 con 0x7f200c1027a0 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f200ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f200c107790 0x7f200c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f200ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f200c107790 0x7f200c19f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50478/0 (socket says 172.21.15.40:50478) 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f200ad9d700 1 -- 172.21.15.40:0/3247528350 learned_addr learned my addr 172.21.15.40:0/3247528350 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f2009d9b700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f200c103150 0x7f200c19b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f200a59c700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200c1027a0 0x7f200c19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f200ad9d700 1 -- 172.21.15.40:0/3247528350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200c1027a0 msgr2=0x7f200c19ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f200ad9d700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200c1027a0 0x7f200c19ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f200ad9d700 1 -- 172.21.15.40:0/3247528350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f200c103150 msgr2=0x7f200c19b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f200ad9d700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f200c103150 0x7f200c19b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f200ad9d700 1 -- 172.21.15.40:0/3247528350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c19fd20 con 0x7f200c107790 2024-06-06T03:43:12.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f2009d9b700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f200c103150 0x7f200c19b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:43:12.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.292+0000 7f200ad9d700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f200c107790 0x7f200c19f5e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2004006070 tx=0x7f2004007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:12.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.293+0000 7f1ffb7fe700 1 -- 172.21.15.40:0/3247528350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004003c00 con 0x7f200c107790 2024-06-06T03:43:12.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.293+0000 7f1ffb7fe700 1 -- 172.21.15.40:0/3247528350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2004003da0 con 0x7f200c107790 2024-06-06T03:43:12.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.293+0000 7f2010ed4700 1 -- 172.21.15.40:0/3247528350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200c19ffb0 con 0x7f200c107790 2024-06-06T03:43:12.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.293+0000 7f1ffb7fe700 1 -- 172.21.15.40:0/3247528350 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200401b960 con 0x7f200c107790 2024-06-06T03:43:12.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.293+0000 7f2010ed4700 1 -- 172.21.15.40:0/3247528350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200c1a7890 con 0x7f200c107790 2024-06-06T03:43:12.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.295+0000 7f1ffb7fe700 1 -- 172.21.15.40:0/3247528350 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f200401bb00 con 0x7f200c107790 2024-06-06T03:43:12.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.296+0000 7f1ffb7fe700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f1ff40a2eb0 0x7f1ff40a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.296+0000 7f1ffb7fe700 1 -- 172.21.15.40:0/3247528350 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f20040d08a0 con 0x7f200c107790 2024-06-06T03:43:12.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.296+0000 7f200a59c700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f1ff40a2eb0 0x7f1ff40a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.296+0000 7f200a59c700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f1ff40a2eb0 0x7f1ff40a5370 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009bf0 tx=0x7f1ffc009260 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.296+0000 7f201b757b80 1 Processor -- start 2024-06-06T03:43:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f201b757b80 1 -- start start 2024-06-06T03:43:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f201b757b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f09205a0 0x55c6f0920980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f201b757b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f0920ec0 0x55c6f091a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f201b757b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f091acf0 0x55c6f091b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f201b757b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c6f080d100 con 0x55c6f0920ec0 2024-06-06T03:43:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f201b757b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c6f086b9f0 con 0x55c6f091acf0 2024-06-06T03:43:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f201b757b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c6f07d3600 con 0x55c6f09205a0 2024-06-06T03:43:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f2009d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f0920ec0 0x55c6f091a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f2009d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f0920ec0 0x55c6f091a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50494/0 (socket says 172.21.15.40:50494) 2024-06-06T03:43:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f2009d9b700 1 -- 172.21.15.40:0/2435449781 learned_addr learned my addr 172.21.15.40:0/2435449781 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f200ad9d700 1 --2- 172.21.15.40:0/2435449781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f091acf0 0x55c6f091b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f200a59c700 1 --2- 172.21.15.40:0/2435449781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f09205a0 0x55c6f0920980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f2009d9b700 1 -- 172.21.15.40:0/2435449781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f09205a0 msgr2=0x55c6f0920980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f2009d9b700 1 --2- 172.21.15.40:0/2435449781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f09205a0 0x55c6f0920980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f2009d9b700 1 -- 172.21.15.40:0/2435449781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f091acf0 msgr2=0x55c6f091b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f2009d9b700 1 --2- 172.21.15.40:0/2435449781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f091acf0 0x55c6f091b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.297+0000 7f2009d9b700 1 -- 172.21.15.40:0/2435449781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6f07baa70 con 0x55c6f0920ec0 2024-06-06T03:43:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.298+0000 7f2009d9b700 1 --2- 172.21.15.40:0/2435449781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f0920ec0 0x55c6f091a7b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f200000f3c0 tx=0x7f200000f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3f40d0a6757a2f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.298+0000 7f1ff8ff9700 1 -- 172.21.15.40:0/2435449781 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000004a30 con 0x55c6f0920ec0 2024-06-06T03:43:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.298+0000 7f1ff8ff9700 1 -- 172.21.15.40:0/2435449781 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2000005500 con 0x55c6f0920ec0 2024-06-06T03:43:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.298+0000 7f1ff8ff9700 1 -- 172.21.15.40:0/2435449781 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20000188c0 con 0x55c6f0920ec0 2024-06-06T03:43:12.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 -- 172.21.15.40:0/2435449781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f0920ec0 msgr2=0x55c6f091a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:12.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 --2- 172.21.15.40:0/2435449781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f0920ec0 0x55c6f091a7b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f200000f3c0 tx=0x7f200000f6d0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 -- 172.21.15.40:0/2435449781 shutdown_connections 2024-06-06T03:43:12.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 --2- 172.21.15.40:0/2435449781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f09205a0 0x55c6f0920980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 --2- 172.21.15.40:0/2435449781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f091acf0 0x55c6f091b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 --2- 172.21.15.40:0/2435449781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f0920ec0 0x55c6f091a7b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 -- 172.21.15.40:0/2435449781 >> 172.21.15.40:0/2435449781 conn(0x55c6f0847220 msgr2=0x55c6f0915aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:12.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 -- 172.21.15.40:0/2435449781 shutdown_connections 2024-06-06T03:43:12.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 -- 172.21.15.40:0/2435449781 wait complete. 2024-06-06T03:43:12.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 -- start start 2024-06-06T03:43:12.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 Processor -- start 2024-06-06T03:43:12.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f091acf0 0x55c6f0926b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f09205a0 0x55c6f0923e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f0920ec0 0x55c6f09243d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c6f080c160 con 0x55c6f091acf0 2024-06-06T03:43:12.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c6f08679b0 con 0x55c6f0920ec0 2024-06-06T03:43:12.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.299+0000 7f201b757b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c6f09278a0 con 0x55c6f09205a0 2024-06-06T03:43:12.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f201b757b80 10 client.0 _ll_register_callbacks cb 0x55c6f0801750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:43:12.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f200ad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f0920ec0 0x55c6f09243d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f200ad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f0920ec0 0x55c6f09243d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:40318/0 (socket says 172.21.15.40:40318) 2024-06-06T03:43:12.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f200ad9d700 1 -- 172.21.15.40:0/275411093 learned_addr learned my addr 172.21.15.40:0/275411093 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:12.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f2009d9b700 1 --2- 172.21.15.40:0/275411093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f09205a0 0x55c6f0923e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f200a59c700 1 --2- 172.21.15.40:0/275411093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f091acf0 0x55c6f0926b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f200ad9d700 1 -- 172.21.15.40:0/275411093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f09205a0 msgr2=0x55c6f0923e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f200ad9d700 1 --2- 172.21.15.40:0/275411093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f09205a0 0x55c6f0923e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f200ad9d700 1 -- 172.21.15.40:0/275411093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f091acf0 msgr2=0x55c6f0926b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f200ad9d700 1 --2- 172.21.15.40:0/275411093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f091acf0 0x55c6f0926b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f200ad9d700 1 -- 172.21.15.40:0/275411093 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6f08318f0 con 0x55c6f0920ec0 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f200a59c700 1 --2- 172.21.15.40:0/275411093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f091acf0 0x55c6f0926b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f200ad9d700 1 --2- 172.21.15.40:0/275411093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f0920ec0 0x55c6f09243d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2004009850 tx=0x7f200409c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f354dac3427d12bf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f1ff1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.300+0000 7f1ff1ffb700 1 -- 172.21.15.40:0/275411093 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20040a5a60 con 0x55c6f0920ec0 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.301+0000 7f201b757b80 1 -- 172.21.15.40:0/275411093 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2000005500 con 0x55c6f0920ec0 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.301+0000 7f1ff1ffb700 1 -- 172.21.15.40:0/275411093 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f200409b180 con 0x55c6f0920ec0 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.301+0000 7f1ff1ffb700 1 -- 172.21.15.40:0/275411093 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f20040074e0 con 0x55c6f0920ec0 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.301+0000 7f201b757b80 10 client.25259 fetch_fsmap learned FSMap version 36 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.301+0000 7f201b757b80 1 -- 172.21.15.40:0/275411093 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c6f0931f90 con 0x55c6f0920ec0 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.301+0000 7f1ff1ffb700 1 -- 172.21.15.40:0/275411093 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2000005500 con 0x55c6f0920ec0 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.301+0000 7f1ff1ffb700 1 -- 172.21.15.40:0/275411093 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f200402e060 con 0x55c6f0920ec0 2024-06-06T03:43:12.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.301+0000 7f201b757b80 10 client.25259 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T03:43:12.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.301+0000 7f201b757b80 10 client.25259 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:43:12.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.301+0000 7f201b757b80 20 client.25259 populate_metadata read hostname 'smithi040' 2024-06-06T03:43:12.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.301+0000 7f201b757b80 1 --2- 172.21.15.40:0/275411093 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c6f09345d0 0x55c6f09369f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.301+0000 7f201b757b80 4 client.25259 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:43:12.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.301+0000 7f201b757b80 1 -- 172.21.15.40:0/275411093 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6f0878ef0 con 0x55c6f09345d0 2024-06-06T03:43:12.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.303+0000 7f200a59c700 1 --2- 172.21.15.40:0/275411093 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c6f09345d0 0x55c6f09369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.304+0000 7f200a59c700 1 --2- 172.21.15.40:0/275411093 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c6f09345d0 0x55c6f09369f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d142375b6302aa server_cookie=9a328a2d6f53a35d in_seq=0 out_seq=0 2024-06-06T03:43:12.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.304+0000 7f1ff1ffb700 10 client.25259 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:43:12.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.305+0000 7f1ff1ffb700 1 -- 172.21.15.40:0/275411093 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c6f0878ef0 con 0x55c6f09345d0 2024-06-06T03:43:12.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.305+0000 7f1ff1ffb700 10 client.25259 handle_command_reply: tid=0 2024-06-06T03:43:12.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.305+0000 7f201b757b80 1 client.25259 shutdown 2024-06-06T03:43:12.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.306+0000 7f201b757b80 1 -- 172.21.15.40:0/275411093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f0920ec0 msgr2=0x55c6f09243d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:12.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.306+0000 7f201b757b80 1 --2- 172.21.15.40:0/275411093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f0920ec0 0x55c6f09243d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2004009850 tx=0x7f200409c8b0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.306+0000 7f201b757b80 1 -- 172.21.15.40:0/275411093 shutdown_connections 2024-06-06T03:43:12.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.306+0000 7f201b757b80 1 --2- 172.21.15.40:0/275411093 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c6f09345d0 0x55c6f09369f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.306+0000 7f201b757b80 1 --2- 172.21.15.40:0/275411093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f0920ec0 0x55c6f09243d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.306+0000 7f201b757b80 1 --2- 172.21.15.40:0/275411093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f091acf0 0x55c6f0926b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.306+0000 7f201b757b80 1 --2- 172.21.15.40:0/275411093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f09205a0 0x55c6f0923e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.306+0000 7f1ff1ffb700 0 client.25259 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:43:12.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.306+0000 7f201b757b80 1 -- 172.21.15.40:0/275411093 >> 172.21.15.40:0/275411093 conn(0x55c6f0847220 msgr2=0x55c6f0914c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:12.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.306+0000 7f201b757b80 1 -- 172.21.15.40:0/275411093 shutdown_connections 2024-06-06T03:43:12.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.306+0000 7f201b757b80 1 -- 172.21.15.40:0/275411093 wait complete. 2024-06-06T03:43:12.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.306+0000 7f201b757b80 20 client.25259 trim_cache size 0 max 16384 2024-06-06T03:43:12.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.326+0000 7f201b757b80 1 Processor -- start 2024-06-06T03:43:12.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.326+0000 7f201b757b80 1 -- start start 2024-06-06T03:43:12.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.326+0000 7f201b757b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f091acf0 0x55c6f09266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.326+0000 7f201b757b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f09205a0 0x55c6f09153d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.326+0000 7f201b757b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f0920ec0 0x55c6f0915910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f201b757b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1aebc0 con 0x55c6f09205a0 2024-06-06T03:43:12.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f201b757b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1b07f0 con 0x55c6f091acf0 2024-06-06T03:43:12.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f201b757b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1af290 con 0x55c6f0920ec0 2024-06-06T03:43:12.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f2009d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f09205a0 0x55c6f09153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f2009d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f09205a0 0x55c6f09153d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50508/0 (socket says 172.21.15.40:50508) 2024-06-06T03:43:12.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f2009d9b700 1 -- 172.21.15.40:0/228273309 learned_addr learned my addr 172.21.15.40:0/228273309 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:12.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f200a59c700 1 --2- 172.21.15.40:0/228273309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f091acf0 0x55c6f09266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f200ad9d700 1 --2- 172.21.15.40:0/228273309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f0920ec0 0x55c6f0915910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f200a59c700 1 -- 172.21.15.40:0/228273309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f0920ec0 msgr2=0x55c6f0915910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:12.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f200a59c700 1 --2- 172.21.15.40:0/228273309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f0920ec0 0x55c6f0915910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f200a59c700 1 -- 172.21.15.40:0/228273309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f09205a0 msgr2=0x55c6f09153d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:12.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f200a59c700 1 --2- 172.21.15.40:0/228273309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f09205a0 0x55c6f09153d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f200a59c700 1 -- 172.21.15.40:0/228273309 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6f0937050 con 0x55c6f091acf0 2024-06-06T03:43:12.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f2009d9b700 1 --2- 172.21.15.40:0/228273309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f09205a0 0x55c6f09153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:43:12.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f200a59c700 1 --2- 172.21.15.40:0/228273309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f091acf0 0x55c6f09266d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55c6f0937220 tx=0x7f1ffc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=955f03054c0514da server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:12.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f1ff3fff700 1 -- 172.21.15.40:0/228273309 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc012bc0 con 0x55c6f091acf0 2024-06-06T03:43:12.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f1ff3fff700 1 -- 172.21.15.40:0/228273309 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ffc02e070 con 0x55c6f091acf0 2024-06-06T03:43:12.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.327+0000 7f1ff3fff700 1 -- 172.21.15.40:0/228273309 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc020670 con 0x55c6f091acf0 2024-06-06T03:43:12.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.328+0000 7f201b757b80 1 -- 172.21.15.40:0/228273309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f091acf0 msgr2=0x55c6f09266d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:12.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.328+0000 7f201b757b80 1 --2- 172.21.15.40:0/228273309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f091acf0 0x55c6f09266d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55c6f0937220 tx=0x7f1ffc017000 comp rx=0 tx=0).stop 2024-06-06T03:43:12.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.328+0000 7f201b757b80 1 -- 172.21.15.40:0/228273309 shutdown_connections 2024-06-06T03:43:12.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.328+0000 7f201b757b80 1 --2- 172.21.15.40:0/228273309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f0920ec0 0x55c6f0915910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.328+0000 7f201b757b80 1 --2- 172.21.15.40:0/228273309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f091acf0 0x55c6f09266d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.328+0000 7f201b757b80 1 --2- 172.21.15.40:0/228273309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f09205a0 0x55c6f09153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.328+0000 7f201b757b80 1 -- 172.21.15.40:0/228273309 >> 172.21.15.40:0/228273309 conn(0x55c6f0847220 msgr2=0x55c6f091a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:12.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.328+0000 7f201b757b80 1 -- 172.21.15.40:0/228273309 shutdown_connections 2024-06-06T03:43:12.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.328+0000 7f201b757b80 1 -- 172.21.15.40:0/228273309 wait complete. 2024-06-06T03:43:12.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.328+0000 7f201b757b80 1 -- start start 2024-06-06T03:43:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f201b757b80 1 Processor -- start 2024-06-06T03:43:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f201b757b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f091acf0 0x55c6f0929b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f201b757b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f09205a0 0x55c6f092a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f201b757b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f0920ec0 0x55c6f092a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f201b757b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1b0060 con 0x55c6f0920ec0 2024-06-06T03:43:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f201b757b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1af960 con 0x55c6f09205a0 2024-06-06T03:43:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f201b757b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1acbe0 con 0x55c6f091acf0 2024-06-06T03:43:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f201b757b80 10 client.0 _ll_register_callbacks cb 0x55c6f0801750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:43:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f200ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f0920ec0 0x55c6f092a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f200a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f091acf0 0x55c6f0929b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f200ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f0920ec0 0x55c6f092a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50522/0 (socket says 172.21.15.40:50522) 2024-06-06T03:43:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f200ad9d700 1 -- 172.21.15.40:0/1373140040 learned_addr learned my addr 172.21.15.40:0/1373140040 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f2009d9b700 1 --2- 172.21.15.40:0/1373140040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f09205a0 0x55c6f092a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f200ad9d700 1 -- 172.21.15.40:0/1373140040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f091acf0 msgr2=0x55c6f0929b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f200ad9d700 1 --2- 172.21.15.40:0/1373140040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f091acf0 0x55c6f0929b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f200ad9d700 1 -- 172.21.15.40:0/1373140040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f09205a0 msgr2=0x55c6f092a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f200ad9d700 1 --2- 172.21.15.40:0/1373140040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f09205a0 0x55c6f092a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f200ad9d700 1 -- 172.21.15.40:0/1373140040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6f099f3f0 con 0x55c6f0920ec0 2024-06-06T03:43:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f200ad9d700 1 --2- 172.21.15.40:0/1373140040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f0920ec0 0x55c6f092a5b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2004005a30 tx=0x7f200409d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadf90360b6eb232 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.329+0000 7f1ff1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:43:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f1ff1ffb700 1 -- 172.21.15.40:0/1373140040 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20040a5a60 con 0x55c6f0920ec0 2024-06-06T03:43:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f1ff1ffb700 1 -- 172.21.15.40:0/1373140040 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f20040994f0 con 0x55c6f0920ec0 2024-06-06T03:43:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f201b757b80 1 -- 172.21.15.40:0/1373140040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ffc02e070 con 0x55c6f0920ec0 2024-06-06T03:43:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f1ff1ffb700 1 -- 172.21.15.40:0/1373140040 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f200409d180 con 0x55c6f0920ec0 2024-06-06T03:43:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f201b757b80 10 client.15591 fetch_fsmap learned FSMap version 36 2024-06-06T03:43:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f201b757b80 1 -- 172.21.15.40:0/1373140040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c6f099f3f0 con 0x55c6f0920ec0 2024-06-06T03:43:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f1ff1ffb700 1 -- 172.21.15.40:0/1373140040 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ffc02e070 con 0x55c6f0920ec0 2024-06-06T03:43:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f1ff1ffb700 1 -- 172.21.15.40:0/1373140040 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f2004011070 con 0x55c6f0920ec0 2024-06-06T03:43:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f201b757b80 10 client.15591 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T03:43:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f201b757b80 10 client.15591 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:43:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f201b757b80 20 client.15591 populate_metadata read hostname 'smithi040' 2024-06-06T03:43:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f201b757b80 1 --2- 172.21.15.40:0/1373140040 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c6f09a0b20 0x55c6f09a2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f201b757b80 4 client.15591 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}] 2024-06-06T03:43:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f201b757b80 1 -- 172.21.15.40:0/1373140040 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}) -- 0x55c6f0873470 con 0x55c6f09a0b20 2024-06-06T03:43:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.330+0000 7f200a59c700 1 --2- 172.21.15.40:0/1373140040 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c6f09a0b20 0x55c6f09a2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.331+0000 7f200a59c700 1 --2- 172.21.15.40:0/1373140040 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c6f09a0b20 0x55c6f09a2f40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49bad172e92fb678 server_cookie=73a3ba8e35d02eb in_seq=0 out_seq=0 2024-06-06T03:43:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:12.331+0000 7f1ff1ffb700 10 client.15591 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:43:13.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:12 smithi131 ceph-mon[36687]: pgmap v140: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 469 KiB/s wr, 34 op/s 2024-06-06T03:43:13.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:12 smithi040 ceph-mon[32220]: pgmap v140: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 469 KiB/s wr, 34 op/s 2024-06-06T03:43:13.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:12 smithi104 ceph-mon[36569]: pgmap v140: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 469 KiB/s wr, 34 op/s 2024-06-06T03:43:15.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:14 smithi131 ceph-mon[36687]: pgmap v141: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 469 KiB/s wr, 34 op/s 2024-06-06T03:43:15.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:14 smithi040 ceph-mon[32220]: pgmap v141: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 469 KiB/s wr, 34 op/s 2024-06-06T03:43:15.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:14 smithi104 ceph-mon[36569]: pgmap v141: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 469 KiB/s wr, 34 op/s 2024-06-06T03:43:17.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:16 smithi131 ceph-mon[36687]: pgmap v142: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.7 MiB/s wr, 76 op/s 2024-06-06T03:43:17.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:16 smithi040 ceph-mon[32220]: pgmap v142: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.7 MiB/s wr, 76 op/s 2024-06-06T03:43:17.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:16 smithi104 ceph-mon[36569]: pgmap v142: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.7 MiB/s wr, 76 op/s 2024-06-06T03:43:19.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:18 smithi131 ceph-mon[36687]: pgmap v143: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.7 MiB/s wr, 52 op/s 2024-06-06T03:43:19.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:18 smithi040 ceph-mon[32220]: pgmap v143: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.7 MiB/s wr, 52 op/s 2024-06-06T03:43:19.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:18 smithi104 ceph-mon[36569]: pgmap v143: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.7 MiB/s wr, 52 op/s 2024-06-06T03:43:21.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:20 smithi131 ceph-mon[36687]: pgmap v144: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 138 op/s 2024-06-06T03:43:21.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:20 smithi040 ceph-mon[32220]: pgmap v144: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 138 op/s 2024-06-06T03:43:21.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:20 smithi104 ceph-mon[36569]: pgmap v144: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 138 op/s 2024-06-06T03:43:23.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:22 smithi131 ceph-mon[36687]: pgmap v145: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 127 op/s 2024-06-06T03:43:23.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:22 smithi040 ceph-mon[32220]: pgmap v145: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 127 op/s 2024-06-06T03:43:23.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:22 smithi104 ceph-mon[36569]: pgmap v145: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 127 op/s 2024-06-06T03:43:25.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:24 smithi131 ceph-mon[36687]: pgmap v146: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 127 op/s 2024-06-06T03:43:25.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:24 smithi040 ceph-mon[32220]: pgmap v146: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 127 op/s 2024-06-06T03:43:25.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:24 smithi104 ceph-mon[36569]: pgmap v146: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 127 op/s 2024-06-06T03:43:27.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:26 smithi131 ceph-mon[36687]: pgmap v147: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.6 MiB/s wr, 513 op/s 2024-06-06T03:43:27.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:26 smithi040 ceph-mon[32220]: pgmap v147: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.6 MiB/s wr, 513 op/s 2024-06-06T03:43:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:26 smithi104 ceph-mon[36569]: pgmap v147: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.6 MiB/s wr, 513 op/s 2024-06-06T03:43:29.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:28 smithi131 ceph-mon[36687]: pgmap v148: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 4.4 MiB/s wr, 471 op/s 2024-06-06T03:43:29.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:28 smithi040 ceph-mon[32220]: pgmap v148: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 4.4 MiB/s wr, 471 op/s 2024-06-06T03:43:29.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:28 smithi104 ceph-mon[36569]: pgmap v148: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 4.4 MiB/s wr, 471 op/s 2024-06-06T03:43:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:30 smithi131 ceph-mon[36687]: pgmap v149: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.3 MiB/s wr, 877 op/s 2024-06-06T03:43:31.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:30 smithi040 ceph-mon[32220]: pgmap v149: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.3 MiB/s wr, 877 op/s 2024-06-06T03:43:31.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:30 smithi104 ceph-mon[36569]: pgmap v149: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.3 MiB/s wr, 877 op/s 2024-06-06T03:43:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:33 smithi131 ceph-mon[36687]: pgmap v150: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.0 MiB/s wr, 791 op/s 2024-06-06T03:43:33.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:33 smithi040 ceph-mon[32220]: pgmap v150: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.0 MiB/s wr, 791 op/s 2024-06-06T03:43:33.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:33 smithi104 ceph-mon[36569]: pgmap v150: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.0 MiB/s wr, 791 op/s 2024-06-06T03:43:33.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f1ff1ffb700 1 -- 172.21.15.40:0/1373140040 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c6f0873470 con 0x55c6f09a0b20 2024-06-06T03:43:33.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f1ff1ffb700 10 client.15591 handle_command_reply: tid=0 2024-06-06T03:43:33.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f201b757b80 1 client.15591 shutdown 2024-06-06T03:43:33.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f201b757b80 1 -- 172.21.15.40:0/1373140040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f0920ec0 msgr2=0x55c6f092a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:33.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f201b757b80 1 --2- 172.21.15.40:0/1373140040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f0920ec0 0x55c6f092a5b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2004005a30 tx=0x7f200409d6b0 comp rx=0 tx=0).stop 2024-06-06T03:43:33.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f201b757b80 1 -- 172.21.15.40:0/1373140040 shutdown_connections 2024-06-06T03:43:33.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f201b757b80 1 --2- 172.21.15.40:0/1373140040 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55c6f09a0b20 0x55c6f09a2f40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:33.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f201b757b80 1 --2- 172.21.15.40:0/1373140040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c6f091acf0 0x55c6f0929b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:33.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f201b757b80 1 --2- 172.21.15.40:0/1373140040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c6f0920ec0 0x55c6f092a5b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:33.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f1ff1ffb700 0 client.15591 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:43:33.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f201b757b80 1 --2- 172.21.15.40:0/1373140040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6f09205a0 0x55c6f092a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:33.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f201b757b80 1 -- 172.21.15.40:0/1373140040 >> 172.21.15.40:0/1373140040 conn(0x55c6f0847220 msgr2=0x55c6f09197a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:33.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f201b757b80 1 -- 172.21.15.40:0/1373140040 shutdown_connections 2024-06-06T03:43:33.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f201b757b80 1 -- 172.21.15.40:0/1373140040 wait complete. 2024-06-06T03:43:33.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.522+0000 7f201b757b80 20 client.15591 trim_cache size 0 max 16384 2024-06-06T03:43:33.524 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:43:33.524 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 36, 2024-06-06T03:43:33.524 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:43:33.525 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 14, 2024-06-06T03:43:33.525 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": { 2024-06-06T03:43:33.525 INFO:teuthology.orchestra.run.smithi040.stdout: "ad9f5028": { 2024-06-06T03:43:33.525 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 14, 2024-06-06T03:43:33.525 INFO:teuthology.orchestra.run.smithi040.stdout: "age_ref": 0.0, 2024-06-06T03:43:33.525 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:43:33.525 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "QUIESCED", 2024-06-06T03:43:33.525 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:43:33.525 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:43:33.525 INFO:teuthology.orchestra.run.smithi040.stdout: "timeout": 90.0, 2024-06-06T03:43:33.525 INFO:teuthology.orchestra.run.smithi040.stdout: "expiration": 145.2, 2024-06-06T03:43:33.525 INFO:teuthology.orchestra.run.smithi040.stdout: "members": { 2024-06-06T03:43:33.525 INFO:teuthology.orchestra.run.smithi040.stdout: "file:/": { 2024-06-06T03:43:33.526 INFO:teuthology.orchestra.run.smithi040.stdout: "excluded": false, 2024-06-06T03:43:33.526 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:43:33.526 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "QUIESCED", 2024-06-06T03:43:33.526 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:43:33.526 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:33.526 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:33.526 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:33.526 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:33.526 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:33.526 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:43:33.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.523+0000 7f2010ed4700 1 -- 172.21.15.40:0/3247528350 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f1ff40a2eb0 msgr2=0x7f1ff40a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:33.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.523+0000 7f2010ed4700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f1ff40a2eb0 0x7f1ff40a5370 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009bf0 tx=0x7f1ffc009260 comp rx=0 tx=0).stop 2024-06-06T03:43:33.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.523+0000 7f2010ed4700 1 -- 172.21.15.40:0/3247528350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f200c107790 msgr2=0x7f200c19f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:33.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.523+0000 7f2010ed4700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f200c107790 0x7f200c19f5e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2004006070 tx=0x7f2004007a20 comp rx=0 tx=0).stop 2024-06-06T03:43:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.523+0000 7f2010ed4700 1 -- 172.21.15.40:0/3247528350 shutdown_connections 2024-06-06T03:43:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.523+0000 7f2010ed4700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f1ff40a2eb0 0x7f1ff40a5370 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.523+0000 7f2010ed4700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200c1027a0 0x7f200c19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.523+0000 7f2010ed4700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f200c107790 0x7f200c19f5e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.523+0000 7f2010ed4700 1 --2- 172.21.15.40:0/3247528350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f200c103150 0x7f200c19b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.523+0000 7f2010ed4700 1 -- 172.21.15.40:0/3247528350 >> 172.21.15.40:0/3247528350 conn(0x7f200c079560 msgr2=0x7f200c1006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.523+0000 7f2010ed4700 1 -- 172.21.15.40:0/3247528350 shutdown_connections 2024-06-06T03:43:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:33.523+0000 7f2010ed4700 1 -- 172.21.15.40:0/3247528350 wait complete. 2024-06-06T03:43:34.351 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ad9f5028', quiesce took 27.6 seconds. Will release after: -2.3500000000000014 2024-06-06T03:43:34.352 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:43:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:43:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:35 smithi131 ceph-mon[36687]: pgmap v151: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.0 MiB/s wr, 791 op/s 2024-06-06T03:43:35.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:35 smithi040 ceph-mon[32220]: pgmap v151: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.0 MiB/s wr, 791 op/s 2024-06-06T03:43:35.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:35 smithi104 ceph-mon[36569]: pgmap v151: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.0 MiB/s wr, 791 op/s 2024-06-06T03:43:36.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.429+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1625246468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0102740 msgr2=0x7f35f0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:36.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.429+0000 7f35f5ff6700 1 --2- 172.21.15.40:0/1625246468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0102740 0x7f35f0102b20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f35e000b0d0 tx=0x7f35e000b3e0 comp rx=0 tx=0).stop 2024-06-06T03:43:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.429+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1625246468 shutdown_connections 2024-06-06T03:43:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.429+0000 7f35f5ff6700 1 --2- 172.21.15.40:0/1625246468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35f0107840 0x7f35f010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.429+0000 7f35f5ff6700 1 --2- 172.21.15.40:0/1625246468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0102740 0x7f35f0102b20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.429+0000 7f35f5ff6700 1 --2- 172.21.15.40:0/1625246468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35f01030f0 0x7f35f0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.429+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1625246468 >> 172.21.15.40:0/1625246468 conn(0x7f35f00795f0 msgr2=0x7f35f0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.429+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1625246468 shutdown_connections 2024-06-06T03:43:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.429+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1625246468 wait complete. 2024-06-06T03:43:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.430+0000 7f35f5ff6700 1 Processor -- start 2024-06-06T03:43:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.430+0000 7f35f5ff6700 1 -- start start 2024-06-06T03:43:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.430+0000 7f35f5ff6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0102740 0x7f35f019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.430+0000 7f35f5ff6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35f01030f0 0x7f35f019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.430+0000 7f35f5ff6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35f0107840 0x7f35f01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.430+0000 7f35f5ff6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0112780 con 0x7f35f0102740 2024-06-06T03:43:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.430+0000 7f35f5ff6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0112600 con 0x7f35f0107840 2024-06-06T03:43:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.430+0000 7f35f5ff6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0112900 con 0x7f35f01030f0 2024-06-06T03:43:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.430+0000 7f35ef7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0102740 0x7f35f019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.430+0000 7f35ef7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0102740 0x7f35f019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46518/0 (socket says 172.21.15.40:46518) 2024-06-06T03:43:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.430+0000 7f35ef7fe700 1 -- 172.21.15.40:0/1215148968 learned_addr learned my addr 172.21.15.40:0/1215148968 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35effff700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35f0107840 0x7f35f01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35eeffd700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35f01030f0 0x7f35f019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35ef7fe700 1 -- 172.21.15.40:0/1215148968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35f01030f0 msgr2=0x7f35f019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35ef7fe700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35f01030f0 0x7f35f019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35ef7fe700 1 -- 172.21.15.40:0/1215148968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35f0107840 msgr2=0x7f35f01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35ef7fe700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35f0107840 0x7f35f01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35ef7fe700 1 -- 172.21.15.40:0/1215148968 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f0199100 con 0x7f35f0102740 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35eeffd700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35f01030f0 0x7f35f019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35effff700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35f0107840 0x7f35f01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35ef7fe700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0102740 0x7f35f019e790 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f35e000b820 tx=0x7f35e0004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35ecff9700 1 -- 172.21.15.40:0/1215148968 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0021020 con 0x7f35f0102740 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35ecff9700 1 -- 172.21.15.40:0/1215148968 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e0007d40 con 0x7f35f0102740 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1215148968 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f0199390 con 0x7f35f0102740 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35ecff9700 1 -- 172.21.15.40:0/1215148968 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0005370 con 0x7f35f0102740 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.431+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1215148968 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f01a5660 con 0x7f35f0102740 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.432+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1215148968 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f0069180 con 0x7f35f0102740 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.434+0000 7f35ecff9700 1 -- 172.21.15.40:0/1215148968 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f35e0024020 con 0x7f35f0102740 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.435+0000 7f35ecff9700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f35dc0a2d90 0x7f35dc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.435+0000 7f35eeffd700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f35dc0a2d90 0x7f35dc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.435+0000 7f35ecff9700 1 -- 172.21.15.40:0/1215148968 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f35e000f020 con 0x7f35f0102740 2024-06-06T03:43:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.435+0000 7f35eeffd700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f35dc0a2d90 0x7f35dc0a5250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f35d8005fd0 tx=0x7f35d8005e30 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:36.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.439+0000 7f35ecff9700 1 -- 172.21.15.40:0/1215148968 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f35e009dc60 con 0x7f35f0102740 2024-06-06T03:43:36.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.692+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1215148968 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35f002dca0 con 0x7f35f0102740 2024-06-06T03:43:36.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.693+0000 7f35ecff9700 1 -- 172.21.15.40:0/1215148968 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11941 (secure 0 0 0) 0x7f35e009d560 con 0x7f35f0102740 2024-06-06T03:43:36.693 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:43:36.693 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:43:05.692670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24398,24403,24425,24443,24470]},"id":1}]} 2024-06-06T03:43:36.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.695+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1215148968 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f35dc0a2d90 msgr2=0x7f35dc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:36.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.695+0000 7f35f5ff6700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f35dc0a2d90 0x7f35dc0a5250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f35d8005fd0 tx=0x7f35d8005e30 comp rx=0 tx=0).stop 2024-06-06T03:43:36.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.695+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1215148968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0102740 msgr2=0x7f35f019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:36.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.695+0000 7f35f5ff6700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0102740 0x7f35f019e790 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f35e000b820 tx=0x7f35e0004e60 comp rx=0 tx=0).stop 2024-06-06T03:43:36.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.695+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1215148968 shutdown_connections 2024-06-06T03:43:36.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.695+0000 7f35f5ff6700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f35dc0a2d90 0x7f35dc0a5250 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:36.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.695+0000 7f35f5ff6700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35f0107840 0x7f35f01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:36.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.695+0000 7f35f5ff6700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0102740 0x7f35f019e790 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:36.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.695+0000 7f35f5ff6700 1 --2- 172.21.15.40:0/1215148968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35f01030f0 0x7f35f019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:36.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.695+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1215148968 >> 172.21.15.40:0/1215148968 conn(0x7f35f00795f0 msgr2=0x7f35f0100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:36.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.695+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1215148968 shutdown_connections 2024-06-06T03:43:36.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:36.695+0000 7f35f5ff6700 1 -- 172.21.15.40:0/1215148968 wait complete. 2024-06-06T03:43:36.700 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 36 2024-06-06T03:43:37.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:37 smithi131 ceph-mon[36687]: pgmap v152: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-06-06T03:43:37.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:37 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1215148968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:43:37.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:37 smithi040 ceph-mon[32220]: pgmap v152: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-06-06T03:43:37.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:37 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1215148968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:43:37.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:37 smithi104 ceph-mon[36569]: pgmap v152: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-06-06T03:43:37.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:37 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1215148968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:43:37.377 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id ad9f5028 --release --await' 2024-06-06T03:43:37.378 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id ad9f5028 --release --await 2024-06-06T03:43:37.628 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:43:39.214 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:39 smithi040 ceph-mon[32220]: pgmap v153: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.9 MiB/s wr, 672 op/s 2024-06-06T03:43:39.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:39 smithi131 ceph-mon[36687]: pgmap v153: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.9 MiB/s wr, 672 op/s 2024-06-06T03:43:39.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:39 smithi104 ceph-mon[36569]: pgmap v153: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.9 MiB/s wr, 672 op/s 2024-06-06T03:43:39.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.502+0000 7f94032d2700 1 -- 172.21.15.40:0/2816776541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93fc1030f0 msgr2=0x7f93fc107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:39.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.502+0000 7f94032d2700 1 --2- 172.21.15.40:0/2816776541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93fc1030f0 0x7f93fc107110 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0099c0 tx=0x7f93ec009cd0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.502+0000 7f94032d2700 1 -- 172.21.15.40:0/2816776541 shutdown_connections 2024-06-06T03:43:39.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.502+0000 7f94032d2700 1 --2- 172.21.15.40:0/2816776541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93fc102740 0x7f93fc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.502+0000 7f94032d2700 1 --2- 172.21.15.40:0/2816776541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93fc107840 0x7f93fc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.502+0000 7f94032d2700 1 --2- 172.21.15.40:0/2816776541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93fc1030f0 0x7f93fc107110 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.502+0000 7f94032d2700 1 -- 172.21.15.40:0/2816776541 >> 172.21.15.40:0/2816776541 conn(0x7f93fc0795f0 msgr2=0x7f93fc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:39.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.502+0000 7f94032d2700 1 -- 172.21.15.40:0/2816776541 shutdown_connections 2024-06-06T03:43:39.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.502+0000 7f94032d2700 1 -- 172.21.15.40:0/2816776541 wait complete. 2024-06-06T03:43:39.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f94032d2700 1 Processor -- start 2024-06-06T03:43:39.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f94032d2700 1 -- start start 2024-06-06T03:43:39.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f94032d2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93fc102740 0x7f93fc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f94032d2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93fc1030f0 0x7f93fc074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f94032d2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93fc107840 0x7f93fc075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f94032d2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc112770 con 0x7f93fc102740 2024-06-06T03:43:39.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f94032d2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1125f0 con 0x7f93fc1030f0 2024-06-06T03:43:39.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f94032d2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1128f0 con 0x7f93fc107840 2024-06-06T03:43:39.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f940106e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93fc102740 0x7f93fc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f940106e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93fc102740 0x7f93fc074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58804/0 (socket says 172.21.15.40:58804) 2024-06-06T03:43:39.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f940106e700 1 -- 172.21.15.40:0/505177727 learned_addr learned my addr 172.21.15.40:0/505177727 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:39.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f940086d700 1 --2- 172.21.15.40:0/505177727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93fc1030f0 0x7f93fc074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f940186f700 1 --2- 172.21.15.40:0/505177727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93fc107840 0x7f93fc075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f940106e700 1 -- 172.21.15.40:0/505177727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93fc107840 msgr2=0x7f93fc075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f940106e700 1 --2- 172.21.15.40:0/505177727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93fc107840 0x7f93fc075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f940106e700 1 -- 172.21.15.40:0/505177727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93fc1030f0 msgr2=0x7f93fc074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f940106e700 1 --2- 172.21.15.40:0/505177727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93fc1030f0 0x7f93fc074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.503+0000 7f940106e700 1 -- 172.21.15.40:0/505177727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93fc1a7180 con 0x7f93fc102740 2024-06-06T03:43:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.504+0000 7f940106e700 1 --2- 172.21.15.40:0/505177727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93fc102740 0x7f93fc074790 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f93f800d980 tx=0x7f93f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.504+0000 7f93f27fc700 1 -- 172.21.15.40:0/505177727 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8017070 con 0x7f93fc102740 2024-06-06T03:43:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.504+0000 7f93f27fc700 1 -- 172.21.15.40:0/505177727 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f8005020 con 0x7f93fc102740 2024-06-06T03:43:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.504+0000 7f94032d2700 1 -- 172.21.15.40:0/505177727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93fc1a7470 con 0x7f93fc102740 2024-06-06T03:43:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.504+0000 7f93f27fc700 1 -- 172.21.15.40:0/505177727 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f800fbf0 con 0x7f93fc102740 2024-06-06T03:43:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.504+0000 7f94032d2700 1 -- 172.21.15.40:0/505177727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93fc1a7980 con 0x7f93fc102740 2024-06-06T03:43:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.506+0000 7f93f27fc700 1 -- 172.21.15.40:0/505177727 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f93f80051d0 con 0x7f93fc102740 2024-06-06T03:43:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.507+0000 7f93f27fc700 1 --2- 172.21.15.40:0/505177727 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f93e80a2e10 0x7f93e80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.507+0000 7f940086d700 1 --2- 172.21.15.40:0/505177727 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f93e80a2e10 0x7f93e80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.507+0000 7f93f27fc700 1 -- 172.21.15.40:0/505177727 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f93f80cf1f0 con 0x7f93fc102740 2024-06-06T03:43:39.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.508+0000 7f940086d700 1 --2- 172.21.15.40:0/505177727 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f93e80a2e10 0x7f93e80a52d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0761d0 tx=0x7f93ec005fb0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:39.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.508+0000 7f940db55b80 1 Processor -- start 2024-06-06T03:43:39.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.508+0000 7f940db55b80 1 -- start start 2024-06-06T03:43:39.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.508+0000 7f940db55b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603500 0x560db66038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.508+0000 7f940db55b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603e20 0x560db65fd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.508+0000 7f940db55b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db65fe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.508+0000 7f940db55b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560db64f0060 con 0x560db65fdc50 2024-06-06T03:43:39.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.508+0000 7f940db55b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560db654e950 con 0x560db6603e20 2024-06-06T03:43:39.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.508+0000 7f940db55b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560db64b6560 con 0x560db6603500 2024-06-06T03:43:39.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.508+0000 7f940186f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db65fe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.509+0000 7f940186f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db65fe070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58814/0 (socket says 172.21.15.40:58814) 2024-06-06T03:43:39.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.509+0000 7f940186f700 1 -- 172.21.15.40:0/2973708721 learned_addr learned my addr 172.21.15.40:0/2973708721 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:39.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.509+0000 7f940086d700 1 --2- 172.21.15.40:0/2973708721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603e20 0x560db65fd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.509+0000 7f940106e700 1 --2- 172.21.15.40:0/2973708721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603500 0x560db66038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.509+0000 7f940186f700 1 -- 172.21.15.40:0/2973708721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603500 msgr2=0x560db66038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:39.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.509+0000 7f940186f700 1 --2- 172.21.15.40:0/2973708721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603500 0x560db66038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.509+0000 7f940186f700 1 -- 172.21.15.40:0/2973708721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603e20 msgr2=0x560db65fd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:39.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.509+0000 7f940186f700 1 --2- 172.21.15.40:0/2973708721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603e20 0x560db65fd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.509+0000 7f940186f700 1 -- 172.21.15.40:0/2973708721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560db649d9d0 con 0x560db65fdc50 2024-06-06T03:43:39.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.509+0000 7f940086d700 1 --2- 172.21.15.40:0/2973708721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603e20 0x560db65fd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:43:39.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.509+0000 7f940186f700 1 --2- 172.21.15.40:0/2973708721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db65fe070 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f93f4003f30 tx=0x7f93f4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1493fdc5b14412 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:39.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.509+0000 7f93e77fe700 1 -- 172.21.15.40:0/2973708721 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4004670 con 0x560db65fdc50 2024-06-06T03:43:39.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.509+0000 7f93e77fe700 1 -- 172.21.15.40:0/2973708721 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f401a070 con 0x560db65fdc50 2024-06-06T03:43:39.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.509+0000 7f93e77fe700 1 -- 172.21.15.40:0/2973708721 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40156e0 con 0x560db65fdc50 2024-06-06T03:43:39.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.510+0000 7f940db55b80 1 -- 172.21.15.40:0/2973708721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 msgr2=0x560db65fe070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:39.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.510+0000 7f940db55b80 1 --2- 172.21.15.40:0/2973708721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db65fe070 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f93f4003f30 tx=0x7f93f4009c20 comp rx=0 tx=0).stop 2024-06-06T03:43:39.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.510+0000 7f940db55b80 1 -- 172.21.15.40:0/2973708721 shutdown_connections 2024-06-06T03:43:39.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.510+0000 7f940db55b80 1 --2- 172.21.15.40:0/2973708721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603500 0x560db66038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.510+0000 7f940db55b80 1 --2- 172.21.15.40:0/2973708721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db65fe070 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.510+0000 7f940db55b80 1 --2- 172.21.15.40:0/2973708721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603e20 0x560db65fd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.510+0000 7f940db55b80 1 -- 172.21.15.40:0/2973708721 >> 172.21.15.40:0/2973708721 conn(0x560db652a180 msgr2=0x560db65f8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:39.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.510+0000 7f940db55b80 1 -- 172.21.15.40:0/2973708721 shutdown_connections 2024-06-06T03:43:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.510+0000 7f940db55b80 1 -- 172.21.15.40:0/2973708721 wait complete. 2024-06-06T03:43:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.511+0000 7f940db55b80 1 -- start start 2024-06-06T03:43:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.511+0000 7f940db55b80 1 Processor -- start 2024-06-06T03:43:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.511+0000 7f940db55b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db65fdc50 0x560db6609b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.511+0000 7f940db55b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db6603500 0x560db6606e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.511+0000 7f940db55b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 0x560db66073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.511+0000 7f940db55b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560db64ef0c0 con 0x560db6603500 2024-06-06T03:43:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.511+0000 7f940db55b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560db654a910 con 0x560db65fdc50 2024-06-06T03:43:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.511+0000 7f940db55b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560db660a800 con 0x560db6603e20 2024-06-06T03:43:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.511+0000 7f940db55b80 10 client.0 _ll_register_callbacks cb 0x560db64e46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:43:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.511+0000 7f940086d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db6603500 0x560db6606e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.511+0000 7f940086d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db6603500 0x560db6606e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58816/0 (socket says 172.21.15.40:58816) 2024-06-06T03:43:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.511+0000 7f940086d700 1 -- 172.21.15.40:0/3675962093 learned_addr learned my addr 172.21.15.40:0/3675962093 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.511+0000 7f940106e700 1 --2- 172.21.15.40:0/3675962093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db65fdc50 0x560db6609b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.512+0000 7f940186f700 1 --2- 172.21.15.40:0/3675962093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 0x560db66073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.512+0000 7f940086d700 1 -- 172.21.15.40:0/3675962093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 msgr2=0x560db66073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.512+0000 7f940086d700 1 --2- 172.21.15.40:0/3675962093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 0x560db66073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.512+0000 7f940086d700 1 -- 172.21.15.40:0/3675962093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db65fdc50 msgr2=0x560db6609b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.512+0000 7f940086d700 1 --2- 172.21.15.40:0/3675962093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db65fdc50 0x560db6609b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.512+0000 7f940086d700 1 -- 172.21.15.40:0/3675962093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560db6514850 con 0x560db6603500 2024-06-06T03:43:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.512+0000 7f940086d700 1 --2- 172.21.15.40:0/3675962093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db6603500 0x560db6606e60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0099c0 tx=0x7f93ec020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce715461ab1a65e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.512+0000 7f93e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:43:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.512+0000 7f93e4ff9700 1 -- 172.21.15.40:0/3675962093 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec032630 con 0x560db6603500 2024-06-06T03:43:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.512+0000 7f93e4ff9700 1 -- 172.21.15.40:0/3675962093 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f93ec024410 con 0x560db6603500 2024-06-06T03:43:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.512+0000 7f940db55b80 1 -- 172.21.15.40:0/3675962093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f401a070 con 0x560db6603500 2024-06-06T03:43:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.512+0000 7f93e4ff9700 1 -- 172.21.15.40:0/3675962093 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f93ec00be70 con 0x560db6603500 2024-06-06T03:43:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.512+0000 7f940db55b80 10 client.15609 fetch_fsmap learned FSMap version 36 2024-06-06T03:43:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.512+0000 7f940db55b80 1 -- 172.21.15.40:0/3675962093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560db6614f40 con 0x560db6603500 2024-06-06T03:43:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.513+0000 7f93e4ff9700 1 -- 172.21.15.40:0/3675962093 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f401a070 con 0x560db6603500 2024-06-06T03:43:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.513+0000 7f93e4ff9700 1 -- 172.21.15.40:0/3675962093 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f93ec024b10 con 0x560db6603500 2024-06-06T03:43:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.513+0000 7f940db55b80 10 client.15609 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T03:43:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.513+0000 7f940db55b80 10 client.15609 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:43:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.513+0000 7f940db55b80 20 client.15609 populate_metadata read hostname 'smithi040' 2024-06-06T03:43:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.513+0000 7f940db55b80 1 --2- 172.21.15.40:0/3675962093 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560db66175d0 0x560db6619a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.513+0000 7f940db55b80 4 client.15609 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:43:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.513+0000 7f940db55b80 1 -- 172.21.15.40:0/3675962093 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560db655be50 con 0x560db66175d0 2024-06-06T03:43:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.513+0000 7f940106e700 1 --2- 172.21.15.40:0/3675962093 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560db66175d0 0x560db6619a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.514+0000 7f940106e700 1 --2- 172.21.15.40:0/3675962093 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560db66175d0 0x560db6619a20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee426cc85ce1d1fd server_cookie=183b4ec55a7536c in_seq=0 out_seq=0 2024-06-06T03:43:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.514+0000 7f93e4ff9700 10 client.15609 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:43:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.514+0000 7f93e4ff9700 1 -- 172.21.15.40:0/3675962093 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560db655be50 con 0x560db66175d0 2024-06-06T03:43:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.514+0000 7f93e4ff9700 10 client.15609 handle_command_reply: tid=0 2024-06-06T03:43:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.514+0000 7f940db55b80 1 client.15609 shutdown 2024-06-06T03:43:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.515+0000 7f940db55b80 1 -- 172.21.15.40:0/3675962093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db6603500 msgr2=0x560db6606e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.515+0000 7f940db55b80 1 --2- 172.21.15.40:0/3675962093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db6603500 0x560db6606e60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0099c0 tx=0x7f93ec020000 comp rx=0 tx=0).stop 2024-06-06T03:43:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.515+0000 7f940db55b80 1 -- 172.21.15.40:0/3675962093 shutdown_connections 2024-06-06T03:43:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.515+0000 7f940db55b80 1 --2- 172.21.15.40:0/3675962093 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560db66175d0 0x560db6619a20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.515+0000 7f940db55b80 1 --2- 172.21.15.40:0/3675962093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 0x560db66073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.515+0000 7f940db55b80 1 --2- 172.21.15.40:0/3675962093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db65fdc50 0x560db6609b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.515+0000 7f93e4ff9700 0 client.15609 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:43:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.515+0000 7f940db55b80 1 --2- 172.21.15.40:0/3675962093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db6603500 0x560db6606e60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.515+0000 7f940db55b80 1 -- 172.21.15.40:0/3675962093 >> 172.21.15.40:0/3675962093 conn(0x560db652a180 msgr2=0x560db65f7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.515+0000 7f940db55b80 1 -- 172.21.15.40:0/3675962093 shutdown_connections 2024-06-06T03:43:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.515+0000 7f940db55b80 1 -- 172.21.15.40:0/3675962093 wait complete. 2024-06-06T03:43:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.515+0000 7f940db55b80 20 client.15609 trim_cache size 0 max 16384 2024-06-06T03:43:39.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.535+0000 7f940db55b80 1 Processor -- start 2024-06-06T03:43:39.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.535+0000 7f940db55b80 1 -- start start 2024-06-06T03:43:39.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940db55b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db65fd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940db55b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603500 0x560db6607490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940db55b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 0x560db6604a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940db55b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1aee30 con 0x560db65fdc50 2024-06-06T03:43:39.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940db55b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1b0a60 con 0x560db6603500 2024-06-06T03:43:39.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940db55b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1af500 con 0x560db6603e20 2024-06-06T03:43:39.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940106e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db65fd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940106e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db65fd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58832/0 (socket says 172.21.15.40:58832) 2024-06-06T03:43:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940106e700 1 -- 172.21.15.40:0/3464215657 learned_addr learned my addr 172.21.15.40:0/3464215657 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940186f700 1 --2- 172.21.15.40:0/3464215657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 0x560db6604a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940106e700 1 -- 172.21.15.40:0/3464215657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 msgr2=0x560db6604a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940106e700 1 --2- 172.21.15.40:0/3464215657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 0x560db6604a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940106e700 1 -- 172.21.15.40:0/3464215657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603500 msgr2=0x560db6607490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940086d700 1 --2- 172.21.15.40:0/3464215657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603500 0x560db6607490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940106e700 1 --2- 172.21.15.40:0/3464215657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603500 0x560db6607490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940106e700 1 -- 172.21.15.40:0/3464215657 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560db6606c30 con 0x560db65fdc50 2024-06-06T03:43:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f940106e700 1 --2- 172.21.15.40:0/3464215657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db65fd710 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f93f8097ba0 tx=0x7f93f809b1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8bb7164914326f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f93e6ffd700 1 -- 172.21.15.40:0/3464215657 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f80997e0 con 0x560db65fdc50 2024-06-06T03:43:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f93e6ffd700 1 -- 172.21.15.40:0/3464215657 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f8015040 con 0x560db65fdc50 2024-06-06T03:43:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.536+0000 7f93e6ffd700 1 -- 172.21.15.40:0/3464215657 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f809b9a0 con 0x560db65fdc50 2024-06-06T03:43:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.537+0000 7f940db55b80 1 -- 172.21.15.40:0/3464215657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 msgr2=0x560db65fd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.537+0000 7f940db55b80 1 --2- 172.21.15.40:0/3464215657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db65fd710 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f93f8097ba0 tx=0x7f93f809b1c0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.537+0000 7f940db55b80 1 -- 172.21.15.40:0/3464215657 shutdown_connections 2024-06-06T03:43:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.537+0000 7f940db55b80 1 --2- 172.21.15.40:0/3464215657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 0x560db6604a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.537+0000 7f940db55b80 1 --2- 172.21.15.40:0/3464215657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db65fd710 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.537+0000 7f940db55b80 1 --2- 172.21.15.40:0/3464215657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603500 0x560db6607490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.537+0000 7f940db55b80 1 -- 172.21.15.40:0/3464215657 >> 172.21.15.40:0/3464215657 conn(0x560db652a180 msgr2=0x560db6608640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.537+0000 7f940db55b80 1 -- 172.21.15.40:0/3464215657 shutdown_connections 2024-06-06T03:43:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.537+0000 7f940db55b80 1 -- 172.21.15.40:0/3464215657 wait complete. 2024-06-06T03:43:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.538+0000 7f940db55b80 1 -- start start 2024-06-06T03:43:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.538+0000 7f940db55b80 1 Processor -- start 2024-06-06T03:43:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.538+0000 7f940db55b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db6610040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.538+0000 7f940db55b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603500 0x560db660cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.538+0000 7f940db55b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 0x560db660d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.538+0000 7f940db55b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1b02d0 con 0x560db65fdc50 2024-06-06T03:43:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.538+0000 7f940db55b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1afbd0 con 0x560db6603500 2024-06-06T03:43:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.538+0000 7f940db55b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1ace30 con 0x560db6603e20 2024-06-06T03:43:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.538+0000 7f940106e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db6610040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.538+0000 7f940db55b80 10 client.0 _ll_register_callbacks cb 0x560db64e46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.538+0000 7f940106e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db6610040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58838/0 (socket says 172.21.15.40:58838) 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.538+0000 7f940106e700 1 -- 172.21.15.40:0/2371918232 learned_addr learned my addr 172.21.15.40:0/2371918232 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.538+0000 7f940086d700 1 --2- 172.21.15.40:0/2371918232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603500 0x560db660cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f940186f700 1 --2- 172.21.15.40:0/2371918232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 0x560db660d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f940106e700 1 -- 172.21.15.40:0/2371918232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 msgr2=0x560db660d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f940106e700 1 --2- 172.21.15.40:0/2371918232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 0x560db660d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f940106e700 1 -- 172.21.15.40:0/2371918232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603500 msgr2=0x560db660cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f940106e700 1 --2- 172.21.15.40:0/2371918232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603500 0x560db660cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f940106e700 1 -- 172.21.15.40:0/2371918232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560db660d5c0 con 0x560db65fdc50 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f940106e700 1 --2- 172.21.15.40:0/2371918232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db6610040 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f93f8086010 tx=0x7f93f80a4470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3917e9263d38d67e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f93e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f93e4ff9700 1 -- 172.21.15.40:0/2371918232 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f80ba050 con 0x560db65fdc50 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f93e4ff9700 1 -- 172.21.15.40:0/2371918232 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8382+0+0 (secure 0 0 0) 0x7f93f8097db0 con 0x560db65fdc50 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f940db55b80 1 -- 172.21.15.40:0/2371918232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f8015040 con 0x560db65fdc50 2024-06-06T03:43:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f93e4ff9700 1 -- 172.21.15.40:0/2371918232 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f93f80a9050 con 0x560db65fdc50 2024-06-06T03:43:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f940db55b80 10 client.15615 fetch_fsmap learned FSMap version 36 2024-06-06T03:43:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f940db55b80 1 -- 172.21.15.40:0/2371918232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560db6683120 con 0x560db65fdc50 2024-06-06T03:43:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.539+0000 7f93e4ff9700 1 -- 172.21.15.40:0/2371918232 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f8015040 con 0x560db65fdc50 2024-06-06T03:43:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.540+0000 7f93e4ff9700 1 -- 172.21.15.40:0/2371918232 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f93f809b9a0 con 0x560db65fdc50 2024-06-06T03:43:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.540+0000 7f940db55b80 10 client.15615 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T03:43:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.540+0000 7f940db55b80 10 client.15615 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:43:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.540+0000 7f940db55b80 20 client.15615 populate_metadata read hostname 'smithi040' 2024-06-06T03:43:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.540+0000 7f940db55b80 1 --2- 172.21.15.40:0/2371918232 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560db6683800 0x560db6685c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:43:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.540+0000 7f940db55b80 4 client.15615 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ad9f5028", "release": true, "await": true}] 2024-06-06T03:43:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.540+0000 7f940db55b80 1 -- 172.21.15.40:0/2371918232 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ad9f5028", "release": true, "await": true}) -- 0x560db65563d0 con 0x560db6683800 2024-06-06T03:43:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.540+0000 7f940086d700 1 --2- 172.21.15.40:0/2371918232 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560db6683800 0x560db6685c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:43:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.540+0000 7f940086d700 1 --2- 172.21.15.40:0/2371918232 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560db6683800 0x560db6685c20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6c09975a236e93c server_cookie=c67b681a85f544f in_seq=0 out_seq=0 2024-06-06T03:43:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:39.540+0000 7f93e4ff9700 10 client.15615 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:43:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.560+0000 7f93e4ff9700 1 -- 172.21.15.40:0/2371918232 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560db65563d0 con 0x560db6683800 2024-06-06T03:43:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.560+0000 7f93e4ff9700 10 client.15615 handle_command_reply: tid=0 2024-06-06T03:43:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.560+0000 7f940db55b80 1 client.15615 shutdown 2024-06-06T03:43:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.560+0000 7f940db55b80 1 -- 172.21.15.40:0/2371918232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 msgr2=0x560db6610040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.560+0000 7f940db55b80 1 --2- 172.21.15.40:0/2371918232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db6610040 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f93f8086010 tx=0x7f93f80a4470 comp rx=0 tx=0).stop 2024-06-06T03:43:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.560+0000 7f940db55b80 1 -- 172.21.15.40:0/2371918232 shutdown_connections 2024-06-06T03:43:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.560+0000 7f940db55b80 1 --2- 172.21.15.40:0/2371918232 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560db6683800 0x560db6685c20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.561+0000 7f940db55b80 1 --2- 172.21.15.40:0/2371918232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560db6603e20 0x560db660d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.561+0000 7f940db55b80 1 --2- 172.21.15.40:0/2371918232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560db65fdc50 0x560db6610040 secure :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f93f8086010 tx=0x7f93f80a4470 comp rx=0 tx=0).stop 2024-06-06T03:43:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.561+0000 7f940db55b80 1 --2- 172.21.15.40:0/2371918232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560db6603500 0x560db660cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.561+0000 7f940db55b80 1 -- 172.21.15.40:0/2371918232 >> 172.21.15.40:0/2371918232 conn(0x560db652a180 msgr2=0x560db6608ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.561+0000 7f93e4ff9700 0 client.15615 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:43:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.561+0000 7f940db55b80 1 -- 172.21.15.40:0/2371918232 shutdown_connections 2024-06-06T03:43:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.561+0000 7f940db55b80 1 -- 172.21.15.40:0/2371918232 wait complete. 2024-06-06T03:43:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.561+0000 7f940db55b80 20 client.15615 trim_cache size 0 max 16384 2024-06-06T03:43:40.564 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:43:40.564 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 36, 2024-06-06T03:43:40.564 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:43:40.564 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 16, 2024-06-06T03:43:40.564 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": { 2024-06-06T03:43:40.564 INFO:teuthology.orchestra.run.smithi040.stdout: "ad9f5028": { 2024-06-06T03:43:40.564 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 16, 2024-06-06T03:43:40.564 INFO:teuthology.orchestra.run.smithi040.stdout: "age_ref": 0.0, 2024-06-06T03:43:40.564 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:43:40.564 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RELEASED", 2024-06-06T03:43:40.565 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:43:40.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:43:40.565 INFO:teuthology.orchestra.run.smithi040.stdout: "timeout": 90.0, 2024-06-06T03:43:40.565 INFO:teuthology.orchestra.run.smithi040.stdout: "expiration": 145.2, 2024-06-06T03:43:40.565 INFO:teuthology.orchestra.run.smithi040.stdout: "members": { 2024-06-06T03:43:40.565 INFO:teuthology.orchestra.run.smithi040.stdout: "file:/": { 2024-06-06T03:43:40.565 INFO:teuthology.orchestra.run.smithi040.stdout: "excluded": false, 2024-06-06T03:43:40.565 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:43:40.565 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RELEASED", 2024-06-06T03:43:40.565 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:43:40.565 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:40.565 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:40.566 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:40.566 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:40.566 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:43:40.566 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:43:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.563+0000 7f94032d2700 1 -- 172.21.15.40:0/505177727 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f93e80a2e10 msgr2=0x7f93e80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.563+0000 7f94032d2700 1 --2- 172.21.15.40:0/505177727 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f93e80a2e10 0x7f93e80a52d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0761d0 tx=0x7f93ec005fb0 comp rx=0 tx=0).stop 2024-06-06T03:43:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.563+0000 7f94032d2700 1 -- 172.21.15.40:0/505177727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93fc102740 msgr2=0x7f93fc074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:43:40.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.563+0000 7f94032d2700 1 --2- 172.21.15.40:0/505177727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93fc102740 0x7f93fc074790 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f93f800d980 tx=0x7f93f800dc90 comp rx=0 tx=0).stop 2024-06-06T03:43:40.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.563+0000 7f94032d2700 1 -- 172.21.15.40:0/505177727 shutdown_connections 2024-06-06T03:43:40.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.563+0000 7f94032d2700 1 --2- 172.21.15.40:0/505177727 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f93e80a2e10 0x7f93e80a52d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:40.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.563+0000 7f94032d2700 1 --2- 172.21.15.40:0/505177727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93fc107840 0x7f93fc075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:40.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.563+0000 7f94032d2700 1 --2- 172.21.15.40:0/505177727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93fc102740 0x7f93fc074790 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:40.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.563+0000 7f94032d2700 1 --2- 172.21.15.40:0/505177727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93fc1030f0 0x7f93fc074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:43:40.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.563+0000 7f94032d2700 1 -- 172.21.15.40:0/505177727 >> 172.21.15.40:0/505177727 conn(0x7f93fc0795f0 msgr2=0x7f93fc108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:43:40.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.563+0000 7f94032d2700 1 -- 172.21.15.40:0/505177727 shutdown_connections 2024-06-06T03:43:40.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:43:40.563+0000 7f94032d2700 1 -- 172.21.15.40:0/505177727 wait complete. 2024-06-06T03:43:41.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:41 smithi040 ceph-mon[32220]: pgmap v154: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 679 op/s 2024-06-06T03:43:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:41 smithi131 ceph-mon[36687]: pgmap v154: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 679 op/s 2024-06-06T03:43:41.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:41 smithi104 ceph-mon[36569]: pgmap v154: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 679 op/s 2024-06-06T03:43:41.403 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ad9f5028', total seconds elapsed: 34.7 2024-06-06T03:43:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:43 smithi131 ceph-mon[36687]: pgmap v155: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 273 op/s 2024-06-06T03:43:43.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:43 smithi040 ceph-mon[32220]: pgmap v155: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 273 op/s 2024-06-06T03:43:43.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:43 smithi104 ceph-mon[36569]: pgmap v155: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 273 op/s 2024-06-06T03:43:45.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:45 smithi131 ceph-mon[36687]: pgmap v156: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 273 op/s 2024-06-06T03:43:45.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:45 smithi040 ceph-mon[32220]: pgmap v156: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 273 op/s 2024-06-06T03:43:45.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:45 smithi104 ceph-mon[36569]: pgmap v156: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 273 op/s 2024-06-06T03:43:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:47 smithi131 ceph-mon[36687]: pgmap v157: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 290 op/s 2024-06-06T03:43:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:47 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:43:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:47 smithi040 ceph-mon[32220]: pgmap v157: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 290 op/s 2024-06-06T03:43:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:47 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:43:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:47 smithi104 ceph-mon[36569]: pgmap v157: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 290 op/s 2024-06-06T03:43:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:47 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:43:48.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:48 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:43:48.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:48 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:43:48.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:48 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:43:48.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:48 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:43:48.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:48 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:43:48.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:48 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:43:48.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:48 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:43:48.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:48 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:43:48.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:48 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:43:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:49 smithi131 ceph-mon[36687]: pgmap v158: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 673 KiB/s wr, 23 op/s 2024-06-06T03:43:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:49 smithi040 ceph-mon[32220]: pgmap v158: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 673 KiB/s wr, 23 op/s 2024-06-06T03:43:49.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:49 smithi104 ceph-mon[36569]: pgmap v158: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 673 KiB/s wr, 23 op/s 2024-06-06T03:43:51.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:51 smithi131 ceph-mon[36687]: pgmap v159: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 37 op/s 2024-06-06T03:43:51.295 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:51 smithi040 ceph-mon[32220]: pgmap v159: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 37 op/s 2024-06-06T03:43:51.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:51 smithi104 ceph-mon[36569]: pgmap v159: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 37 op/s 2024-06-06T03:43:53.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:53 smithi131 ceph-mon[36687]: pgmap v160: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 29 op/s 2024-06-06T03:43:53.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:53 smithi040 ceph-mon[32220]: pgmap v160: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 29 op/s 2024-06-06T03:43:53.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:53 smithi104 ceph-mon[36569]: pgmap v160: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 29 op/s 2024-06-06T03:43:55.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:55 smithi131 ceph-mon[36687]: pgmap v161: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 29 op/s 2024-06-06T03:43:55.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:55 smithi040 ceph-mon[32220]: pgmap v161: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 29 op/s 2024-06-06T03:43:55.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:55 smithi104 ceph-mon[36569]: pgmap v161: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 29 op/s 2024-06-06T03:43:57.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:57 smithi131 ceph-mon[36687]: pgmap v162: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 72 op/s 2024-06-06T03:43:57.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:57 smithi040 ceph-mon[32220]: pgmap v162: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 72 op/s 2024-06-06T03:43:57.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:57 smithi104 ceph-mon[36569]: pgmap v162: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 72 op/s 2024-06-06T03:43:59.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:43:59 smithi040 ceph-mon[32220]: pgmap v163: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 56 op/s 2024-06-06T03:43:59.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:43:59 smithi104 ceph-mon[36569]: pgmap v163: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 56 op/s 2024-06-06T03:43:59.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:43:59 smithi131 ceph-mon[36687]: pgmap v163: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 56 op/s 2024-06-06T03:44:01.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:01 smithi040 ceph-mon[32220]: pgmap v164: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 420 op/s 2024-06-06T03:44:01.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:01 smithi104 ceph-mon[36569]: pgmap v164: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 420 op/s 2024-06-06T03:44:01.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:01 smithi131 ceph-mon[36687]: pgmap v164: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 420 op/s 2024-06-06T03:44:03.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:03 smithi040 ceph-mon[32220]: pgmap v165: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.7 MiB/s wr, 406 op/s 2024-06-06T03:44:03.334 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:03 smithi131 ceph-mon[36687]: pgmap v165: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.7 MiB/s wr, 406 op/s 2024-06-06T03:44:03.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:03 smithi104 ceph-mon[36569]: pgmap v165: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.7 MiB/s wr, 406 op/s 2024-06-06T03:44:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:05 smithi040 ceph-mon[32220]: pgmap v166: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.7 MiB/s wr, 406 op/s 2024-06-06T03:44:05.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:05 smithi104 ceph-mon[36569]: pgmap v166: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.7 MiB/s wr, 406 op/s 2024-06-06T03:44:05.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:05 smithi131 ceph-mon[36687]: pgmap v166: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.7 MiB/s wr, 406 op/s 2024-06-06T03:44:07.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:07 smithi040 ceph-mon[32220]: pgmap v167: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 777 op/s 2024-06-06T03:44:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:07 smithi104 ceph-mon[36569]: pgmap v167: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 777 op/s 2024-06-06T03:44:07.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:07 smithi131 ceph-mon[36687]: pgmap v167: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 777 op/s 2024-06-06T03:44:09.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:09 smithi040 ceph-mon[32220]: pgmap v168: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 16 MiB/s wr, 734 op/s 2024-06-06T03:44:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:09 smithi104 ceph-mon[36569]: pgmap v168: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 16 MiB/s wr, 734 op/s 2024-06-06T03:44:09.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:09 smithi131 ceph-mon[36687]: pgmap v168: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 16 MiB/s wr, 734 op/s 2024-06-06T03:44:11.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:11 smithi040 ceph-mon[32220]: pgmap v169: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 17 MiB/s wr, 760 op/s 2024-06-06T03:44:11.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:11 smithi104 ceph-mon[36569]: pgmap v169: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 17 MiB/s wr, 760 op/s 2024-06-06T03:44:11.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:11 smithi131 ceph-mon[36687]: pgmap v169: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 17 MiB/s wr, 760 op/s 2024-06-06T03:44:12.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:12 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:44:12.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:12 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:44:12.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:12 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:44:13.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:13 smithi040 ceph-mon[32220]: pgmap v170: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.9 MiB/s wr, 397 op/s 2024-06-06T03:44:13.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:13 smithi104 ceph-mon[36569]: pgmap v170: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.9 MiB/s wr, 397 op/s 2024-06-06T03:44:13.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:13 smithi131 ceph-mon[36687]: pgmap v170: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.9 MiB/s wr, 397 op/s 2024-06-06T03:44:15.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:15 smithi040 ceph-mon[32220]: pgmap v171: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.9 MiB/s wr, 397 op/s 2024-06-06T03:44:15.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:15 smithi104 ceph-mon[36569]: pgmap v171: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.9 MiB/s wr, 397 op/s 2024-06-06T03:44:15.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:15 smithi131 ceph-mon[36687]: pgmap v171: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.9 MiB/s wr, 397 op/s 2024-06-06T03:44:17.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:17 smithi040 ceph-mon[32220]: pgmap v172: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 10 MiB/s wr, 417 op/s 2024-06-06T03:44:17.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:17 smithi104 ceph-mon[36569]: pgmap v172: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 10 MiB/s wr, 417 op/s 2024-06-06T03:44:17.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:17 smithi131 ceph-mon[36687]: pgmap v172: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 10 MiB/s wr, 417 op/s 2024-06-06T03:44:19.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:19 smithi040 ceph-mon[32220]: pgmap v173: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 46 op/s 2024-06-06T03:44:19.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:19 smithi104 ceph-mon[36569]: pgmap v173: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 46 op/s 2024-06-06T03:44:19.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:19 smithi131 ceph-mon[36687]: pgmap v173: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 46 op/s 2024-06-06T03:44:21.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:21 smithi040 ceph-mon[32220]: pgmap v174: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.2 MiB/s wr, 279 op/s 2024-06-06T03:44:21.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:21 smithi104 ceph-mon[36569]: pgmap v174: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.2 MiB/s wr, 279 op/s 2024-06-06T03:44:21.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:21 smithi131 ceph-mon[36687]: pgmap v174: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.2 MiB/s wr, 279 op/s 2024-06-06T03:44:23.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:23 smithi040 ceph-mon[32220]: pgmap v175: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 254 op/s 2024-06-06T03:44:23.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:23 smithi104 ceph-mon[36569]: pgmap v175: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 254 op/s 2024-06-06T03:44:23.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:23 smithi131 ceph-mon[36687]: pgmap v175: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 254 op/s 2024-06-06T03:44:25.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:25 smithi040 ceph-mon[32220]: pgmap v176: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 254 op/s 2024-06-06T03:44:25.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:25 smithi104 ceph-mon[36569]: pgmap v176: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 254 op/s 2024-06-06T03:44:25.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:25 smithi131 ceph-mon[36687]: pgmap v176: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 254 op/s 2024-06-06T03:44:27.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:27 smithi040 ceph-mon[32220]: pgmap v177: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.5 MiB/s wr, 269 op/s 2024-06-06T03:44:27.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:27 smithi104 ceph-mon[36569]: pgmap v177: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.5 MiB/s wr, 269 op/s 2024-06-06T03:44:27.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:27 smithi131 ceph-mon[36687]: pgmap v177: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.5 MiB/s wr, 269 op/s 2024-06-06T03:44:29.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:29 smithi040 ceph-mon[32220]: pgmap v178: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 248 op/s 2024-06-06T03:44:29.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:29 smithi104 ceph-mon[36569]: pgmap v178: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 248 op/s 2024-06-06T03:44:29.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:29 smithi131 ceph-mon[36687]: pgmap v178: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 248 op/s 2024-06-06T03:44:31.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:31 smithi104 ceph-mon[36569]: pgmap v179: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.6 MiB/s wr, 266 op/s 2024-06-06T03:44:31.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:31 smithi131 ceph-mon[36687]: pgmap v179: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.6 MiB/s wr, 266 op/s 2024-06-06T03:44:31.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:31 smithi040 ceph-mon[32220]: pgmap v179: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.6 MiB/s wr, 266 op/s 2024-06-06T03:44:33.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:33 smithi104 ceph-mon[36569]: pgmap v180: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 33 op/s 2024-06-06T03:44:33.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:33 smithi131 ceph-mon[36687]: pgmap v180: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 33 op/s 2024-06-06T03:44:33.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:33 smithi040 ceph-mon[32220]: pgmap v180: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 33 op/s 2024-06-06T03:44:35.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:35 smithi131 ceph-mon[36687]: pgmap v181: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 33 op/s 2024-06-06T03:44:35.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:35 smithi040 ceph-mon[32220]: pgmap v181: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 33 op/s 2024-06-06T03:44:35.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:35 smithi104 ceph-mon[36569]: pgmap v181: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 33 op/s 2024-06-06T03:44:37.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:37 smithi131 ceph-mon[36687]: pgmap v182: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 134 op/s 2024-06-06T03:44:37.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:37 smithi040 ceph-mon[32220]: pgmap v182: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 134 op/s 2024-06-06T03:44:37.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:37 smithi104 ceph-mon[36569]: pgmap v182: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 134 op/s 2024-06-06T03:44:39.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:39 smithi131 ceph-mon[36687]: pgmap v183: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 2.1 MiB/s wr, 119 op/s 2024-06-06T03:44:39.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:39 smithi040 ceph-mon[32220]: pgmap v183: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 2.1 MiB/s wr, 119 op/s 2024-06-06T03:44:39.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:39 smithi104 ceph-mon[36569]: pgmap v183: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 2.1 MiB/s wr, 119 op/s 2024-06-06T03:44:41.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:41 smithi131 ceph-mon[36687]: pgmap v184: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 392 op/s 2024-06-06T03:44:41.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:41 smithi040 ceph-mon[32220]: pgmap v184: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 392 op/s 2024-06-06T03:44:41.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:41 smithi104 ceph-mon[36569]: pgmap v184: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 392 op/s 2024-06-06T03:44:43.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:43 smithi131 ceph-mon[36687]: pgmap v185: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.0 MiB/s wr, 496 op/s 2024-06-06T03:44:43.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:43 smithi040 ceph-mon[32220]: pgmap v185: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.0 MiB/s wr, 496 op/s 2024-06-06T03:44:43.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:43 smithi104 ceph-mon[36569]: pgmap v185: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.0 MiB/s wr, 496 op/s 2024-06-06T03:44:45.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:45 smithi131 ceph-mon[36687]: pgmap v186: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.0 MiB/s wr, 496 op/s 2024-06-06T03:44:45.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:45 smithi040 ceph-mon[32220]: pgmap v186: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.0 MiB/s wr, 496 op/s 2024-06-06T03:44:45.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:45 smithi104 ceph-mon[36569]: pgmap v186: 97 pgs: 97 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.0 MiB/s wr, 496 op/s 2024-06-06T03:44:47.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:47 smithi131 ceph-mon[36687]: pgmap v187: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 13 MiB/s wr, 876 op/s 2024-06-06T03:44:47.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:47 smithi040 ceph-mon[32220]: pgmap v187: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 13 MiB/s wr, 876 op/s 2024-06-06T03:44:47.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:47 smithi104 ceph-mon[36569]: pgmap v187: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 13 MiB/s wr, 876 op/s 2024-06-06T03:44:48.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:48 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:44:48.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:48 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:44:48.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:48 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:44:49.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:49 smithi131 ceph-mon[36687]: pgmap v188: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 774 op/s 2024-06-06T03:44:49.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:49 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:44:49.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:49 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:44:49.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:49 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:44:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:49 smithi040 ceph-mon[32220]: pgmap v188: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 774 op/s 2024-06-06T03:44:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:49 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:44:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:49 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:44:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:49 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:44:49.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:49 smithi104 ceph-mon[36569]: pgmap v188: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 774 op/s 2024-06-06T03:44:49.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:49 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:44:49.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:49 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:44:49.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:49 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:44:51.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:51 smithi131 ceph-mon[36687]: pgmap v189: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 12 MiB/s wr, 796 op/s 2024-06-06T03:44:51.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:51 smithi040 ceph-mon[32220]: pgmap v189: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 12 MiB/s wr, 796 op/s 2024-06-06T03:44:51.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:51 smithi104 ceph-mon[36569]: pgmap v189: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 12 MiB/s wr, 796 op/s 2024-06-06T03:44:53.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:53 smithi131 ceph-mon[36687]: pgmap v190: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 525 op/s 2024-06-06T03:44:53.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:53 smithi040 ceph-mon[32220]: pgmap v190: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 525 op/s 2024-06-06T03:44:53.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:53 smithi104 ceph-mon[36569]: pgmap v190: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 525 op/s 2024-06-06T03:44:55.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:55 smithi131 ceph-mon[36687]: pgmap v191: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 403 op/s 2024-06-06T03:44:55.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:55 smithi040 ceph-mon[32220]: pgmap v191: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 403 op/s 2024-06-06T03:44:55.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:55 smithi104 ceph-mon[36569]: pgmap v191: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 403 op/s 2024-06-06T03:44:57.382 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:57 smithi040 ceph-mon[32220]: pgmap v192: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 418 op/s 2024-06-06T03:44:57.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:57 smithi131 ceph-mon[36687]: pgmap v192: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 418 op/s 2024-06-06T03:44:57.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:57 smithi104 ceph-mon[36569]: pgmap v192: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 418 op/s 2024-06-06T03:44:59.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:44:59 smithi131 ceph-mon[36687]: pgmap v193: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 39 op/s 2024-06-06T03:44:59.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:44:59 smithi040 ceph-mon[32220]: pgmap v193: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 39 op/s 2024-06-06T03:44:59.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:44:59 smithi104 ceph-mon[36569]: pgmap v193: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 39 op/s 2024-06-06T03:45:01.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:01 smithi131 ceph-mon[36687]: pgmap v194: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 54 op/s 2024-06-06T03:45:01.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:01 smithi040 ceph-mon[32220]: pgmap v194: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 54 op/s 2024-06-06T03:45:01.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:01 smithi104 ceph-mon[36569]: pgmap v194: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 54 op/s 2024-06-06T03:45:03.392 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:03 smithi104 ceph-mon[36569]: pgmap v195: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.0 MiB/s wr, 35 op/s 2024-06-06T03:45:03.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:03 smithi131 ceph-mon[36687]: pgmap v195: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.0 MiB/s wr, 35 op/s 2024-06-06T03:45:03.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:03 smithi040 ceph-mon[32220]: pgmap v195: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.0 MiB/s wr, 35 op/s 2024-06-06T03:45:05.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:05 smithi131 ceph-mon[36687]: pgmap v196: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.0 MiB/s wr, 32 op/s 2024-06-06T03:45:05.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:05 smithi040 ceph-mon[32220]: pgmap v196: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.0 MiB/s wr, 32 op/s 2024-06-06T03:45:05.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:05 smithi104 ceph-mon[36569]: pgmap v196: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.0 MiB/s wr, 32 op/s 2024-06-06T03:45:07.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:07 smithi131 ceph-mon[36687]: pgmap v197: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 70 op/s 2024-06-06T03:45:07.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:07 smithi131 ceph-mon[36687]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T03:45:07.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:07 smithi040 ceph-mon[32220]: pgmap v197: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 70 op/s 2024-06-06T03:45:07.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:07 smithi040 ceph-mon[32220]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T03:45:07.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:07 smithi104 ceph-mon[36569]: pgmap v197: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 70 op/s 2024-06-06T03:45:07.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:07 smithi104 ceph-mon[36569]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T03:45:09.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:09 smithi131 ceph-mon[36687]: pgmap v199: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 155 op/s 2024-06-06T03:45:09.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:09 smithi040 ceph-mon[32220]: pgmap v199: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 155 op/s 2024-06-06T03:45:09.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:09 smithi104 ceph-mon[36569]: pgmap v199: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 155 op/s 2024-06-06T03:45:11.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:11 smithi131 ceph-mon[36687]: pgmap v200: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 14 MiB/s wr, 942 op/s 2024-06-06T03:45:11.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:11 smithi040 ceph-mon[32220]: pgmap v200: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 14 MiB/s wr, 942 op/s 2024-06-06T03:45:11.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:11 smithi104 ceph-mon[36569]: pgmap v200: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 14 MiB/s wr, 942 op/s 2024-06-06T03:45:13.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:13 smithi131 ceph-mon[36687]: pgmap v201: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 1.20k op/s 2024-06-06T03:45:13.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:13 smithi040 ceph-mon[32220]: pgmap v201: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 1.20k op/s 2024-06-06T03:45:13.590 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:13 smithi104 ceph-mon[36569]: pgmap v201: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 1.20k op/s 2024-06-06T03:45:15.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:14 smithi131 ceph-mon[36687]: pgmap v202: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 1.20k op/s 2024-06-06T03:45:15.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:14 smithi131 ceph-mon[36687]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T03:45:15.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:14 smithi040 ceph-mon[32220]: pgmap v202: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 1.20k op/s 2024-06-06T03:45:15.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:14 smithi040 ceph-mon[32220]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T03:45:15.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:14 smithi104 ceph-mon[36569]: pgmap v202: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 1.20k op/s 2024-06-06T03:45:15.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:14 smithi104 ceph-mon[36569]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T03:45:15.189 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 12.6 2024-06-06T03:45:15.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:45:15.452 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:45:17.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:16 smithi131 ceph-mon[36687]: pgmap v204: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 1.29k op/s 2024-06-06T03:45:17.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:16 smithi040 ceph-mon[32220]: pgmap v204: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 1.29k op/s 2024-06-06T03:45:17.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:16 smithi104 ceph-mon[36569]: pgmap v204: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 1.29k op/s 2024-06-06T03:45:18.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.121+0000 7fbe18aa5700 1 -- 172.21.15.40:0/3853309816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe14107420 msgr2=0x7fbe14107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:18.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.121+0000 7fbe18aa5700 1 --2- 172.21.15.40:0/3853309816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe14107420 0x7fbe14107800 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009990 tx=0x7fbe04009ca0 comp rx=0 tx=0).stop 2024-06-06T03:45:18.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.121+0000 7fbe18aa5700 1 -- 172.21.15.40:0/3853309816 shutdown_connections 2024-06-06T03:45:18.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.121+0000 7fbe18aa5700 1 --2- 172.21.15.40:0/3853309816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe14101d30 0x7fbe1410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.121+0000 7fbe18aa5700 1 --2- 172.21.15.40:0/3853309816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe14107420 0x7fbe14107800 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.121+0000 7fbe18aa5700 1 --2- 172.21.15.40:0/3853309816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe14101370 0x7fbe141017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.121+0000 7fbe18aa5700 1 -- 172.21.15.40:0/3853309816 >> 172.21.15.40:0/3853309816 conn(0x7fbe14078340 msgr2=0x7fbe14078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.121+0000 7fbe18aa5700 1 -- 172.21.15.40:0/3853309816 shutdown_connections 2024-06-06T03:45:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.121+0000 7fbe18aa5700 1 -- 172.21.15.40:0/3853309816 wait complete. 2024-06-06T03:45:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.122+0000 7fbe18aa5700 1 Processor -- start 2024-06-06T03:45:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.122+0000 7fbe18aa5700 1 -- start start 2024-06-06T03:45:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.122+0000 7fbe18aa5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe14101370 0x7fbe14103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.122+0000 7fbe18aa5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe14101d30 0x7fbe14103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.122+0000 7fbe18aa5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe14107420 0x7fbe14103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.122+0000 7fbe18aa5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbe14111630 con 0x7fbe14101370 2024-06-06T03:45:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.122+0000 7fbe18aa5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbe141114b0 con 0x7fbe14101d30 2024-06-06T03:45:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.122+0000 7fbe18aa5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbe141117b0 con 0x7fbe14107420 2024-06-06T03:45:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.122+0000 7fbe1259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe14101370 0x7fbe14103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.122+0000 7fbe1259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe14101370 0x7fbe14103020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38700/0 (socket says 172.21.15.40:38700) 2024-06-06T03:45:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.122+0000 7fbe1259c700 1 -- 172.21.15.40:0/1651733266 learned_addr learned my addr 172.21.15.40:0/1651733266 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:45:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.122+0000 7fbe11d9b700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe14101d30 0x7fbe14103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.122+0000 7fbe12d9d700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe14107420 0x7fbe14103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.123+0000 7fbe1259c700 1 -- 172.21.15.40:0/1651733266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe14107420 msgr2=0x7fbe14103aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.123+0000 7fbe1259c700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe14107420 0x7fbe14103aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.123+0000 7fbe1259c700 1 -- 172.21.15.40:0/1651733266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe14101d30 msgr2=0x7fbe14103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.123+0000 7fbe1259c700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe14101d30 0x7fbe14103560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.123+0000 7fbe1259c700 1 -- 172.21.15.40:0/1651733266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe141a70e0 con 0x7fbe14101370 2024-06-06T03:45:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.123+0000 7fbe11d9b700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe14101d30 0x7fbe14103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:45:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.123+0000 7fbe1259c700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe14101370 0x7fbe14103020 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbe04000c00 tx=0x7fbe040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.123+0000 7fbe037fe700 1 -- 172.21.15.40:0/1651733266 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04027070 con 0x7fbe14101370 2024-06-06T03:45:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.123+0000 7fbe18aa5700 1 -- 172.21.15.40:0/1651733266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe141a7370 con 0x7fbe14101370 2024-06-06T03:45:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.123+0000 7fbe037fe700 1 -- 172.21.15.40:0/1651733266 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe04003780 con 0x7fbe14101370 2024-06-06T03:45:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.123+0000 7fbe037fe700 1 -- 172.21.15.40:0/1651733266 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04022dd0 con 0x7fbe14101370 2024-06-06T03:45:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.123+0000 7fbe18aa5700 1 -- 172.21.15.40:0/1651733266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe141a78a0 con 0x7fbe14101370 2024-06-06T03:45:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.124+0000 7fbe18aa5700 1 -- 172.21.15.40:0/1651733266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe14069180 con 0x7fbe14101370 2024-06-06T03:45:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.125+0000 7fbe037fe700 1 -- 172.21.15.40:0/1651733266 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbe0402a070 con 0x7fbe14101370 2024-06-06T03:45:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.126+0000 7fbe037fe700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fbdfc0a2d90 0x7fbdfc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.126+0000 7fbe037fe700 1 -- 172.21.15.40:0/1651733266 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8426+0+0 (secure 0 0 0) 0x7fbe040d0470 con 0x7fbe14101370 2024-06-06T03:45:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.126+0000 7fbe11d9b700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fbdfc0a2d90 0x7fbdfc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.126+0000 7fbe11d9b700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fbdfc0a2d90 0x7fbdfc0a5250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbe08005950 tx=0x7fbe080058e0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.130+0000 7fbe037fe700 1 -- 172.21.15.40:0/1651733266 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbe04013070 con 0x7fbe14101370 2024-06-06T03:45:18.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.385+0000 7fbe18aa5700 1 -- 172.21.15.40:0/1651733266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe141a7bc0 con 0x7fbe14101370 2024-06-06T03:45:18.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.385+0000 7fbe037fe700 1 -- 172.21.15.40:0/1651733266 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11940 (secure 0 0 0) 0x7fbe0409d210 con 0x7fbe14101370 2024-06-06T03:45:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:45:18.386 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:44:10.814777+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24443,24425,24398,24403]},"id":1}]} 2024-06-06T03:45:18.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.388+0000 7fbe18aa5700 1 -- 172.21.15.40:0/1651733266 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fbdfc0a2d90 msgr2=0x7fbdfc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.388+0000 7fbe18aa5700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fbdfc0a2d90 0x7fbdfc0a5250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbe08005950 tx=0x7fbe080058e0 comp rx=0 tx=0).stop 2024-06-06T03:45:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.388+0000 7fbe18aa5700 1 -- 172.21.15.40:0/1651733266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe14101370 msgr2=0x7fbe14103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.388+0000 7fbe18aa5700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe14101370 0x7fbe14103020 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbe04000c00 tx=0x7fbe040047b0 comp rx=0 tx=0).stop 2024-06-06T03:45:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.388+0000 7fbe18aa5700 1 -- 172.21.15.40:0/1651733266 shutdown_connections 2024-06-06T03:45:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.388+0000 7fbe18aa5700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fbdfc0a2d90 0x7fbdfc0a5250 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.388+0000 7fbe18aa5700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe14107420 0x7fbe14103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.388+0000 7fbe18aa5700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe14101370 0x7fbe14103020 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.388+0000 7fbe18aa5700 1 --2- 172.21.15.40:0/1651733266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe14101d30 0x7fbe14103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.388+0000 7fbe18aa5700 1 -- 172.21.15.40:0/1651733266 >> 172.21.15.40:0/1651733266 conn(0x7fbe14078340 msgr2=0x7fbe1406ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.388+0000 7fbe18aa5700 1 -- 172.21.15.40:0/1651733266 shutdown_connections 2024-06-06T03:45:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:18.388+0000 7fbe18aa5700 1 -- 172.21.15.40:0/1651733266 wait complete. 2024-06-06T03:45:18.390 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 37 2024-06-06T03:45:19.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:18 smithi131 ceph-mon[36687]: pgmap v205: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 1.26k op/s 2024-06-06T03:45:19.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:18 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1651733266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:45:19.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:18 smithi040 ceph-mon[32220]: pgmap v205: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 1.26k op/s 2024-06-06T03:45:19.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:18 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1651733266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:45:19.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:18 smithi104 ceph-mon[36569]: pgmap v205: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 1.26k op/s 2024-06-06T03:45:19.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:18 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1651733266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:45:19.224 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 132.6 --await-for 45.0' 2024-06-06T03:45:19.224 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 132.6 --await-for 45.0 2024-06-06T03:45:19.476 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:45:20.924 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:20 smithi040 ceph-mon[32220]: pgmap v206: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 10 MiB/s wr, 498 op/s 2024-06-06T03:45:21.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:20 smithi131 ceph-mon[36687]: pgmap v206: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 10 MiB/s wr, 498 op/s 2024-06-06T03:45:21.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:20 smithi104 ceph-mon[36569]: pgmap v206: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 10 MiB/s wr, 498 op/s 2024-06-06T03:45:21.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.316+0000 7f080a06e700 1 -- 172.21.15.40:0/886927627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0804103150 msgr2=0x7f08041070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:21.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.316+0000 7f080a06e700 1 --2- 172.21.15.40:0/886927627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0804103150 0x7f08041070b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0099c0 tx=0x7f07ec009cd0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.316+0000 7f080a06e700 1 -- 172.21.15.40:0/886927627 shutdown_connections 2024-06-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.316+0000 7f080a06e700 1 --2- 172.21.15.40:0/886927627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08041027a0 0x7f0804102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.316+0000 7f080a06e700 1 --2- 172.21.15.40:0/886927627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0804107790 0x7f080410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.316+0000 7f080a06e700 1 --2- 172.21.15.40:0/886927627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0804103150 0x7f08041070b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.316+0000 7f080a06e700 1 -- 172.21.15.40:0/886927627 >> 172.21.15.40:0/886927627 conn(0x7f0804079560 msgr2=0x7f0804079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.316+0000 7f080a06e700 1 -- 172.21.15.40:0/886927627 shutdown_connections 2024-06-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.316+0000 7f080a06e700 1 -- 172.21.15.40:0/886927627 wait complete. 2024-06-06T03:45:21.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.316+0000 7f080a06e700 1 Processor -- start 2024-06-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f080a06e700 1 -- start start 2024-06-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f080a06e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08041027a0 0x7f080419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f080a06e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0804103150 0x7f080419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f080a06e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0804107790 0x7f0804198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f080a06e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f08041127b0 con 0x7f0804103150 2024-06-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f080a06e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0804112630 con 0x7f08041027a0 2024-06-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f080a06e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0804112930 con 0x7f0804107790 2024-06-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f07faffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0804103150 0x7f080419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f07faffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0804103150 0x7f080419ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48580/0 (socket says 172.21.15.40:48580) 2024-06-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f07faffd700 1 -- 172.21.15.40:0/3832263471 learned_addr learned my addr 172.21.15.40:0/3832263471 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f0803fff700 1 --2- 172.21.15.40:0/3832263471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0804107790 0x7f0804198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f08037fe700 1 --2- 172.21.15.40:0/3832263471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08041027a0 0x7f080419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f07faffd700 1 -- 172.21.15.40:0/3832263471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0804107790 msgr2=0x7f0804198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f07faffd700 1 --2- 172.21.15.40:0/3832263471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0804107790 0x7f0804198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f07faffd700 1 -- 172.21.15.40:0/3832263471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08041027a0 msgr2=0x7f080419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f07faffd700 1 --2- 172.21.15.40:0/3832263471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08041027a0 0x7f080419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.317+0000 7f07faffd700 1 -- 172.21.15.40:0/3832263471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0804199170 con 0x7f0804103150 2024-06-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.318+0000 7f07faffd700 1 --2- 172.21.15.40:0/3832263471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0804103150 0x7f080419ed90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0059d0 tx=0x7f07ec0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.318+0000 7f08017fa700 1 -- 172.21.15.40:0/3832263471 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec027070 con 0x7f0804103150 2024-06-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.318+0000 7f08017fa700 1 -- 172.21.15.40:0/3832263471 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ec004050 con 0x7f0804103150 2024-06-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.318+0000 7f080a06e700 1 -- 172.21.15.40:0/3832263471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0804199400 con 0x7f0804103150 2024-06-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.318+0000 7f08017fa700 1 -- 172.21.15.40:0/3832263471 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec022d20 con 0x7f0804103150 2024-06-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.318+0000 7f080a06e700 1 -- 172.21.15.40:0/3832263471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08041a5720 con 0x7f0804103150 2024-06-06T03:45:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.320+0000 7f08017fa700 1 -- 172.21.15.40:0/3832263471 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f07ec02a070 con 0x7f0804103150 2024-06-06T03:45:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.321+0000 7f08017fa700 1 --2- 172.21.15.40:0/3832263471 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f07f00a2eb0 0x7f07f00a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.322+0000 7f08037fe700 1 --2- 172.21.15.40:0/3832263471 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f07f00a2eb0 0x7f07f00a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.322+0000 7f08017fa700 1 -- 172.21.15.40:0/3832263471 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8426+0+0 (secure 0 0 0) 0x7f07ec0d03f0 con 0x7f0804103150 2024-06-06T03:45:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.322+0000 7f08037fe700 1 --2- 172.21.15.40:0/3832263471 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f07f00a2eb0 0x7f07f00a5370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0804199b50 tx=0x7f07f4009040 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.322+0000 7f08148f1b80 1 Processor -- start 2024-06-06T03:45:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.322+0000 7f08148f1b80 1 -- start start 2024-06-06T03:45:21.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f08148f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9590 0x563e167a9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f08148f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9eb0 0x563e167a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f08148f1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 0x563e167a4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f08148f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563e166960f0 con 0x563e167a9590 2024-06-06T03:45:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f08148f1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563e166f49e0 con 0x563e167a3ce0 2024-06-06T03:45:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f08148f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563e1665c5f0 con 0x563e167a9eb0 2024-06-06T03:45:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f07faffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9eb0 0x563e167a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f08037fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9590 0x563e167a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f07faffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9eb0 0x563e167a37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.40:51424/0 (socket says 172.21.15.40:51424) 2024-06-06T03:45:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f07faffd700 1 -- 172.21.15.40:0/2106467107 learned_addr learned my addr 172.21.15.40:0/2106467107 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:45:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f0803fff700 1 --2- 172.21.15.40:0/2106467107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 0x563e167a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f07faffd700 1 -- 172.21.15.40:0/2106467107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 msgr2=0x563e167a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f07faffd700 1 --2- 172.21.15.40:0/2106467107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 0x563e167a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f07faffd700 1 -- 172.21.15.40:0/2106467107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9590 msgr2=0x563e167a9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f07faffd700 1 --2- 172.21.15.40:0/2106467107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9590 0x563e167a9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f07faffd700 1 -- 172.21.15.40:0/2106467107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e16643a60 con 0x563e167a9eb0 2024-06-06T03:45:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f0803fff700 1 --2- 172.21.15.40:0/2106467107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 0x563e167a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:45:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f07faffd700 1 --2- 172.21.15.40:0/2106467107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9eb0 0x563e167a37a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f07ec099c20 tx=0x7f07ec01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8348299de05fcc7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.323+0000 7f07f9ffb700 1 -- 172.21.15.40:0/2106467107 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec09d7c0 con 0x563e167a9eb0 2024-06-06T03:45:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.324+0000 7f07f9ffb700 1 -- 172.21.15.40:0/2106467107 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ec0a1b70 con 0x563e167a9eb0 2024-06-06T03:45:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.324+0000 7f07f9ffb700 1 -- 172.21.15.40:0/2106467107 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec0a53e0 con 0x563e167a9eb0 2024-06-06T03:45:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.324+0000 7f08148f1b80 1 -- 172.21.15.40:0/2106467107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9eb0 msgr2=0x563e167a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.324+0000 7f08148f1b80 1 --2- 172.21.15.40:0/2106467107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9eb0 0x563e167a37a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f07ec099c20 tx=0x7f07ec01f000 comp rx=0 tx=0).stop 2024-06-06T03:45:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.324+0000 7f08148f1b80 1 -- 172.21.15.40:0/2106467107 shutdown_connections 2024-06-06T03:45:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.324+0000 7f08148f1b80 1 --2- 172.21.15.40:0/2106467107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 0x563e167a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.324+0000 7f08148f1b80 1 --2- 172.21.15.40:0/2106467107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9590 0x563e167a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.324+0000 7f08148f1b80 1 --2- 172.21.15.40:0/2106467107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9eb0 0x563e167a37a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.324+0000 7f08148f1b80 1 -- 172.21.15.40:0/2106467107 >> 172.21.15.40:0/2106467107 conn(0x563e166d0210 msgr2=0x563e1679ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:21.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.324+0000 7f08148f1b80 1 -- 172.21.15.40:0/2106467107 shutdown_connections 2024-06-06T03:45:21.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.324+0000 7f08148f1b80 1 -- 172.21.15.40:0/2106467107 wait complete. 2024-06-06T03:45:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.325+0000 7f08148f1b80 1 -- start start 2024-06-06T03:45:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.325+0000 7f08148f1b80 1 Processor -- start 2024-06-06T03:45:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.325+0000 7f08148f1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 0x563e167afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.325+0000 7f08148f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9590 0x563e167acf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.325+0000 7f08148f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9eb0 0x563e167ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.325+0000 7f08148f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563e16695150 con 0x563e167a9eb0 2024-06-06T03:45:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.325+0000 7f08148f1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563e166f09a0 con 0x563e167a3ce0 2024-06-06T03:45:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.325+0000 7f08148f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563e167b0640 con 0x563e167a9590 2024-06-06T03:45:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.325+0000 7f08148f1b80 10 client.0 _ll_register_callbacks cb 0x563e1668a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:45:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.325+0000 7f07faffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9590 0x563e167acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.325+0000 7f0803fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9eb0 0x563e167ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.325+0000 7f07faffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9590 0x563e167acf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.40:51436/0 (socket says 172.21.15.40:51436) 2024-06-06T03:45:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.325+0000 7f0803fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9eb0 0x563e167ad4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48600/0 (socket says 172.21.15.40:48600) 2024-06-06T03:45:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.325+0000 7f07faffd700 1 -- 172.21.15.40:0/3841284053 learned_addr learned my addr 172.21.15.40:0/3841284053 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:45:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f08037fe700 1 --2- 172.21.15.40:0/3841284053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 0x563e167afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f0803fff700 1 -- 172.21.15.40:0/3841284053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9590 msgr2=0x563e167acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f0803fff700 1 --2- 172.21.15.40:0/3841284053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9590 0x563e167acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f0803fff700 1 -- 172.21.15.40:0/3841284053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 msgr2=0x563e167afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f0803fff700 1 --2- 172.21.15.40:0/3841284053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 0x563e167afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f0803fff700 1 -- 172.21.15.40:0/3841284053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e166ba8e0 con 0x563e167a9eb0 2024-06-06T03:45:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f08037fe700 1 --2- 172.21.15.40:0/3841284053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 0x563e167afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:45:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f0803fff700 1 --2- 172.21.15.40:0/3841284053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9eb0 0x563e167ad4a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0116e0 tx=0x7f07fc0119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97a79abe9ad8ee0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f07e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:45:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f07e2ffd700 1 -- 172.21.15.40:0/3841284053 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc004a30 con 0x563e167a9eb0 2024-06-06T03:45:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f08148f1b80 1 -- 172.21.15.40:0/3841284053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07ec0a1b70 con 0x563e167a9eb0 2024-06-06T03:45:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f07e2ffd700 1 -- 172.21.15.40:0/3841284053 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8426+0+0 (secure 0 0 0) 0x7f07fc004bd0 con 0x563e167a9eb0 2024-06-06T03:45:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f07e2ffd700 1 -- 172.21.15.40:0/3841284053 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f07fc004dd0 con 0x563e167a9eb0 2024-06-06T03:45:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f08148f1b80 10 client.15633 fetch_fsmap learned FSMap version 37 2024-06-06T03:45:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.326+0000 7f08148f1b80 1 -- 172.21.15.40:0/3841284053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563e167bad70 con 0x563e167a9eb0 2024-06-06T03:45:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.327+0000 7f07e2ffd700 1 -- 172.21.15.40:0/3841284053 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ec0a1b70 con 0x563e167a9eb0 2024-06-06T03:45:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.327+0000 7f07e2ffd700 1 -- 172.21.15.40:0/3841284053 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f07fc012c20 con 0x563e167a9eb0 2024-06-06T03:45:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.327+0000 7f08148f1b80 10 client.15633 fetch_fsmap finished waiting for FSMap version 37 2024-06-06T03:45:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.327+0000 7f08148f1b80 10 client.15633 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:45:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.327+0000 7f08148f1b80 20 client.15633 populate_metadata read hostname 'smithi040' 2024-06-06T03:45:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.327+0000 7f08148f1b80 1 --2- 172.21.15.40:0/3841284053 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563e167bd350 0x563e167bf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.327+0000 7f08148f1b80 4 client.15633 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:45:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.327+0000 7f08148f1b80 1 -- 172.21.15.40:0/3841284053 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e16701ee0 con 0x563e167bd350 2024-06-06T03:45:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.327+0000 7f08037fe700 1 --2- 172.21.15.40:0/3841284053 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563e167bd350 0x563e167bf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.328+0000 7f08037fe700 1 --2- 172.21.15.40:0/3841284053 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563e167bd350 0x563e167bf7b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed20d2c8575d4cd4 server_cookie=b2d6a21586fe9f59 in_seq=0 out_seq=0 2024-06-06T03:45:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.328+0000 7f07e2ffd700 10 client.15633 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:45:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.328+0000 7f07e2ffd700 1 -- 172.21.15.40:0/3841284053 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563e16701ee0 con 0x563e167bd350 2024-06-06T03:45:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.328+0000 7f07e2ffd700 10 client.15633 handle_command_reply: tid=0 2024-06-06T03:45:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.328+0000 7f08148f1b80 1 client.15633 shutdown 2024-06-06T03:45:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.329+0000 7f08148f1b80 1 -- 172.21.15.40:0/3841284053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9eb0 msgr2=0x563e167ad4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.329+0000 7f08148f1b80 1 --2- 172.21.15.40:0/3841284053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9eb0 0x563e167ad4a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0116e0 tx=0x7f07fc0119f0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.329+0000 7f08148f1b80 1 -- 172.21.15.40:0/3841284053 shutdown_connections 2024-06-06T03:45:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.329+0000 7f08148f1b80 1 --2- 172.21.15.40:0/3841284053 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563e167bd350 0x563e167bf7b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.329+0000 7f08148f1b80 1 --2- 172.21.15.40:0/3841284053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9eb0 0x563e167ad4a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.329+0000 7f07e2ffd700 0 client.15633 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:45:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.329+0000 7f08148f1b80 1 --2- 172.21.15.40:0/3841284053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 0x563e167afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.329+0000 7f08148f1b80 1 --2- 172.21.15.40:0/3841284053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9590 0x563e167acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.329+0000 7f08148f1b80 1 -- 172.21.15.40:0/3841284053 >> 172.21.15.40:0/3841284053 conn(0x563e166d0210 msgr2=0x563e1679dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.329+0000 7f08148f1b80 1 -- 172.21.15.40:0/3841284053 shutdown_connections 2024-06-06T03:45:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.329+0000 7f08148f1b80 1 -- 172.21.15.40:0/3841284053 wait complete. 2024-06-06T03:45:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.329+0000 7f08148f1b80 20 client.15633 trim_cache size 0 max 16384 2024-06-06T03:45:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.351+0000 7f08148f1b80 1 Processor -- start 2024-06-06T03:45:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.351+0000 7f08148f1b80 1 -- start start 2024-06-06T03:45:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.351+0000 7f08148f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a3ce0 0x563e167a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.351+0000 7f08148f1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a9590 0x563e167a8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.351+0000 7f08148f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9eb0 0x563e167af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.351+0000 7f08148f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f08041acb60 con 0x563e167a9eb0 2024-06-06T03:45:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.351+0000 7f08148f1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08041ae760 con 0x563e167a9590 2024-06-06T03:45:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.351+0000 7f08148f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f08041ad230 con 0x563e167a3ce0 2024-06-06T03:45:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.351+0000 7f0803fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9eb0 0x563e167af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.351+0000 7f08037fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a3ce0 0x563e167a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.351+0000 7f08037fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a3ce0 0x563e167a37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.40:51440/0 (socket says 172.21.15.40:51440) 2024-06-06T03:45:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.351+0000 7f0803fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9eb0 0x563e167af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48608/0 (socket says 172.21.15.40:48608) 2024-06-06T03:45:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.351+0000 7f08037fe700 1 -- 172.21.15.40:0/2066964301 learned_addr learned my addr 172.21.15.40:0/2066964301 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:45:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.352+0000 7f07faffd700 1 --2- 172.21.15.40:0/2066964301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a9590 0x563e167a8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.352+0000 7f08037fe700 1 -- 172.21.15.40:0/2066964301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a9590 msgr2=0x563e167a8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.352+0000 7f08037fe700 1 --2- 172.21.15.40:0/2066964301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a9590 0x563e167a8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.352+0000 7f08037fe700 1 -- 172.21.15.40:0/2066964301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9eb0 msgr2=0x563e167af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.352+0000 7f08037fe700 1 --2- 172.21.15.40:0/2066964301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9eb0 0x563e167af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.352+0000 7f08037fe700 1 -- 172.21.15.40:0/2066964301 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e167ab4d0 con 0x563e167a3ce0 2024-06-06T03:45:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.352+0000 7f0803fff700 1 --2- 172.21.15.40:0/2066964301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9eb0 0x563e167af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:45:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.352+0000 7f08037fe700 1 --2- 172.21.15.40:0/2066964301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a3ce0 0x563e167a37a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f07f4016690 tx=0x7f07f4015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1e7fdde83fbf7ad server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.352+0000 7f07f97fa700 1 -- 172.21.15.40:0/2066964301 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f4024410 con 0x563e167a3ce0 2024-06-06T03:45:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.352+0000 7f07f97fa700 1 -- 172.21.15.40:0/2066964301 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07f4024a90 con 0x563e167a3ce0 2024-06-06T03:45:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.352+0000 7f07f97fa700 1 -- 172.21.15.40:0/2066964301 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f4014ad0 con 0x563e167a3ce0 2024-06-06T03:45:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.353+0000 7f08148f1b80 1 -- 172.21.15.40:0/2066964301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a3ce0 msgr2=0x563e167a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.353+0000 7f08148f1b80 1 --2- 172.21.15.40:0/2066964301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a3ce0 0x563e167a37a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f07f4016690 tx=0x7f07f4015000 comp rx=0 tx=0).stop 2024-06-06T03:45:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.353+0000 7f08148f1b80 1 -- 172.21.15.40:0/2066964301 shutdown_connections 2024-06-06T03:45:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.353+0000 7f08148f1b80 1 --2- 172.21.15.40:0/2066964301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a3ce0 0x563e167a37a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.353+0000 7f08148f1b80 1 --2- 172.21.15.40:0/2066964301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9eb0 0x563e167af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.353+0000 7f08148f1b80 1 --2- 172.21.15.40:0/2066964301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a9590 0x563e167a8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.353+0000 7f08148f1b80 1 -- 172.21.15.40:0/2066964301 >> 172.21.15.40:0/2066964301 conn(0x563e166d0210 msgr2=0x563e167a8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.353+0000 7f08148f1b80 1 -- 172.21.15.40:0/2066964301 shutdown_connections 2024-06-06T03:45:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.353+0000 7f08148f1b80 1 -- 172.21.15.40:0/2066964301 wait complete. 2024-06-06T03:45:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.353+0000 7f08148f1b80 1 -- start start 2024-06-06T03:45:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f08148f1b80 1 Processor -- start 2024-06-06T03:45:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f08148f1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 0x563e167b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f08148f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9590 0x563e167b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f08148f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9eb0 0x563e167b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f08148f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f08041adfd0 con 0x563e167a9590 2024-06-06T03:45:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f08148f1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08041ad900 con 0x563e167a3ce0 2024-06-06T03:45:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f08148f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f08041aab60 con 0x563e167a9eb0 2024-06-06T03:45:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f08148f1b80 10 client.0 _ll_register_callbacks cb 0x563e1668a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:45:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f07faffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9590 0x563e167b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f0803fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9eb0 0x563e167b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f07faffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9590 0x563e167b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48622/0 (socket says 172.21.15.40:48622) 2024-06-06T03:45:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f07faffd700 1 -- 172.21.15.40:0/2863668138 learned_addr learned my addr 172.21.15.40:0/2863668138 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:45:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f08037fe700 1 --2- 172.21.15.40:0/2863668138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 0x563e167b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f07faffd700 1 -- 172.21.15.40:0/2863668138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9eb0 msgr2=0x563e167b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f07faffd700 1 --2- 172.21.15.40:0/2863668138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9eb0 0x563e167b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f07faffd700 1 -- 172.21.15.40:0/2863668138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 msgr2=0x563e167b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f07faffd700 1 --2- 172.21.15.40:0/2863668138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 0x563e167b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.354+0000 7f07faffd700 1 -- 172.21.15.40:0/2863668138 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e167b3360 con 0x563e167a9590 2024-06-06T03:45:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.355+0000 7f07faffd700 1 --2- 172.21.15.40:0/2863668138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9590 0x563e167b2860 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f07ec09a190 tx=0x7f07ec009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d8eec292e6187 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.355+0000 7f07e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:45:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.355+0000 7f07e2ffd700 1 -- 172.21.15.40:0/2863668138 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec080070 con 0x563e167a9590 2024-06-06T03:45:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.355+0000 7f07e2ffd700 1 -- 172.21.15.40:0/2863668138 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8426+0+0 (secure 0 0 0) 0x7f07ec0a5180 con 0x563e167a9590 2024-06-06T03:45:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.355+0000 7f08148f1b80 1 -- 172.21.15.40:0/2863668138 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07f4024a90 con 0x563e167a9590 2024-06-06T03:45:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.355+0000 7f07e2ffd700 1 -- 172.21.15.40:0/2863668138 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f07ec09d7c0 con 0x563e167a9590 2024-06-06T03:45:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.355+0000 7f08148f1b80 10 client.15639 fetch_fsmap learned FSMap version 37 2024-06-06T03:45:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.355+0000 7f08148f1b80 1 -- 172.21.15.40:0/2863668138 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563e1682d220 con 0x563e167a9590 2024-06-06T03:45:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.355+0000 7f07e2ffd700 1 -- 172.21.15.40:0/2863668138 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07f4024a90 con 0x563e167a9590 2024-06-06T03:45:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.355+0000 7f07e2ffd700 1 -- 172.21.15.40:0/2863668138 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f07ec09cc10 con 0x563e167a9590 2024-06-06T03:45:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.355+0000 7f08148f1b80 10 client.15639 fetch_fsmap finished waiting for FSMap version 37 2024-06-06T03:45:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.355+0000 7f08148f1b80 10 client.15639 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:45:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.356+0000 7f08148f1b80 20 client.15639 populate_metadata read hostname 'smithi040' 2024-06-06T03:45:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.356+0000 7f08148f1b80 1 --2- 172.21.15.40:0/2863668138 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563e167bd350 0x563e1682f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.356+0000 7f08148f1b80 4 client.15639 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 132.6, "await_for": 45.0}] 2024-06-06T03:45:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.356+0000 7f08148f1b80 1 -- 172.21.15.40:0/2863668138 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 132.6, "await_for": 45.0}) -- 0x563e166fc460 con 0x563e167bd350 2024-06-06T03:45:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.356+0000 7f08037fe700 1 --2- 172.21.15.40:0/2863668138 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563e167bd350 0x563e1682f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.356+0000 7f08037fe700 1 --2- 172.21.15.40:0/2863668138 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563e167bd350 0x563e1682f6a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0415aa344495448 server_cookie=c434a386a6e87220 in_seq=0 out_seq=0 2024-06-06T03:45:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:21.356+0000 7f07e2ffd700 10 client.15639 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:45:23.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:22 smithi131 ceph-mon[36687]: pgmap v207: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 239 op/s 2024-06-06T03:45:23.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:22 smithi040 ceph-mon[32220]: pgmap v207: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 239 op/s 2024-06-06T03:45:23.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:22 smithi104 ceph-mon[36569]: pgmap v207: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 239 op/s 2024-06-06T03:45:25.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:24 smithi131 ceph-mon[36687]: pgmap v208: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 239 op/s 2024-06-06T03:45:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:24 smithi040 ceph-mon[32220]: pgmap v208: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 239 op/s 2024-06-06T03:45:25.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:24 smithi104 ceph-mon[36569]: pgmap v208: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 239 op/s 2024-06-06T03:45:27.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:26 smithi131 ceph-mon[36687]: pgmap v209: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 240 op/s 2024-06-06T03:45:27.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:26 smithi040 ceph-mon[32220]: pgmap v209: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 240 op/s 2024-06-06T03:45:27.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:26 smithi104 ceph-mon[36569]: pgmap v209: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 240 op/s 2024-06-06T03:45:29.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:28 smithi131 ceph-mon[36687]: pgmap v210: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-06-06T03:45:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:28 smithi040 ceph-mon[32220]: pgmap v210: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-06-06T03:45:29.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:28 smithi104 ceph-mon[36569]: pgmap v210: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-06-06T03:45:31.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:30 smithi131 ceph-mon[36687]: pgmap v211: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.7 MiB/s wr, 313 op/s 2024-06-06T03:45:31.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:30 smithi040 ceph-mon[32220]: pgmap v211: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.7 MiB/s wr, 313 op/s 2024-06-06T03:45:31.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:30 smithi104 ceph-mon[36569]: pgmap v211: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.7 MiB/s wr, 313 op/s 2024-06-06T03:45:33.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:32 smithi131 ceph-mon[36687]: pgmap v212: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.6 MiB/s wr, 368 op/s 2024-06-06T03:45:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:32 smithi040 ceph-mon[32220]: pgmap v212: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.6 MiB/s wr, 368 op/s 2024-06-06T03:45:33.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:32 smithi104 ceph-mon[36569]: pgmap v212: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.6 MiB/s wr, 368 op/s 2024-06-06T03:45:35.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:34 smithi040 ceph-mon[32220]: pgmap v213: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.6 MiB/s wr, 362 op/s 2024-06-06T03:45:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f07e2ffd700 1 -- 172.21.15.40:0/2863668138 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563e166fc460 con 0x563e167bd350 2024-06-06T03:45:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f07e2ffd700 10 client.15639 handle_command_reply: tid=0 2024-06-06T03:45:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f08148f1b80 1 client.15639 shutdown 2024-06-06T03:45:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f08148f1b80 1 -- 172.21.15.40:0/2863668138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9590 msgr2=0x563e167b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f08148f1b80 1 --2- 172.21.15.40:0/2863668138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9590 0x563e167b2860 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f07ec09a190 tx=0x7f07ec009710 comp rx=0 tx=0).stop 2024-06-06T03:45:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f08148f1b80 1 -- 172.21.15.40:0/2863668138 shutdown_connections 2024-06-06T03:45:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f08148f1b80 1 --2- 172.21.15.40:0/2863668138 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563e167bd350 0x563e1682f6a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f08148f1b80 1 --2- 172.21.15.40:0/2863668138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563e167a9eb0 0x563e167b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f08148f1b80 1 --2- 172.21.15.40:0/2863668138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e167a3ce0 0x563e167b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f08148f1b80 1 --2- 172.21.15.40:0/2863668138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e167a9590 0x563e167b2860 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f07e2ffd700 0 client.15639 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:45:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f08148f1b80 1 -- 172.21.15.40:0/2863668138 >> 172.21.15.40:0/2863668138 conn(0x563e166d0210 msgr2=0x563e167a8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f08148f1b80 1 -- 172.21.15.40:0/2863668138 shutdown_connections 2024-06-06T03:45:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f08148f1b80 1 -- 172.21.15.40:0/2863668138 wait complete. 2024-06-06T03:45:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.083+0000 7f08148f1b80 20 client.15639 trim_cache size 0 max 16384 2024-06-06T03:45:35.085 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:45:35.085 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 37, 2024-06-06T03:45:35.085 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:45:35.085 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 18, 2024-06-06T03:45:35.086 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": { 2024-06-06T03:45:35.086 INFO:teuthology.orchestra.run.smithi040.stdout: "26b2a5b8": { 2024-06-06T03:45:35.086 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 18, 2024-06-06T03:45:35.086 INFO:teuthology.orchestra.run.smithi040.stdout: "age_ref": 0.0, 2024-06-06T03:45:35.086 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:45:35.086 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "QUIESCED", 2024-06-06T03:45:35.086 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:45:35.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:45:35.086 INFO:teuthology.orchestra.run.smithi040.stdout: "timeout": 90.0, 2024-06-06T03:45:35.086 INFO:teuthology.orchestra.run.smithi040.stdout: "expiration": 132.6, 2024-06-06T03:45:35.086 INFO:teuthology.orchestra.run.smithi040.stdout: "members": { 2024-06-06T03:45:35.086 INFO:teuthology.orchestra.run.smithi040.stdout: "file:/": { 2024-06-06T03:45:35.086 INFO:teuthology.orchestra.run.smithi040.stdout: "excluded": false, 2024-06-06T03:45:35.087 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:45:35.087 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "QUIESCED", 2024-06-06T03:45:35.087 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:45:35.087 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:45:35.087 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:45:35.087 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:45:35.087 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:45:35.087 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:45:35.087 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:45:35.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.084+0000 7f080a06e700 1 -- 172.21.15.40:0/3832263471 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f07f00a2eb0 msgr2=0x7f07f00a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.084+0000 7f080a06e700 1 --2- 172.21.15.40:0/3832263471 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f07f00a2eb0 0x7f07f00a5370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0804199b50 tx=0x7f07f4009040 comp rx=0 tx=0).stop 2024-06-06T03:45:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.084+0000 7f080a06e700 1 -- 172.21.15.40:0/3832263471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0804103150 msgr2=0x7f080419ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.084+0000 7f080a06e700 1 --2- 172.21.15.40:0/3832263471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0804103150 0x7f080419ed90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0059d0 tx=0x7f07ec0038b0 comp rx=0 tx=0).stop 2024-06-06T03:45:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.084+0000 7f080a06e700 1 -- 172.21.15.40:0/3832263471 shutdown_connections 2024-06-06T03:45:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.084+0000 7f080a06e700 1 --2- 172.21.15.40:0/3832263471 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f07f00a2eb0 0x7f07f00a5370 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.084+0000 7f080a06e700 1 --2- 172.21.15.40:0/3832263471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0804107790 0x7f0804198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.084+0000 7f080a06e700 1 --2- 172.21.15.40:0/3832263471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08041027a0 0x7f080419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.084+0000 7f080a06e700 1 --2- 172.21.15.40:0/3832263471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0804103150 0x7f080419ed90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.084+0000 7f080a06e700 1 -- 172.21.15.40:0/3832263471 >> 172.21.15.40:0/3832263471 conn(0x7f0804079560 msgr2=0x7f0804101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.085+0000 7f080a06e700 1 -- 172.21.15.40:0/3832263471 shutdown_connections 2024-06-06T03:45:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:35.085+0000 7f080a06e700 1 -- 172.21.15.40:0/3832263471 wait complete. 2024-06-06T03:45:35.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:34 smithi104 ceph-mon[36569]: pgmap v213: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.6 MiB/s wr, 362 op/s 2024-06-06T03:45:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:34 smithi131 ceph-mon[36687]: pgmap v213: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.6 MiB/s wr, 362 op/s 2024-06-06T03:45:35.785 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '26b2a5b8', quiesce took 20.6 seconds. Will release after: -8.000000000000002 2024-06-06T03:45:35.785 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:45:36.044 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:45:37.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:36 smithi040 ceph-mon[32220]: pgmap v214: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 621 op/s 2024-06-06T03:45:37.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:36 smithi104 ceph-mon[36569]: pgmap v214: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 621 op/s 2024-06-06T03:45:37.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:36 smithi131 ceph-mon[36687]: pgmap v214: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 621 op/s 2024-06-06T03:45:38.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.938+0000 7f186c8ab700 1 -- 172.21.15.40:0/1856753906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1868102380 msgr2=0x7f1868105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:38.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.938+0000 7f186c8ab700 1 --2- 172.21.15.40:0/1856753906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1868102380 0x7f1868105ff0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f185c0099c0 tx=0x7f185c009cd0 comp rx=0 tx=0).stop 2024-06-06T03:45:38.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.938+0000 7f186c8ab700 1 -- 172.21.15.40:0/1856753906 shutdown_connections 2024-06-06T03:45:38.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.938+0000 7f186c8ab700 1 --2- 172.21.15.40:0/1856753906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18681019d0 0x7f1868101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.938+0000 7f186c8ab700 1 --2- 172.21.15.40:0/1856753906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1868106720 0x7f186810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.938+0000 7f186c8ab700 1 --2- 172.21.15.40:0/1856753906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1868102380 0x7f1868105ff0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.938+0000 7f186c8ab700 1 -- 172.21.15.40:0/1856753906 >> 172.21.15.40:0/1856753906 conn(0x7f1868078340 msgr2=0x7f1868078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.938+0000 7f186c8ab700 1 -- 172.21.15.40:0/1856753906 shutdown_connections 2024-06-06T03:45:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.938+0000 7f186c8ab700 1 -- 172.21.15.40:0/1856753906 wait complete. 2024-06-06T03:45:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.939+0000 7f186c8ab700 1 Processor -- start 2024-06-06T03:45:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.939+0000 7f186c8ab700 1 -- start start 2024-06-06T03:45:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186c8ab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18681019d0 0x7f186819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186c8ab700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1868102380 0x7f186819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186c8ab700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1868106720 0x7f18681989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186c8ab700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1868111690 con 0x7f18681019d0 2024-06-06T03:45:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186c8ab700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1868111510 con 0x7f1868106720 2024-06-06T03:45:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186c8ab700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1868111810 con 0x7f1868102380 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18681019d0 0x7f186819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18681019d0 0x7f186819e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58372/0 (socket says 172.21.15.40:58372) 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186659c700 1 -- 172.21.15.40:0/1769447544 learned_addr learned my addr 172.21.15.40:0/1769447544 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f1865d9b700 1 --2- 172.21.15.40:0/1769447544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1868102380 0x7f186819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f1866d9d700 1 --2- 172.21.15.40:0/1769447544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1868106720 0x7f18681989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186659c700 1 -- 172.21.15.40:0/1769447544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1868102380 msgr2=0x7f186819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186659c700 1 --2- 172.21.15.40:0/1769447544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1868102380 0x7f186819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186659c700 1 -- 172.21.15.40:0/1769447544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1868106720 msgr2=0x7f18681989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186659c700 1 --2- 172.21.15.40:0/1769447544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1868106720 0x7f18681989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186659c700 1 -- 172.21.15.40:0/1769447544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1868199100 con 0x7f18681019d0 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f186659c700 1 --2- 172.21.15.40:0/1769447544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18681019d0 0x7f186819e7e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f185800c9b0 tx=0x7f185800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.940+0000 7f18577fe700 1 -- 172.21.15.40:0/1769447544 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858007b00 con 0x7f18681019d0 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.941+0000 7f186c8ab700 1 -- 172.21.15.40:0/1769447544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18681993f0 con 0x7f18681019d0 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.941+0000 7f18577fe700 1 -- 172.21.15.40:0/1769447544 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f185800ce50 con 0x7f18681019d0 2024-06-06T03:45:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.941+0000 7f18577fe700 1 -- 172.21.15.40:0/1769447544 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858018660 con 0x7f18681019d0 2024-06-06T03:45:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.941+0000 7f186c8ab700 1 -- 172.21.15.40:0/1769447544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18681a56b0 con 0x7f18681019d0 2024-06-06T03:45:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.942+0000 7f186c8ab700 1 -- 172.21.15.40:0/1769447544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1868102ad0 con 0x7f18681019d0 2024-06-06T03:45:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.943+0000 7f18577fe700 1 -- 172.21.15.40:0/1769447544 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f185800f450 con 0x7f18681019d0 2024-06-06T03:45:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.944+0000 7f18577fe700 1 --2- 172.21.15.40:0/1769447544 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f18500a2d90 0x7f18500a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.944+0000 7f1865d9b700 1 --2- 172.21.15.40:0/1769447544 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f18500a2d90 0x7f18500a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.944+0000 7f18577fe700 1 -- 172.21.15.40:0/1769447544 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8426+0+0 (secure 0 0 0) 0x7f18580cf320 con 0x7f18681019d0 2024-06-06T03:45:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.944+0000 7f1865d9b700 1 --2- 172.21.15.40:0/1769447544 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f18500a2d90 0x7f18500a5250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f185c009fd0 tx=0x7f185c005fb0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:37.948+0000 7f18577fe700 1 -- 172.21.15.40:0/1769447544 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1858014030 con 0x7f18681019d0 2024-06-06T03:45:38.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:38.202+0000 7f186c8ab700 1 -- 172.21.15.40:0/1769447544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f186802dca0 con 0x7f18681019d0 2024-06-06T03:45:38.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:38.202+0000 7f18577fe700 1 -- 172.21.15.40:0/1769447544 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11940 (secure 0 0 0) 0x7f185809bfa0 con 0x7f18681019d0 2024-06-06T03:45:38.203 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:45:38.203 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:44:10.814777+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24443,24425,24398,24403]},"id":1}]} 2024-06-06T03:45:38.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:38.205+0000 7f186c8ab700 1 -- 172.21.15.40:0/1769447544 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f18500a2d90 msgr2=0x7f18500a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:38.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:38.205+0000 7f186c8ab700 1 --2- 172.21.15.40:0/1769447544 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f18500a2d90 0x7f18500a5250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f185c009fd0 tx=0x7f185c005fb0 comp rx=0 tx=0).stop 2024-06-06T03:45:38.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:38.205+0000 7f186c8ab700 1 -- 172.21.15.40:0/1769447544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18681019d0 msgr2=0x7f186819e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:38.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:38.205+0000 7f186c8ab700 1 --2- 172.21.15.40:0/1769447544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18681019d0 0x7f186819e7e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f185800c9b0 tx=0x7f185800ccc0 comp rx=0 tx=0).stop 2024-06-06T03:45:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:38.205+0000 7f186c8ab700 1 -- 172.21.15.40:0/1769447544 shutdown_connections 2024-06-06T03:45:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:38.205+0000 7f186c8ab700 1 --2- 172.21.15.40:0/1769447544 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f18500a2d90 0x7f18500a5250 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:38.205+0000 7f186c8ab700 1 --2- 172.21.15.40:0/1769447544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1868106720 0x7f18681989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:38.205+0000 7f186c8ab700 1 --2- 172.21.15.40:0/1769447544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18681019d0 0x7f186819e7e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:38.205+0000 7f186c8ab700 1 --2- 172.21.15.40:0/1769447544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1868102380 0x7f186819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:38.205+0000 7f186c8ab700 1 -- 172.21.15.40:0/1769447544 >> 172.21.15.40:0/1769447544 conn(0x7f1868078340 msgr2=0x7f186806ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:38.205+0000 7f186c8ab700 1 -- 172.21.15.40:0/1769447544 shutdown_connections 2024-06-06T03:45:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:38.205+0000 7f186c8ab700 1 -- 172.21.15.40:0/1769447544 wait complete. 2024-06-06T03:45:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 37 2024-06-06T03:45:38.895 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 26b2a5b8 --release --await' 2024-06-06T03:45:38.896 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 26b2a5b8 --release --await 2024-06-06T03:45:39.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:38 smithi040 ceph-mon[32220]: pgmap v215: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 596 op/s 2024-06-06T03:45:39.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:38 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1769447544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:45:39.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:38 smithi104 ceph-mon[36569]: pgmap v215: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 596 op/s 2024-06-06T03:45:39.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:38 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1769447544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:45:39.156 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:45:39.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:38 smithi131 ceph-mon[36687]: pgmap v215: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 596 op/s 2024-06-06T03:45:39.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:38 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1769447544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:45:40.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:40 smithi040 ceph-mon[32220]: pgmap v216: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 607 op/s 2024-06-06T03:45:41.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:40 smithi104 ceph-mon[36569]: pgmap v216: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 607 op/s 2024-06-06T03:45:41.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.098+0000 7f5349fe5700 1 -- 172.21.15.40:0/1553252386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5344107840 msgr2=0x7f534410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:41.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.098+0000 7f5349fe5700 1 --2- 172.21.15.40:0/1553252386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5344107840 0x7f534410f3c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f533c00b0d0 tx=0x7f533c00b3e0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.098+0000 7f5349fe5700 1 -- 172.21.15.40:0/1553252386 shutdown_connections 2024-06-06T03:45:41.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.098+0000 7f5349fe5700 1 --2- 172.21.15.40:0/1553252386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5344102740 0x7f5344102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.098+0000 7f5349fe5700 1 --2- 172.21.15.40:0/1553252386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5344107840 0x7f534410f3c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.098+0000 7f5349fe5700 1 --2- 172.21.15.40:0/1553252386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53441030f0 0x7f5344107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.098+0000 7f5349fe5700 1 -- 172.21.15.40:0/1553252386 >> 172.21.15.40:0/1553252386 conn(0x7f53440795f0 msgr2=0x7f5344079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:41.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.098+0000 7f5349fe5700 1 -- 172.21.15.40:0/1553252386 shutdown_connections 2024-06-06T03:45:41.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.098+0000 7f5349fe5700 1 -- 172.21.15.40:0/1553252386 wait complete. 2024-06-06T03:45:41.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.098+0000 7f5349fe5700 1 Processor -- start 2024-06-06T03:45:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f5349fe5700 1 -- start start 2024-06-06T03:45:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f5349fe5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5344102740 0x7f534419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f5349fe5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53441030f0 0x7f534419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f5349fe5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5344107840 0x7f5344198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f5349fe5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5344112780 con 0x7f5344102740 2024-06-06T03:45:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f5349fe5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5344112600 con 0x7f53441030f0 2024-06-06T03:45:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f5349fe5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5344112900 con 0x7f5344107840 2024-06-06T03:45:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f53437fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5344102740 0x7f534419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f53437fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5344102740 0x7f534419e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57840/0 (socket says 172.21.15.40:57840) 2024-06-06T03:45:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f53437fe700 1 -- 172.21.15.40:0/2806585369 learned_addr learned my addr 172.21.15.40:0/2806585369 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:45:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f5342ffd700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53441030f0 0x7f534419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f5343fff700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5344107840 0x7f5344198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f53437fe700 1 -- 172.21.15.40:0/2806585369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5344107840 msgr2=0x7f5344198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f53437fe700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5344107840 0x7f5344198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f53437fe700 1 -- 172.21.15.40:0/2806585369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53441030f0 msgr2=0x7f534419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f53437fe700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53441030f0 0x7f534419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f53437fe700 1 -- 172.21.15.40:0/2806585369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53441990c0 con 0x7f5344102740 2024-06-06T03:45:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.099+0000 7f5343fff700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5344107840 0x7f5344198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:45:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.100+0000 7f53437fe700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5344102740 0x7f534419e7a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f533400d980 tx=0x7f533400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.100+0000 7f5340ff9700 1 -- 172.21.15.40:0/2806585369 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5334017070 con 0x7f5344102740 2024-06-06T03:45:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.100+0000 7f5340ff9700 1 -- 172.21.15.40:0/2806585369 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5334005020 con 0x7f5344102740 2024-06-06T03:45:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.100+0000 7f5340ff9700 1 -- 172.21.15.40:0/2806585369 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5334009bc0 con 0x7f5344102740 2024-06-06T03:45:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.100+0000 7f5349fe5700 1 -- 172.21.15.40:0/2806585369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53441993b0 con 0x7f5344102740 2024-06-06T03:45:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.100+0000 7f5349fe5700 1 -- 172.21.15.40:0/2806585369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53441a5670 con 0x7f5344102740 2024-06-06T03:45:41.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.102+0000 7f5340ff9700 1 -- 172.21.15.40:0/2806585369 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f53340051d0 con 0x7f5344102740 2024-06-06T03:45:41.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.103+0000 7f5340ff9700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f532c0a2e10 0x7f532c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.103+0000 7f5342ffd700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f532c0a2e10 0x7f532c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.103+0000 7f5340ff9700 1 -- 172.21.15.40:0/2806585369 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8426+0+0 (secure 0 0 0) 0x7f53340cf2d0 con 0x7f5344102740 2024-06-06T03:45:41.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.104+0000 7f5342ffd700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f532c0a2e10 0x7f532c0a52d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5344102010 tx=0x7f5338009260 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:41.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.104+0000 7f5354868b80 1 Processor -- start 2024-06-06T03:45:41.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.104+0000 7f5354868b80 1 -- start start 2024-06-06T03:45:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.104+0000 7f5354868b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218050 0x55fa2a218430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.104+0000 7f5354868b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218970 0x55fa2a2122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.104+0000 7f5354868b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 0x55fa2a212c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.104+0000 7f5354868b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fa2a15f910 con 0x55fa2a218970 2024-06-06T03:45:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.104+0000 7f5354868b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fa2a107820 con 0x55fa2a2127e0 2024-06-06T03:45:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.104+0000 7f5354868b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fa2a15b8d0 con 0x55fa2a218050 2024-06-06T03:45:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.104+0000 7f5342ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218970 0x55fa2a2122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.104+0000 7f5342ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218970 0x55fa2a2122a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57844/0 (socket says 172.21.15.40:57844) 2024-06-06T03:45:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.104+0000 7f5342ffd700 1 -- 172.21.15.40:0/73913507 learned_addr learned my addr 172.21.15.40:0/73913507 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:45:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.105+0000 7f5343fff700 1 --2- 172.21.15.40:0/73913507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 0x55fa2a212c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.105+0000 7f53437fe700 1 --2- 172.21.15.40:0/73913507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218050 0x55fa2a218430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.105+0000 7f5342ffd700 1 -- 172.21.15.40:0/73913507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218050 msgr2=0x55fa2a218430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.105+0000 7f5342ffd700 1 --2- 172.21.15.40:0/73913507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218050 0x55fa2a218430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.105+0000 7f5342ffd700 1 -- 172.21.15.40:0/73913507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 msgr2=0x55fa2a212c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.105+0000 7f5342ffd700 1 --2- 172.21.15.40:0/73913507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 0x55fa2a212c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.105+0000 7f5342ffd700 1 -- 172.21.15.40:0/73913507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa2a0fe050 con 0x55fa2a218970 2024-06-06T03:45:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.105+0000 7f5342ffd700 1 --2- 172.21.15.40:0/73913507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218970 0x55fa2a2122a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5338011d50 tx=0x7f5338014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbba8abb3923bb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.105+0000 7f5329ffb700 1 -- 172.21.15.40:0/73913507 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533800ebd0 con 0x55fa2a218970 2024-06-06T03:45:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.105+0000 7f5329ffb700 1 -- 172.21.15.40:0/73913507 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533800ed70 con 0x55fa2a218970 2024-06-06T03:45:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.105+0000 7f5329ffb700 1 -- 172.21.15.40:0/73913507 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533801d7b0 con 0x55fa2a218970 2024-06-06T03:45:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.106+0000 7f5354868b80 1 -- 172.21.15.40:0/73913507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218970 msgr2=0x55fa2a2122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.106+0000 7f5354868b80 1 --2- 172.21.15.40:0/73913507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218970 0x55fa2a2122a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5338011d50 tx=0x7f5338014000 comp rx=0 tx=0).stop 2024-06-06T03:45:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.106+0000 7f5354868b80 1 -- 172.21.15.40:0/73913507 shutdown_connections 2024-06-06T03:45:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.106+0000 7f5354868b80 1 --2- 172.21.15.40:0/73913507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218050 0x55fa2a218430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.106+0000 7f5354868b80 1 --2- 172.21.15.40:0/73913507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 0x55fa2a212c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.106+0000 7f5354868b80 1 --2- 172.21.15.40:0/73913507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218970 0x55fa2a2122a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.106+0000 7f5354868b80 1 -- 172.21.15.40:0/73913507 >> 172.21.15.40:0/73913507 conn(0x55fa2a138dc0 msgr2=0x55fa2a209120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.106+0000 7f5354868b80 1 -- 172.21.15.40:0/73913507 shutdown_connections 2024-06-06T03:45:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.106+0000 7f5354868b80 1 -- 172.21.15.40:0/73913507 wait complete. 2024-06-06T03:45:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.106+0000 7f5354868b80 1 -- start start 2024-06-06T03:45:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.107+0000 7f5354868b80 1 Processor -- start 2024-06-06T03:45:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.107+0000 7f5354868b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 0x55fa2a21e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.107+0000 7f5354868b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218050 0x55fa2a21b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.107+0000 7f5354868b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218970 0x55fa2a21beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.107+0000 7f5354868b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fa2a21f3e0 con 0x55fa2a218050 2024-06-06T03:45:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.107+0000 7f5354868b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fa2a0ffcf0 con 0x55fa2a2127e0 2024-06-06T03:45:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.107+0000 7f5354868b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fa2a21f560 con 0x55fa2a218970 2024-06-06T03:45:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.107+0000 7f5354868b80 10 client.0 _ll_register_callbacks cb 0x55fa2a0f9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:45:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.107+0000 7f5342ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218050 0x55fa2a21b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.107+0000 7f5342ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218050 0x55fa2a21b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57846/0 (socket says 172.21.15.40:57846) 2024-06-06T03:45:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.107+0000 7f5342ffd700 1 -- 172.21.15.40:0/3163444655 learned_addr learned my addr 172.21.15.40:0/3163444655 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:45:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.107+0000 7f53437fe700 1 --2- 172.21.15.40:0/3163444655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 0x55fa2a21e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.107+0000 7f5343fff700 1 --2- 172.21.15.40:0/3163444655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218970 0x55fa2a21beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.108+0000 7f5342ffd700 1 -- 172.21.15.40:0/3163444655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218970 msgr2=0x55fa2a21beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.108+0000 7f5342ffd700 1 --2- 172.21.15.40:0/3163444655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218970 0x55fa2a21beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.108+0000 7f5342ffd700 1 -- 172.21.15.40:0/3163444655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 msgr2=0x55fa2a21e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.108+0000 7f5342ffd700 1 --2- 172.21.15.40:0/3163444655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 0x55fa2a21e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.108+0000 7f5342ffd700 1 -- 172.21.15.40:0/3163444655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa2a0ae9d0 con 0x55fa2a218050 2024-06-06T03:45:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.108+0000 7f53437fe700 1 --2- 172.21.15.40:0/3163444655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 0x55fa2a21e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:45:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.108+0000 7f5342ffd700 1 --2- 172.21.15.40:0/3163444655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218050 0x55fa2a21b970 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5338011d20 tx=0x7f533800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19658c634e31daf7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.108+0000 7f5322ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:45:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.108+0000 7f5322ffd700 1 -- 172.21.15.40:0/3163444655 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338033060 con 0x55fa2a218050 2024-06-06T03:45:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.108+0000 7f5322ffd700 1 -- 172.21.15.40:0/3163444655 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8426+0+0 (secure 0 0 0) 0x7f5338005250 con 0x55fa2a218050 2024-06-06T03:45:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.108+0000 7f5354868b80 1 -- 172.21.15.40:0/3163444655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f533800ed70 con 0x55fa2a218050 2024-06-06T03:45:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.108+0000 7f5322ffd700 1 -- 172.21.15.40:0/3163444655 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5338005450 con 0x55fa2a218050 2024-06-06T03:45:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.108+0000 7f5354868b80 10 client.15657 fetch_fsmap learned FSMap version 37 2024-06-06T03:45:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.108+0000 7f5354868b80 1 -- 172.21.15.40:0/3163444655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fa2a229c70 con 0x55fa2a218050 2024-06-06T03:45:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.109+0000 7f5322ffd700 1 -- 172.21.15.40:0/3163444655 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533800ed70 con 0x55fa2a218050 2024-06-06T03:45:41.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.109+0000 7f5322ffd700 1 -- 172.21.15.40:0/3163444655 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f5338003980 con 0x55fa2a218050 2024-06-06T03:45:41.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.109+0000 7f5354868b80 10 client.15657 fetch_fsmap finished waiting for FSMap version 37 2024-06-06T03:45:41.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.109+0000 7f5354868b80 10 client.15657 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:45:41.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.109+0000 7f5354868b80 20 client.15657 populate_metadata read hostname 'smithi040' 2024-06-06T03:45:41.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.109+0000 7f5354868b80 1 --2- 172.21.15.40:0/3163444655 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55fa2a22c250 0x55fa2a22e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.109+0000 7f5354868b80 4 client.15657 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:45:41.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.109+0000 7f5354868b80 1 -- 172.21.15.40:0/3163444655 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa2a16ce10 con 0x55fa2a22c250 2024-06-06T03:45:41.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.109+0000 7f53437fe700 1 --2- 172.21.15.40:0/3163444655 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55fa2a22c250 0x55fa2a22e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.110+0000 7f5322ffd700 10 client.15657 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:45:41.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.110+0000 7f53437fe700 1 --2- 172.21.15.40:0/3163444655 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55fa2a22c250 0x55fa2a22e640 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6eab21838feb9d server_cookie=fb79e2170ff0f3bc in_seq=0 out_seq=0 2024-06-06T03:45:41.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.110+0000 7f5322ffd700 1 -- 172.21.15.40:0/3163444655 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fa2a16ce10 con 0x55fa2a22c250 2024-06-06T03:45:41.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.110+0000 7f5322ffd700 10 client.15657 handle_command_reply: tid=0 2024-06-06T03:45:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.110+0000 7f5354868b80 1 client.15657 shutdown 2024-06-06T03:45:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.111+0000 7f5354868b80 1 -- 172.21.15.40:0/3163444655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218050 msgr2=0x55fa2a21b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.111+0000 7f5354868b80 1 --2- 172.21.15.40:0/3163444655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218050 0x55fa2a21b970 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5338011d20 tx=0x7f533800ec80 comp rx=0 tx=0).stop 2024-06-06T03:45:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.111+0000 7f5354868b80 1 -- 172.21.15.40:0/3163444655 shutdown_connections 2024-06-06T03:45:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.111+0000 7f5354868b80 1 --2- 172.21.15.40:0/3163444655 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55fa2a22c250 0x55fa2a22e640 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.111+0000 7f5354868b80 1 --2- 172.21.15.40:0/3163444655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218970 0x55fa2a21beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.111+0000 7f5354868b80 1 --2- 172.21.15.40:0/3163444655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 0x55fa2a21e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.111+0000 7f5354868b80 1 --2- 172.21.15.40:0/3163444655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218050 0x55fa2a21b970 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.111+0000 7f5354868b80 1 -- 172.21.15.40:0/3163444655 >> 172.21.15.40:0/3163444655 conn(0x55fa2a138dc0 msgr2=0x55fa2a20ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.111+0000 7f5322ffd700 0 client.15657 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:45:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.111+0000 7f5354868b80 1 -- 172.21.15.40:0/3163444655 shutdown_connections 2024-06-06T03:45:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.111+0000 7f5354868b80 1 -- 172.21.15.40:0/3163444655 wait complete. 2024-06-06T03:45:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.111+0000 7f5354868b80 20 client.15657 trim_cache size 0 max 16384 2024-06-06T03:45:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.135+0000 7f5354868b80 1 Processor -- start 2024-06-06T03:45:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.135+0000 7f5354868b80 1 -- start start 2024-06-06T03:45:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.135+0000 7f5354868b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 0x55fa2a219520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.135+0000 7f5354868b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218050 0x55fa2a233c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.135+0000 7f5354868b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218970 0x55fa2a234150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.135+0000 7f5354868b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f53441aaa40 con 0x55fa2a218970 2024-06-06T03:45:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.135+0000 7f5354868b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f53441ac640 con 0x55fa2a2127e0 2024-06-06T03:45:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.135+0000 7f5354868b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f53441ab110 con 0x55fa2a218050 2024-06-06T03:45:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.135+0000 7f5343fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218970 0x55fa2a234150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.135+0000 7f5343fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218970 0x55fa2a234150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57852/0 (socket says 172.21.15.40:57852) 2024-06-06T03:45:41.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.135+0000 7f5343fff700 1 -- 172.21.15.40:0/1069862346 learned_addr learned my addr 172.21.15.40:0/1069862346 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:45:41.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.135+0000 7f53437fe700 1 --2- 172.21.15.40:0/1069862346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 0x55fa2a219520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.136+0000 7f5342ffd700 1 --2- 172.21.15.40:0/1069862346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218050 0x55fa2a233c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.136+0000 7f5343fff700 1 -- 172.21.15.40:0/1069862346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218050 msgr2=0x55fa2a233c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:41.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.136+0000 7f5343fff700 1 --2- 172.21.15.40:0/1069862346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218050 0x55fa2a233c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.136+0000 7f5343fff700 1 -- 172.21.15.40:0/1069862346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 msgr2=0x55fa2a219520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:41.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.136+0000 7f5343fff700 1 --2- 172.21.15.40:0/1069862346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 0x55fa2a219520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.136+0000 7f5343fff700 1 -- 172.21.15.40:0/1069862346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa2a21b740 con 0x55fa2a218970 2024-06-06T03:45:41.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.136+0000 7f5343fff700 1 --2- 172.21.15.40:0/1069862346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218970 0x55fa2a234150 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f533c009e40 tx=0x7f533c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28fe2263e53d6fb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:41.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.136+0000 7f53297fa700 1 -- 172.21.15.40:0/1069862346 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c00bca0 con 0x55fa2a218970 2024-06-06T03:45:41.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.136+0000 7f53297fa700 1 -- 172.21.15.40:0/1069862346 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533c009480 con 0x55fa2a218970 2024-06-06T03:45:41.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.136+0000 7f53297fa700 1 -- 172.21.15.40:0/1069862346 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c026470 con 0x55fa2a218970 2024-06-06T03:45:41.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.137+0000 7f5354868b80 1 -- 172.21.15.40:0/1069862346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218970 msgr2=0x55fa2a234150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:41.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.137+0000 7f5354868b80 1 --2- 172.21.15.40:0/1069862346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218970 0x55fa2a234150 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f533c009e40 tx=0x7f533c0092e0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.137+0000 7f5354868b80 1 -- 172.21.15.40:0/1069862346 shutdown_connections 2024-06-06T03:45:41.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.137+0000 7f5354868b80 1 --2- 172.21.15.40:0/1069862346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218970 0x55fa2a234150 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.137+0000 7f5354868b80 1 --2- 172.21.15.40:0/1069862346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a2127e0 0x55fa2a219520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.137+0000 7f5354868b80 1 --2- 172.21.15.40:0/1069862346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a218050 0x55fa2a233c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.137+0000 7f5354868b80 1 -- 172.21.15.40:0/1069862346 >> 172.21.15.40:0/1069862346 conn(0x55fa2a138dc0 msgr2=0x55fa2a218e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:41.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.137+0000 7f5354868b80 1 -- 172.21.15.40:0/1069862346 shutdown_connections 2024-06-06T03:45:41.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.137+0000 7f5354868b80 1 -- 172.21.15.40:0/1069862346 wait complete. 2024-06-06T03:45:41.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.137+0000 7f5354868b80 1 -- start start 2024-06-06T03:45:41.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.137+0000 7f5354868b80 1 Processor -- start 2024-06-06T03:45:41.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5354868b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a2127e0 0x55fa2a224a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5354868b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218050 0x55fa2a224f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5354868b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a218970 0x55fa2a2217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5354868b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f53441abeb0 con 0x55fa2a218050 2024-06-06T03:45:41.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5354868b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f53441ab7e0 con 0x55fa2a218970 2024-06-06T03:45:41.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5354868b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f53441a8a00 con 0x55fa2a2127e0 2024-06-06T03:45:41.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5354868b80 10 client.0 _ll_register_callbacks cb 0x55fa2a0f9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:45:41.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5342ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218050 0x55fa2a224f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5342ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218050 0x55fa2a224f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57866/0 (socket says 172.21.15.40:57866) 2024-06-06T03:45:41.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5342ffd700 1 -- 172.21.15.40:0/2776951709 learned_addr learned my addr 172.21.15.40:0/2776951709 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:45:41.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5343fff700 1 --2- 172.21.15.40:0/2776951709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a218970 0x55fa2a2217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f53437fe700 1 --2- 172.21.15.40:0/2776951709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a2127e0 0x55fa2a224a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5342ffd700 1 -- 172.21.15.40:0/2776951709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a2127e0 msgr2=0x55fa2a224a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:41.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5342ffd700 1 --2- 172.21.15.40:0/2776951709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a2127e0 0x55fa2a224a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5342ffd700 1 -- 172.21.15.40:0/2776951709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a218970 msgr2=0x55fa2a2217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5342ffd700 1 --2- 172.21.15.40:0/2776951709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a218970 0x55fa2a2217f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5342ffd700 1 -- 172.21.15.40:0/2776951709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa2a221eb0 con 0x55fa2a218050 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5343fff700 1 --2- 172.21.15.40:0/2776951709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a218970 0x55fa2a2217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5342ffd700 1 --2- 172.21.15.40:0/2776951709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218050 0x55fa2a224f40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f53380117d0 tx=0x7f533800eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d65e6698470a56 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.138+0000 7f5322ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.139+0000 7f5322ffd700 1 -- 172.21.15.40:0/2776951709 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338033040 con 0x55fa2a218050 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.139+0000 7f5322ffd700 1 -- 172.21.15.40:0/2776951709 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8426+0+0 (secure 0 0 0) 0x7f53380055f0 con 0x55fa2a218050 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.139+0000 7f5354868b80 1 -- 172.21.15.40:0/2776951709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fa2a167390 con 0x55fa2a218050 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.139+0000 7f5322ffd700 1 -- 172.21.15.40:0/2776951709 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5338049050 con 0x55fa2a218050 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.139+0000 7f5354868b80 10 client.15663 fetch_fsmap learned FSMap version 37 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.139+0000 7f5354868b80 1 -- 172.21.15.40:0/2776951709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fa2a297570 con 0x55fa2a218050 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.139+0000 7f5322ffd700 1 -- 172.21.15.40:0/2776951709 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fa2a167390 con 0x55fa2a218050 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.139+0000 7f5322ffd700 1 -- 172.21.15.40:0/2776951709 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f5338040070 con 0x55fa2a218050 2024-06-06T03:45:41.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.139+0000 7f5354868b80 10 client.15663 fetch_fsmap finished waiting for FSMap version 37 2024-06-06T03:45:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.139+0000 7f5354868b80 10 client.15663 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:45:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.139+0000 7f5354868b80 20 client.15663 populate_metadata read hostname 'smithi040' 2024-06-06T03:45:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.139+0000 7f5354868b80 1 --2- 172.21.15.40:0/2776951709 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55fa2a297c50 0x55fa2a29a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:45:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.139+0000 7f5354868b80 4 client.15663 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "26b2a5b8", "release": true, "await": true}] 2024-06-06T03:45:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.139+0000 7f5354868b80 1 -- 172.21.15.40:0/2776951709 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "set_id": "26b2a5b8", "release": true, "await": true}) -- 0x55fa29fe1350 con 0x55fa2a297c50 2024-06-06T03:45:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.140+0000 7f53437fe700 1 --2- 172.21.15.40:0/2776951709 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55fa2a297c50 0x55fa2a29a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:45:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.140+0000 7f53437fe700 1 --2- 172.21.15.40:0/2776951709 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55fa2a297c50 0x55fa2a29a070 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b6488ba7d07a538 server_cookie=1e9f959603ad606c in_seq=0 out_seq=0 2024-06-06T03:45:41.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:41.140+0000 7f5322ffd700 10 client.15663 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:45:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:40 smithi131 ceph-mon[36687]: pgmap v216: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 607 op/s 2024-06-06T03:45:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5322ffd700 1 -- 172.21.15.40:0/2776951709 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fa29fe1350 con 0x55fa2a297c50 2024-06-06T03:45:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5322ffd700 10 client.15663 handle_command_reply: tid=0 2024-06-06T03:45:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5354868b80 1 client.15663 shutdown 2024-06-06T03:45:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5354868b80 1 -- 172.21.15.40:0/2776951709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218050 msgr2=0x55fa2a224f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5354868b80 1 --2- 172.21.15.40:0/2776951709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218050 0x55fa2a224f40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f53380117d0 tx=0x7f533800eb60 comp rx=0 tx=0).stop 2024-06-06T03:45:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5354868b80 1 -- 172.21.15.40:0/2776951709 shutdown_connections 2024-06-06T03:45:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5354868b80 1 --2- 172.21.15.40:0/2776951709 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55fa2a297c50 0x55fa2a29a070 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5354868b80 1 --2- 172.21.15.40:0/2776951709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a2127e0 0x55fa2a224a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5354868b80 1 --2- 172.21.15.40:0/2776951709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa2a218970 0x55fa2a2217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:43.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5322ffd700 0 client.15663 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:45:43.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5354868b80 1 --2- 172.21.15.40:0/2776951709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fa2a218050 0x55fa2a224f40 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:43.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5354868b80 1 -- 172.21.15.40:0/2776951709 >> 172.21.15.40:0/2776951709 conn(0x55fa2a138dc0 msgr2=0x55fa2a208830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:43.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5354868b80 1 -- 172.21.15.40:0/2776951709 shutdown_connections 2024-06-06T03:45:43.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5354868b80 1 -- 172.21.15.40:0/2776951709 wait complete. 2024-06-06T03:45:43.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.024+0000 7f5354868b80 20 client.15663 trim_cache size 0 max 16384 2024-06-06T03:45:43.026 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:45:43.026 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 37, 2024-06-06T03:45:43.026 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:45:43.026 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 20, 2024-06-06T03:45:43.026 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": { 2024-06-06T03:45:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "26b2a5b8": { 2024-06-06T03:45:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 20, 2024-06-06T03:45:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "age_ref": 0.0, 2024-06-06T03:45:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:45:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RELEASED", 2024-06-06T03:45:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:45:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:45:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "timeout": 90.0, 2024-06-06T03:45:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "expiration": 132.6, 2024-06-06T03:45:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "members": { 2024-06-06T03:45:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "file:/": { 2024-06-06T03:45:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "excluded": false, 2024-06-06T03:45:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:45:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RELEASED", 2024-06-06T03:45:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:45:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:45:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:45:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:45:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:45:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:45:43.028 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:45:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.025+0000 7f5349fe5700 1 -- 172.21.15.40:0/2806585369 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f532c0a2e10 msgr2=0x7f532c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.025+0000 7f5349fe5700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f532c0a2e10 0x7f532c0a52d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5344102010 tx=0x7f5338009260 comp rx=0 tx=0).stop 2024-06-06T03:45:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.025+0000 7f5349fe5700 1 -- 172.21.15.40:0/2806585369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5344102740 msgr2=0x7f534419e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:45:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.025+0000 7f5349fe5700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5344102740 0x7f534419e7a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f533400d980 tx=0x7f533400dc90 comp rx=0 tx=0).stop 2024-06-06T03:45:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.025+0000 7f5349fe5700 1 -- 172.21.15.40:0/2806585369 shutdown_connections 2024-06-06T03:45:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.025+0000 7f5349fe5700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f532c0a2e10 0x7f532c0a52d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.025+0000 7f5349fe5700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5344107840 0x7f5344198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.025+0000 7f5349fe5700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5344102740 0x7f534419e7a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.025+0000 7f5349fe5700 1 --2- 172.21.15.40:0/2806585369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53441030f0 0x7f534419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:45:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.025+0000 7f5349fe5700 1 -- 172.21.15.40:0/2806585369 >> 172.21.15.40:0/2806585369 conn(0x7f53440795f0 msgr2=0x7f53441005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:45:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.025+0000 7f5349fe5700 1 -- 172.21.15.40:0/2806585369 shutdown_connections 2024-06-06T03:45:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:45:43.025+0000 7f5349fe5700 1 -- 172.21.15.40:0/2806585369 wait complete. 2024-06-06T03:45:43.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:42 smithi040 ceph-mon[32220]: pgmap v217: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 380 op/s 2024-06-06T03:45:43.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:42 smithi104 ceph-mon[36569]: pgmap v217: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 380 op/s 2024-06-06T03:45:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:42 smithi131 ceph-mon[36687]: pgmap v217: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 380 op/s 2024-06-06T03:45:44.359 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '26b2a5b8', total seconds elapsed: 29.2 2024-06-06T03:45:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:44 smithi040 ceph-mon[32220]: pgmap v218: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 292 op/s 2024-06-06T03:45:45.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:44 smithi104 ceph-mon[36569]: pgmap v218: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 292 op/s 2024-06-06T03:45:45.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:44 smithi131 ceph-mon[36687]: pgmap v218: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 292 op/s 2024-06-06T03:45:46.119 INFO:tasks.workunit.client.0.smithi040.stderr:+ cd linux-6.5.11 2024-06-06T03:45:46.122 INFO:tasks.workunit.client.0.smithi040.stderr:+ make defconfig 2024-06-06T03:45:46.430 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/basic/fixdep 2024-06-06T03:45:46.650 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/kconfig/conf.o 2024-06-06T03:45:46.914 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/kconfig/confdata.o 2024-06-06T03:45:47.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:46 smithi040 ceph-mon[32220]: pgmap v219: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 301 op/s 2024-06-06T03:45:47.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:46 smithi104 ceph-mon[36569]: pgmap v219: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 301 op/s 2024-06-06T03:45:47.173 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/kconfig/expr.o 2024-06-06T03:45:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:46 smithi131 ceph-mon[36687]: pgmap v219: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 301 op/s 2024-06-06T03:45:47.390 INFO:tasks.workunit.client.0.smithi040.stdout: LEX scripts/kconfig/lexer.lex.c 2024-06-06T03:45:47.413 INFO:tasks.workunit.client.0.smithi040.stdout: YACC scripts/kconfig/parser.tab.[ch] 2024-06-06T03:45:47.495 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/kconfig/lexer.lex.o 2024-06-06T03:45:47.794 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/kconfig/menu.o 2024-06-06T03:45:47.976 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/kconfig/parser.tab.o 2024-06-06T03:45:48.205 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/kconfig/preprocess.o 2024-06-06T03:45:48.321 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/kconfig/symbol.o 2024-06-06T03:45:48.658 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/kconfig/util.o 2024-06-06T03:45:48.715 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTLD scripts/kconfig/conf 2024-06-06T03:45:48.741 INFO:tasks.workunit.client.0.smithi040.stdout:*** Default configuration is based on 'x86_64_defconfig' 2024-06-06T03:45:49.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:48 smithi040 ceph-mon[32220]: pgmap v220: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 465 KiB/s wr, 43 op/s 2024-06-06T03:45:49.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:48 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:45:49.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:48 smithi104 ceph-mon[36569]: pgmap v220: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 465 KiB/s wr, 43 op/s 2024-06-06T03:45:49.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:48 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:45:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:48 smithi131 ceph-mon[36687]: pgmap v220: 97 pgs: 97 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 465 KiB/s wr, 43 op/s 2024-06-06T03:45:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:48 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:45:50.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:49 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:45:50.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:49 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:45:50.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:49 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:45:50.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:49 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:45:50.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:49 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:45:50.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:49 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:45:50.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:49 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:45:50.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:49 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:45:50.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:49 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:45:51.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:51 smithi131 ceph-mon[36687]: pgmap v221: 97 pgs: 97 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 701 KiB/s wr, 49 op/s 2024-06-06T03:45:51.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:51 smithi040 ceph-mon[32220]: pgmap v221: 97 pgs: 97 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 701 KiB/s wr, 49 op/s 2024-06-06T03:45:51.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:51 smithi104 ceph-mon[36569]: pgmap v221: 97 pgs: 97 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 701 KiB/s wr, 49 op/s 2024-06-06T03:45:53.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:53 smithi131 ceph-mon[36687]: pgmap v222: 97 pgs: 97 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 665 KiB/s wr, 52 op/s 2024-06-06T03:45:53.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:53 smithi040 ceph-mon[32220]: pgmap v222: 97 pgs: 97 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 665 KiB/s wr, 52 op/s 2024-06-06T03:45:53.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:53 smithi104 ceph-mon[36569]: pgmap v222: 97 pgs: 97 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 665 KiB/s wr, 52 op/s 2024-06-06T03:45:55.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:55 smithi131 ceph-mon[36687]: pgmap v223: 97 pgs: 97 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 665 KiB/s wr, 51 op/s 2024-06-06T03:45:55.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:55 smithi040 ceph-mon[32220]: pgmap v223: 97 pgs: 97 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 665 KiB/s wr, 51 op/s 2024-06-06T03:45:55.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:55 smithi104 ceph-mon[36569]: pgmap v223: 97 pgs: 97 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 665 KiB/s wr, 51 op/s 2024-06-06T03:45:57.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:57 smithi131 ceph-mon[36687]: pgmap v224: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-06-06T03:45:57.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:57 smithi040 ceph-mon[32220]: pgmap v224: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-06-06T03:45:57.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:57 smithi104 ceph-mon[36569]: pgmap v224: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-06-06T03:45:57.781 INFO:tasks.workunit.client.0.smithi040.stdout:# 2024-06-06T03:45:57.781 INFO:tasks.workunit.client.0.smithi040.stdout:# configuration written to .config 2024-06-06T03:45:57.782 INFO:tasks.workunit.client.0.smithi040.stdout:# 2024-06-06T03:45:57.783 INFO:tasks.workunit.client.0.smithi040.stderr:++ grep -c processor /proc/cpuinfo 2024-06-06T03:45:57.785 INFO:tasks.workunit.client.0.smithi040.stderr:+ make -j8 2024-06-06T03:45:58.704 INFO:tasks.workunit.client.0.smithi040.stdout: GEN arch/x86/include/generated/asm/orc_hash.h 2024-06-06T03:45:58.710 INFO:tasks.workunit.client.0.smithi040.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_32.h 2024-06-06T03:45:58.711 INFO:tasks.workunit.client.0.smithi040.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_64.h 2024-06-06T03:45:58.718 INFO:tasks.workunit.client.0.smithi040.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_x32.h 2024-06-06T03:45:58.729 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC arch/x86/tools/relocs_32.o 2024-06-06T03:45:58.738 INFO:tasks.workunit.client.0.smithi040.stdout: UPD include/config/kernel.release 2024-06-06T03:45:58.742 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC arch/x86/tools/relocs_64.o 2024-06-06T03:45:58.743 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC arch/x86/tools/relocs_common.o 2024-06-06T03:45:58.745 INFO:tasks.workunit.client.0.smithi040.stdout: SYSTBL arch/x86/include/generated/asm/syscalls_32.h 2024-06-06T03:45:58.748 INFO:tasks.workunit.client.0.smithi040.stdout: SYSHDR arch/x86/include/generated/asm/unistd_32_ia32.h 2024-06-06T03:45:58.763 INFO:tasks.workunit.client.0.smithi040.stdout: UPD include/generated/uapi/linux/version.h 2024-06-06T03:45:58.769 INFO:tasks.workunit.client.0.smithi040.stdout: SYSHDR arch/x86/include/generated/asm/unistd_64_x32.h 2024-06-06T03:45:58.782 INFO:tasks.workunit.client.0.smithi040.stdout: SYSTBL arch/x86/include/generated/asm/syscalls_64.h 2024-06-06T03:45:58.793 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/uapi/asm/bpf_perf_event.h 2024-06-06T03:45:58.797 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/uapi/asm/errno.h 2024-06-06T03:45:58.798 INFO:tasks.workunit.client.0.smithi040.stdout: UPD include/generated/compile.h 2024-06-06T03:45:58.799 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/uapi/asm/fcntl.h 2024-06-06T03:45:58.800 INFO:tasks.workunit.client.0.smithi040.stdout: UPD include/generated/utsrelease.h 2024-06-06T03:45:58.802 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/uapi/asm/ioctl.h 2024-06-06T03:45:58.805 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/uapi/asm/ioctls.h 2024-06-06T03:45:58.809 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/uapi/asm/ipcbuf.h 2024-06-06T03:45:58.809 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/uapi/asm/param.h 2024-06-06T03:45:58.814 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/uapi/asm/poll.h 2024-06-06T03:45:58.818 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/uapi/asm/resource.h 2024-06-06T03:45:58.824 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/uapi/asm/socket.h 2024-06-06T03:45:58.828 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/uapi/asm/sockios.h 2024-06-06T03:45:58.831 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/uapi/asm/termbits.h 2024-06-06T03:45:58.836 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/selinux/genheaders/genheaders 2024-06-06T03:45:58.837 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/uapi/asm/termios.h 2024-06-06T03:45:58.838 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/selinux/mdp/mdp 2024-06-06T03:45:58.841 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/uapi/asm/types.h 2024-06-06T03:45:58.842 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/kallsyms 2024-06-06T03:45:58.845 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/sorttable 2024-06-06T03:45:58.856 INFO:tasks.workunit.client.0.smithi040.stdout: DESCEND objtool 2024-06-06T03:45:59.005 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/asn1_compiler 2024-06-06T03:45:59.006 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTLD arch/x86/tools/relocs 2024-06-06T03:45:59.024 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep.o 2024-06-06T03:45:59.079 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTLD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep-in.o 2024-06-06T03:45:59.111 INFO:tasks.workunit.client.0.smithi040.stdout: LINK /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep 2024-06-06T03:45:59.111 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/asm/early_ioremap.h 2024-06-06T03:45:59.111 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/asm/export.h 2024-06-06T03:45:59.111 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/asm/irq_regs.h 2024-06-06T03:45:59.112 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/asm/mcs_spinlock.h 2024-06-06T03:45:59.112 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/asm/kmap_size.h 2024-06-06T03:45:59.112 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/asm/local64.h 2024-06-06T03:45:59.112 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/asm/mmiowb.h 2024-06-06T03:45:59.112 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/asm/module.lds.h 2024-06-06T03:45:59.112 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/asm/rwonce.h 2024-06-06T03:45:59.113 INFO:tasks.workunit.client.0.smithi040.stdout: WRAP arch/x86/include/generated/asm/unaligned.h 2024-06-06T03:45:59.126 INFO:tasks.workunit.client.0.smithi040.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/exec-cmd.h 2024-06-06T03:45:59.126 INFO:tasks.workunit.client.0.smithi040.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/help.h 2024-06-06T03:45:59.127 INFO:tasks.workunit.client.0.smithi040.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/parse-options.h 2024-06-06T03:45:59.127 INFO:tasks.workunit.client.0.smithi040.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/pager.h 2024-06-06T03:45:59.142 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/exec-cmd.o 2024-06-06T03:45:59.147 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/help.o 2024-06-06T03:45:59.147 INFO:tasks.workunit.client.0.smithi040.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/run-command.h 2024-06-06T03:45:59.149 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/pager.o 2024-06-06T03:45:59.150 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/parse-options.o 2024-06-06T03:45:59.157 INFO:tasks.workunit.client.0.smithi040.stdout: INSTALL libsubcmd_headers 2024-06-06T03:45:59.159 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/run-command.o 2024-06-06T03:45:59.239 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/sigchain.o 2024-06-06T03:45:59.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:45:59 smithi131 ceph-mon[36687]: pgmap v225: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 376 op/s 2024-06-06T03:45:59.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:45:59 smithi040 ceph-mon[32220]: pgmap v225: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 376 op/s 2024-06-06T03:45:59.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:45:59 smithi104 ceph-mon[36569]: pgmap v225: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 376 op/s 2024-06-06T03:45:59.343 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/subcmd-config.o 2024-06-06T03:45:59.589 INFO:tasks.workunit.client.0.smithi040.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/libsubcmd-in.o 2024-06-06T03:45:59.605 INFO:tasks.workunit.client.0.smithi040.stdout: AR /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/libsubcmd.a 2024-06-06T03:46:00.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:00 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:46:00.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:00 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:46:00.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:00 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:46:00.759 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/weak.o 2024-06-06T03:46:00.759 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/check.o 2024-06-06T03:46:00.760 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/special.o 2024-06-06T03:46:00.760 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/builtin-check.o 2024-06-06T03:46:00.761 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/elf.o 2024-06-06T03:46:00.762 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool.o 2024-06-06T03:46:00.767 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/special.o 2024-06-06T03:46:00.893 INFO:tasks.workunit.client.0.smithi040.stdout: MKDIR /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/lib/ 2024-06-06T03:46:00.897 INFO:tasks.workunit.client.0.smithi040.stdout: GEN /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/lib/inat-tables.c 2024-06-06T03:46:00.902 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/orc_gen.o 2024-06-06T03:46:00.907 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/orc_dump.o 2024-06-06T03:46:00.914 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libstring.o 2024-06-06T03:46:00.937 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/decode.o 2024-06-06T03:46:00.974 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libctype.o 2024-06-06T03:46:01.013 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/str_error_r.o 2024-06-06T03:46:01.036 INFO:tasks.workunit.client.0.smithi040.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/librbtree.o 2024-06-06T03:46:01.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:01 smithi131 ceph-mon[36687]: pgmap v226: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 415 op/s 2024-06-06T03:46:01.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:01 smithi040 ceph-mon[32220]: pgmap v226: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 415 op/s 2024-06-06T03:46:01.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:01 smithi104 ceph-mon[36569]: pgmap v226: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 415 op/s 2024-06-06T03:46:01.442 INFO:tasks.workunit.client.0.smithi040.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/objtool-in.o 2024-06-06T03:46:02.761 INFO:tasks.workunit.client.0.smithi040.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool-in.o 2024-06-06T03:46:02.778 INFO:tasks.workunit.client.0.smithi040.stdout: LINK /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool 2024-06-06T03:46:03.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:03 smithi131 ceph-mon[36687]: pgmap v227: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 405 op/s 2024-06-06T03:46:03.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:03 smithi040 ceph-mon[32220]: pgmap v227: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 405 op/s 2024-06-06T03:46:03.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:03 smithi104 ceph-mon[36569]: pgmap v227: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 405 op/s 2024-06-06T03:46:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:05 smithi040 ceph-mon[32220]: pgmap v228: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-06-06T03:46:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:05 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:46:05.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:05 smithi104 ceph-mon[36569]: pgmap v228: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-06-06T03:46:05.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:05 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:46:05.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:05 smithi131 ceph-mon[36687]: pgmap v228: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-06-06T03:46:05.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:05 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:46:07.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:07 smithi040 ceph-mon[32220]: pgmap v229: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 425 op/s 2024-06-06T03:46:07.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:07 smithi040 ceph-mon[32220]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T03:46:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:07 smithi104 ceph-mon[36569]: pgmap v229: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 425 op/s 2024-06-06T03:46:07.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:07 smithi104 ceph-mon[36569]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T03:46:07.391 INFO:tasks.workunit.client.0.smithi040.stdout: CC scripts/mod/empty.o 2024-06-06T03:46:07.391 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/mod/mk_elfconfig 2024-06-06T03:46:07.392 INFO:tasks.workunit.client.0.smithi040.stdout: CC scripts/mod/devicetable-offsets.s 2024-06-06T03:46:07.452 INFO:tasks.workunit.client.0.smithi040.stdout: MKELF scripts/mod/elfconfig.h 2024-06-06T03:46:07.458 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/mod/modpost.o 2024-06-06T03:46:07.458 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/mod/sumversion.o 2024-06-06T03:46:07.486 INFO:tasks.workunit.client.0.smithi040.stdout: UPD scripts/mod/devicetable-offsets.h 2024-06-06T03:46:07.502 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC scripts/mod/file2alias.o 2024-06-06T03:46:07.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:07 smithi131 ceph-mon[36687]: pgmap v229: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 425 op/s 2024-06-06T03:46:07.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:07 smithi131 ceph-mon[36687]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T03:46:07.760 INFO:tasks.quiescer.fs.[cephfs]:Total duration (51.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-06T03:46:07.760 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.6 2024-06-06T03:46:07.760 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:46:07.937 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTLD scripts/mod/modpost 2024-06-06T03:46:07.975 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/bounds.s 2024-06-06T03:46:07.976 INFO:tasks.workunit.client.0.smithi040.stdout: CHKSHA1 include/linux/atomic/atomic-instrumented.h 2024-06-06T03:46:07.976 INFO:tasks.workunit.client.0.smithi040.stdout: CHKSHA1 include/linux/atomic/atomic-arch-fallback.h 2024-06-06T03:46:07.977 INFO:tasks.workunit.client.0.smithi040.stdout: CHKSHA1 include/linux/atomic/atomic-long.h 2024-06-06T03:46:07.989 INFO:tasks.workunit.client.0.smithi040.stdout: UPD include/generated/timeconst.h 2024-06-06T03:46:08.053 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:46:08.103 INFO:tasks.workunit.client.0.smithi040.stdout: UPD include/generated/bounds.h 2024-06-06T03:46:08.109 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/asm-offsets.s 2024-06-06T03:46:08.973 INFO:tasks.workunit.client.0.smithi040.stdout: UPD include/generated/asm-offsets.h 2024-06-06T03:46:08.977 INFO:tasks.workunit.client.0.smithi040.stdout: CALL scripts/checksyscalls.sh 2024-06-06T03:46:09.018 INFO:tasks.workunit.client.0.smithi040.stdout: LDS scripts/module.lds 2024-06-06T03:46:09.067 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC usr/gen_init_cpio 2024-06-06T03:46:09.069 INFO:tasks.workunit.client.0.smithi040.stdout: CC certs/system_keyring.o 2024-06-06T03:46:09.069 INFO:tasks.workunit.client.0.smithi040.stdout: CC init/main.o 2024-06-06T03:46:09.073 INFO:tasks.workunit.client.0.smithi040.stdout: CC ipc/compat.o 2024-06-06T03:46:09.073 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/filemap.o 2024-06-06T03:46:09.094 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/notify/dnotify/dnotify.o 2024-06-06T03:46:09.142 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/entry/vdso/vma.o 2024-06-06T03:46:09.194 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/sched/core.o 2024-06-06T03:46:09.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:09 smithi040 ceph-mon[32220]: pgmap v231: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-06-06T03:46:09.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:09 smithi104 ceph-mon[36569]: pgmap v231: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-06-06T03:46:09.432 INFO:tasks.workunit.client.0.smithi040.stdout: GEN usr/initramfs_data.cpio 2024-06-06T03:46:09.470 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC certs/extract-cert 2024-06-06T03:46:09.487 INFO:tasks.workunit.client.0.smithi040.stdout: COPY usr/initramfs_inc_data 2024-06-06T03:46:09.495 INFO:tasks.workunit.client.0.smithi040.stdout: AS usr/initramfs_data.o 2024-06-06T03:46:09.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:09 smithi131 ceph-mon[36687]: pgmap v231: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-06-06T03:46:09.564 INFO:tasks.workunit.client.0.smithi040.stdout: AR usr/built-in.a 2024-06-06T03:46:09.593 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/entry/vdso/extable.o 2024-06-06T03:46:09.716 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/notify/dnotify/built-in.a 2024-06-06T03:46:09.717 INFO:tasks.workunit.client.0.smithi040.stdout: CC ipc/util.o 2024-06-06T03:46:09.748 INFO:tasks.workunit.client.0.smithi040.stdout: CERT certs/x509_certificate_list 2024-06-06T03:46:09.753 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/notify/inotify/inotify_fsnotify.o 2024-06-06T03:46:09.755 INFO:tasks.workunit.client.0.smithi040.stdout: CERT certs/signing_key.x509 2024-06-06T03:46:09.763 INFO:tasks.workunit.client.0.smithi040.stdout: AS certs/system_certificates.o 2024-06-06T03:46:09.798 INFO:tasks.workunit.client.0.smithi040.stdout: AR certs/built-in.a 2024-06-06T03:46:09.840 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/entry/vdso/vdso32-setup.o 2024-06-06T03:46:09.902 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/mempool.o 2024-06-06T03:46:10.087 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/keys/gc.o 2024-06-06T03:46:10.235 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/notify/inotify/inotify_user.o 2024-06-06T03:46:10.237 INFO:tasks.workunit.client.0.smithi040.stdout: LDS arch/x86/entry/vdso/vdso.lds 2024-06-06T03:46:10.331 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/entry/vdso/vdso-note.o 2024-06-06T03:46:10.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.393+0000 7f67f9ea6700 1 -- 172.21.15.40:0/13851488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67f4110ff0 msgr2=0x7f67f41099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:10.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.393+0000 7f67f9ea6700 1 --2- 172.21.15.40:0/13851488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67f4110ff0 0x7f67f41099d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00b0a0 tx=0x7f67ec00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:46:10.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.394+0000 7f67f9ea6700 1 -- 172.21.15.40:0/13851488 shutdown_connections 2024-06-06T03:46:10.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.394+0000 7f67f9ea6700 1 --2- 172.21.15.40:0/13851488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67f411bc70 0x7f67f411e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:10.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.394+0000 7f67f9ea6700 1 --2- 172.21.15.40:0/13851488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67f4110b80 0x7f67f4109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:10.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.394+0000 7f67f9ea6700 1 --2- 172.21.15.40:0/13851488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67f4110ff0 0x7f67f41099d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:10.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.394+0000 7f67f9ea6700 1 -- 172.21.15.40:0/13851488 >> 172.21.15.40:0/13851488 conn(0x7f67f406d470 msgr2=0x7f67f406d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:10.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.396+0000 7f67f9ea6700 1 -- 172.21.15.40:0/13851488 shutdown_connections 2024-06-06T03:46:10.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.396+0000 7f67f9ea6700 1 -- 172.21.15.40:0/13851488 wait complete. 2024-06-06T03:46:10.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.397+0000 7f67f9ea6700 1 Processor -- start 2024-06-06T03:46:10.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.397+0000 7f67f9ea6700 1 -- start start 2024-06-06T03:46:10.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.398+0000 7f67f9ea6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67f4110b80 0x7f67f41ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:10.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.398+0000 7f67f9ea6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67f4110ff0 0x7f67f41add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:10.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.398+0000 7f67f9ea6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67f411bc70 0x7f67f41a7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:10.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.398+0000 7f67f9ea6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f67f41216c0 con 0x7f67f4110ff0 2024-06-06T03:46:10.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.398+0000 7f67f9ea6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f67f4121540 con 0x7f67f411bc70 2024-06-06T03:46:10.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.398+0000 7f67f9ea6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f67f4121840 con 0x7f67f4110b80 2024-06-06T03:46:10.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.398+0000 7f67f2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67f4110ff0 0x7f67f41add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:10.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.398+0000 7f67f2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67f4110ff0 0x7f67f41add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49608/0 (socket says 172.21.15.40:49608) 2024-06-06T03:46:10.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.398+0000 7f67f2ffd700 1 -- 172.21.15.40:0/542610583 learned_addr learned my addr 172.21.15.40:0/542610583 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:10.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.399+0000 7f67f2ffd700 1 -- 172.21.15.40:0/542610583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67f4110b80 msgr2=0x7f67f41ad7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T03:46:10.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.399+0000 7f67f2ffd700 1 --2- 172.21.15.40:0/542610583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67f4110b80 0x7f67f41ad7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:10.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.399+0000 7f67f2ffd700 1 -- 172.21.15.40:0/542610583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67f411bc70 msgr2=0x7f67f41a7980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T03:46:10.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.399+0000 7f67f2ffd700 1 --2- 172.21.15.40:0/542610583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67f411bc70 0x7f67f41a7980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:10.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.399+0000 7f67f2ffd700 1 -- 172.21.15.40:0/542610583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f41a80e0 con 0x7f67f4110ff0 2024-06-06T03:46:10.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.399+0000 7f67f2ffd700 1 --2- 172.21.15.40:0/542610583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67f4110ff0 0x7f67f41add00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009bd0 tx=0x7f67ec00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:10.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.404+0000 7f67f0ff9700 1 -- 172.21.15.40:0/542610583 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec021070 con 0x7f67f4110ff0 2024-06-06T03:46:10.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.404+0000 7f67f0ff9700 1 -- 172.21.15.40:0/542610583 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ec007db0 con 0x7f67f4110ff0 2024-06-06T03:46:10.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.404+0000 7f67f0ff9700 1 -- 172.21.15.40:0/542610583 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec01d6b0 con 0x7f67f4110ff0 2024-06-06T03:46:10.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.404+0000 7f67f9ea6700 1 -- 172.21.15.40:0/542610583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f41a8370 con 0x7f67f4110ff0 2024-06-06T03:46:10.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.404+0000 7f67f9ea6700 1 -- 172.21.15.40:0/542610583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f407a6f0 con 0x7f67f4110ff0 2024-06-06T03:46:10.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.407+0000 7f67f9ea6700 1 -- 172.21.15.40:0/542610583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f406e8b0 con 0x7f67f4110ff0 2024-06-06T03:46:10.410 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/entry/vdso/vclock_gettime.o 2024-06-06T03:46:10.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.413+0000 7f67f0ff9700 1 -- 172.21.15.40:0/542610583 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f67ec003a00 con 0x7f67f4110ff0 2024-06-06T03:46:10.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.415+0000 7f67f0ff9700 1 --2- 172.21.15.40:0/542610583 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f67dc0a2f90 0x7f67dc0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:10.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.415+0000 7f67f0ff9700 1 -- 172.21.15.40:0/542610583 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7f67ec00f020 con 0x7f67f4110ff0 2024-06-06T03:46:10.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.419+0000 7f67f0ff9700 1 -- 172.21.15.40:0/542610583 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f67ec02a020 con 0x7f67f4110ff0 2024-06-06T03:46:10.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.420+0000 7f67f37fe700 1 --2- 172.21.15.40:0/542610583 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f67dc0a2f90 0x7f67dc0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.426+0000 7f67f37fe700 1 --2- 172.21.15.40:0/542610583 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f67dc0a2f90 0x7f67dc0a5450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67f41a9170 tx=0x7f67e8006c60 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:10.536 INFO:tasks.workunit.client.0.smithi040.stdout: UPD init/utsversion-tmp.h 2024-06-06T03:46:10.577 INFO:tasks.workunit.client.0.smithi040.stdout: CC init/do_mounts.o 2024-06-06T03:46:10.642 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/oom_kill.o 2024-06-06T03:46:10.644 INFO:tasks.workunit.client.0.smithi040.stdout: CC ipc/msgutil.o 2024-06-06T03:46:10.794 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/keys/key.o 2024-06-06T03:46:10.968 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/entry/vdso/vgetcpu.o 2024-06-06T03:46:10.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.966+0000 7f67f9ea6700 1 -- 172.21.15.40:0/542610583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67f4069180 con 0x7f67f4110ff0 2024-06-06T03:46:10.989 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:46:10.989 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:46:03.426078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24443,24470,24425,24403,24398]},"id":1}]} 2024-06-06T03:46:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.989+0000 7f67f0ff9700 1 -- 172.21.15.40:0/542610583 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11947 (secure 0 0 0) 0x7f67f4069180 con 0x7f67f4110ff0 2024-06-06T03:46:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.994+0000 7f67da7fc700 1 -- 172.21.15.40:0/542610583 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f67dc0a2f90 msgr2=0x7f67dc0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.994+0000 7f67da7fc700 1 --2- 172.21.15.40:0/542610583 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f67dc0a2f90 0x7f67dc0a5450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67f41a9170 tx=0x7f67e8006c60 comp rx=0 tx=0).stop 2024-06-06T03:46:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.994+0000 7f67da7fc700 1 -- 172.21.15.40:0/542610583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67f4110ff0 msgr2=0x7f67f41add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.994+0000 7f67da7fc700 1 --2- 172.21.15.40:0/542610583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67f4110ff0 0x7f67f41add00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009bd0 tx=0x7f67ec00cf70 comp rx=0 tx=0).stop 2024-06-06T03:46:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.994+0000 7f67da7fc700 1 -- 172.21.15.40:0/542610583 shutdown_connections 2024-06-06T03:46:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.994+0000 7f67da7fc700 1 --2- 172.21.15.40:0/542610583 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f67dc0a2f90 0x7f67dc0a5450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.994+0000 7f67da7fc700 1 --2- 172.21.15.40:0/542610583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67f4110b80 0x7f67f41ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.994+0000 7f67da7fc700 1 --2- 172.21.15.40:0/542610583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67f411bc70 0x7f67f41a7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.994+0000 7f67da7fc700 1 --2- 172.21.15.40:0/542610583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67f4110ff0 0x7f67f41add00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.994+0000 7f67da7fc700 1 -- 172.21.15.40:0/542610583 >> 172.21.15.40:0/542610583 conn(0x7f67f406d470 msgr2=0x7f67f410e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:10.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:10.999+0000 7f67da7fc700 1 -- 172.21.15.40:0/542610583 shutdown_connections 2024-06-06T03:46:11.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:11.001+0000 7f67da7fc700 1 -- 172.21.15.40:0/542610583 wait complete. 2024-06-06T03:46:11.002 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 39 2024-06-06T03:46:11.138 INFO:tasks.workunit.client.0.smithi040.stdout: HOSTCC arch/x86/entry/vdso/vdso2c 2024-06-06T03:46:11.161 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/notify/inotify/built-in.a 2024-06-06T03:46:11.218 INFO:tasks.workunit.client.0.smithi040.stdout: CC ipc/msg.o 2024-06-06T03:46:11.265 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/notify/fanotify/built-in.a 2024-06-06T03:46:11.322 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/notify/fsnotify.o 2024-06-06T03:46:11.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:11 smithi040 ceph-mon[32220]: pgmap v232: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-06-06T03:46:11.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:11 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/542610583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:46:11.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:11 smithi104 ceph-mon[36569]: pgmap v232: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-06-06T03:46:11.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:11 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/542610583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:46:11.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:11 smithi131 ceph-mon[36687]: pgmap v232: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-06-06T03:46:11.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:11 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/542610583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:46:11.560 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/sched/fair.o 2024-06-06T03:46:11.742 INFO:tasks.workunit.client.0.smithi040.stdout: LDS arch/x86/entry/vdso/vdso32/vdso32.lds 2024-06-06T03:46:11.788 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/entry/vdso/vdso32/note.o 2024-06-06T03:46:11.798 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/keys/keyring.o 2024-06-06T03:46:11.859 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/entry/vdso/vdso32/system_call.o 2024-06-06T03:46:11.861 INFO:tasks.workunit.client.0.smithi040.stdout: CC init/do_mounts_initrd.o 2024-06-06T03:46:11.966 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/entry/vdso/vdso32/sigreturn.o 2024-06-06T03:46:12.062 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/entry/vdso/vdso32/vclock_gettime.o 2024-06-06T03:46:12.065 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/notify/notification.o 2024-06-06T03:46:12.115 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/fadvise.o 2024-06-06T03:46:12.180 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 145.6 --await-for 45.0' 2024-06-06T03:46:12.181 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 145.6 --await-for 45.0 2024-06-06T03:46:12.246 INFO:tasks.workunit.client.0.smithi040.stdout: CC ipc/sem.o 2024-06-06T03:46:12.607 INFO:tasks.workunit.client.0.smithi040.stdout: CC init/initramfs.o 2024-06-06T03:46:12.729 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/notify/group.o 2024-06-06T03:46:12.803 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/keys/keyctl.o 2024-06-06T03:46:12.867 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/entry/vdso/vdso32/vgetcpu.o 2024-06-06T03:46:12.899 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:46:12.904 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/maccess.o 2024-06-06T03:46:13.055 INFO:tasks.workunit.client.0.smithi040.stdout: VDSO arch/x86/entry/vdso/vdso64.so.dbg 2024-06-06T03:46:13.213 INFO:tasks.workunit.client.0.smithi040.stdout: VDSO arch/x86/entry/vdso/vdso32.so.dbg 2024-06-06T03:46:13.254 INFO:tasks.workunit.client.0.smithi040.stdout: OBJCOPY arch/x86/entry/vdso/vdso64.so 2024-06-06T03:46:13.262 INFO:tasks.workunit.client.0.smithi040.stdout: OBJCOPY arch/x86/entry/vdso/vdso32.so 2024-06-06T03:46:13.272 INFO:tasks.workunit.client.0.smithi040.stdout: VDSO2C arch/x86/entry/vdso/vdso-image-64.c 2024-06-06T03:46:13.280 INFO:tasks.workunit.client.0.smithi040.stdout: VDSO2C arch/x86/entry/vdso/vdso-image-32.c 2024-06-06T03:46:13.288 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/entry/vdso/vdso-image-64.o 2024-06-06T03:46:13.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:13 smithi040 ceph-mon[32220]: pgmap v233: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-06-06T03:46:13.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:13 smithi104 ceph-mon[36569]: pgmap v233: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-06-06T03:46:13.366 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/notify/mark.o 2024-06-06T03:46:13.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:13 smithi131 ceph-mon[36687]: pgmap v233: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-06-06T03:46:13.549 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/entry/vdso/vdso-image-32.o 2024-06-06T03:46:13.553 INFO:tasks.workunit.client.0.smithi040.stdout: CC init/calibrate.o 2024-06-06T03:46:13.596 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/page-writeback.o 2024-06-06T03:46:13.776 INFO:tasks.workunit.client.0.smithi040.stdout: AR arch/x86/entry/vdso/built-in.a 2024-06-06T03:46:13.787 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/folio-compat.o 2024-06-06T03:46:13.808 INFO:tasks.workunit.client.0.smithi040.stdout: CC ipc/shm.o 2024-06-06T03:46:13.844 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/keys/permission.o 2024-06-06T03:46:13.847 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/entry/vsyscall/vsyscall_64.o 2024-06-06T03:46:13.874 INFO:tasks.workunit.client.0.smithi040.stdout: CC init/init_task.o 2024-06-06T03:46:14.195 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/notify/fdinfo.o 2024-06-06T03:46:14.490 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/keys/process_keys.o 2024-06-06T03:46:14.658 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/readahead.o 2024-06-06T03:46:14.659 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/entry/vsyscall/vsyscall_emu_64.o 2024-06-06T03:46:14.684 INFO:tasks.workunit.client.0.smithi040.stdout: CC init/version.o 2024-06-06T03:46:14.709 INFO:tasks.workunit.client.0.smithi040.stdout: AR arch/x86/entry/vsyscall/built-in.a 2024-06-06T03:46:14.798 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/entry/entry.o 2024-06-06T03:46:14.834 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/entry/entry_64.o 2024-06-06T03:46:14.835 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/notify/built-in.a 2024-06-06T03:46:14.849 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/sched/build_policy.o 2024-06-06T03:46:14.876 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs_common/nfsacl.o 2024-06-06T03:46:14.918 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/entry/syscall_64.o 2024-06-06T03:46:14.948 INFO:tasks.workunit.client.0.smithi040.stdout: AR init/built-in.a 2024-06-06T03:46:14.951 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:14 smithi040 ceph-mon[32220]: pgmap v234: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-06-06T03:46:14.952 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:14 smithi040 ceph-mon[32220]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T03:46:15.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:14 smithi131 ceph-mon[36687]: pgmap v234: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-06-06T03:46:15.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:14 smithi131 ceph-mon[36687]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T03:46:15.034 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/entry/common.o 2024-06-06T03:46:15.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:14 smithi104 ceph-mon[36569]: pgmap v234: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-06-06T03:46:15.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:14 smithi104 ceph-mon[36569]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T03:46:15.106 INFO:tasks.workunit.client.0.smithi040.stdout: CC ipc/syscall.o 2024-06-06T03:46:15.249 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/entry/thunk_64.o 2024-06-06T03:46:15.388 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/iomap/trace.o 2024-06-06T03:46:15.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.392+0000 7f83c487b700 1 -- 172.21.15.40:0/784829884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0101dd0 msgr2=0x7f83c0102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:15.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.392+0000 7f83c487b700 1 --2- 172.21.15.40:0/784829884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0101dd0 0x7f83c0102250 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009920 tx=0x7f83b4009c30 comp rx=0 tx=0).stop 2024-06-06T03:46:15.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.393+0000 7f83c487b700 1 -- 172.21.15.40:0/784829884 shutdown_connections 2024-06-06T03:46:15.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.393+0000 7f83c487b700 1 --2- 172.21.15.40:0/784829884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83c0106250 0x7f83c0106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.393+0000 7f83c487b700 1 --2- 172.21.15.40:0/784829884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83c0102790 0x7f83c010e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.393+0000 7f83c487b700 1 --2- 172.21.15.40:0/784829884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0101dd0 0x7f83c0102250 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.393+0000 7f83c487b700 1 -- 172.21.15.40:0/784829884 >> 172.21.15.40:0/784829884 conn(0x7f83c00765a0 msgr2=0x7f83c00769b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:15.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.394+0000 7f83c487b700 1 -- 172.21.15.40:0/784829884 shutdown_connections 2024-06-06T03:46:15.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.394+0000 7f83c487b700 1 -- 172.21.15.40:0/784829884 wait complete. 2024-06-06T03:46:15.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.395+0000 7f83c487b700 1 Processor -- start 2024-06-06T03:46:15.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.395+0000 7f83c487b700 1 -- start start 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.395+0000 7f83c487b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83c0101dd0 0x7f83c010ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.395+0000 7f83c487b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102790 0x7f83c010cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.395+0000 7f83c487b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83c0106250 0x7f83c010d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.395+0000 7f83c487b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f83c0110030 con 0x7f83c0102790 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.395+0000 7f83c487b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f83c010feb0 con 0x7f83c0106250 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.395+0000 7f83c487b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f83c01101b0 con 0x7f83c0101dd0 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.396+0000 7f83bdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102790 0x7f83c010cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.396+0000 7f83bdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102790 0x7f83c010cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49622/0 (socket says 172.21.15.40:49622) 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.396+0000 7f83bdd9b700 1 -- 172.21.15.40:0/1743879842 learned_addr learned my addr 172.21.15.40:0/1743879842 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.397+0000 7f83bdd9b700 1 -- 172.21.15.40:0/1743879842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83c0101dd0 msgr2=0x7f83c010ca30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.397+0000 7f83bdd9b700 1 --2- 172.21.15.40:0/1743879842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83c0101dd0 0x7f83c010ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.397+0000 7f83bdd9b700 1 -- 172.21.15.40:0/1743879842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83c0106250 msgr2=0x7f83c010d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.397+0000 7f83bdd9b700 1 --2- 172.21.15.40:0/1743879842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83c0106250 0x7f83c010d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.397+0000 7f83bdd9b700 1 -- 172.21.15.40:0/1743879842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c006bf30 con 0x7f83c0102790 2024-06-06T03:46:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.397+0000 7f83bdd9b700 1 --2- 172.21.15.40:0/1743879842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102790 0x7f83c010cf70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f83b40052a0 tx=0x7f83b4005380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:15.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.398+0000 7f83af7fe700 1 -- 172.21.15.40:0/1743879842 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4027070 con 0x7f83c0102790 2024-06-06T03:46:15.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.398+0000 7f83c487b700 1 -- 172.21.15.40:0/1743879842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c006c160 con 0x7f83c0102790 2024-06-06T03:46:15.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.398+0000 7f83c487b700 1 -- 172.21.15.40:0/1743879842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c006c5f0 con 0x7f83c0102790 2024-06-06T03:46:15.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.399+0000 7f83af7fe700 1 -- 172.21.15.40:0/1743879842 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b4005550 con 0x7f83c0102790 2024-06-06T03:46:15.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.399+0000 7f83af7fe700 1 -- 172.21.15.40:0/1743879842 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4004ce0 con 0x7f83c0102790 2024-06-06T03:46:15.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.407+0000 7f83af7fe700 1 -- 172.21.15.40:0/1743879842 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f83b402a070 con 0x7f83c0102790 2024-06-06T03:46:15.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.407+0000 7f83cf0feb80 1 Processor -- start 2024-06-06T03:46:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.408+0000 7f83cf0feb80 1 -- start start 2024-06-06T03:46:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.409+0000 7f83cf0feb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a36780ee70 0x55a36780f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.410+0000 7f83bed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a36780ee70 0x55a36780f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.410+0000 7f83bed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a36780ee70 0x55a36780f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49628/0 (socket says 172.21.15.40:49628) 2024-06-06T03:46:15.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.410+0000 7f83bed9d700 1 -- 172.21.15.40:0/1881872091 learned_addr learned my addr 172.21.15.40:0/1881872091 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:15.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.410+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/1881872091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a367809080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.410+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/1881872091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3678095c0 0x55a3678099e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.410+0000 7f83cf0feb80 1 -- 172.21.15.40:0/1881872091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a36775a9a0 con 0x55a36780ee70 2024-06-06T03:46:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.410+0000 7f83be59c700 1 --2- 172.21.15.40:0/1881872091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a367809080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.411+0000 7f83cf0feb80 1 -- 172.21.15.40:0/1881872091 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a3677028b0 con 0x55a36780f790 2024-06-06T03:46:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.411+0000 7f83cf0feb80 1 -- 172.21.15.40:0/1881872091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a367756960 con 0x55a3678095c0 2024-06-06T03:46:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.411+0000 7f83bed9d700 1 -- 172.21.15.40:0/1881872091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3678095c0 msgr2=0x55a3678099e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T03:46:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.411+0000 7f83bed9d700 1 --2- 172.21.15.40:0/1881872091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3678095c0 0x55a3678099e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.411+0000 7f83bed9d700 1 -- 172.21.15.40:0/1881872091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 msgr2=0x55a367809080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.411+0000 7f83bed9d700 1 --2- 172.21.15.40:0/1881872091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a367809080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.411+0000 7f83bed9d700 1 -- 172.21.15.40:0/1881872091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3676f90e0 con 0x55a36780ee70 2024-06-06T03:46:15.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.412+0000 7f83be59c700 1 --2- 172.21.15.40:0/1881872091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a367809080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:46:15.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.412+0000 7f83bed9d700 1 --2- 172.21.15.40:0/1881872091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a36780ee70 0x55a36780f250 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f83b800e6c0 tx=0x7f83b8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0eef6c30d5c73c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:15.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.412+0000 7f83af7fe700 1 --2- 172.21.15.40:0/1743879842 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f83a80a3050 0x7f83a80a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.413+0000 7f83acff9700 1 -- 172.21.15.40:0/1881872091 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8004bb0 con 0x55a36780ee70 2024-06-06T03:46:15.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.413+0000 7f83be59c700 1 --2- 172.21.15.40:0/1743879842 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f83a80a3050 0x7f83a80a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.413+0000 7f83af7fe700 1 -- 172.21.15.40:0/1743879842 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8426+0+0 (secure 0 0 0) 0x7f83b40d0f20 con 0x7f83c0102790 2024-06-06T03:46:15.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.414+0000 7f83be59c700 1 --2- 172.21.15.40:0/1743879842 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f83a80a3050 0x7f83a80a5510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f83c0077ba0 tx=0x7f83b0009310 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:15.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.414+0000 7f83acff9700 1 -- 172.21.15.40:0/1881872091 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b8004d50 con 0x55a36780ee70 2024-06-06T03:46:15.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.414+0000 7f83acff9700 1 -- 172.21.15.40:0/1881872091 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8004580 con 0x55a36780ee70 2024-06-06T03:46:15.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.416+0000 7f83cf0feb80 1 -- 172.21.15.40:0/1881872091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a36780ee70 msgr2=0x55a36780f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:15.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.416+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/1881872091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a36780ee70 0x55a36780f250 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f83b800e6c0 tx=0x7f83b8009920 comp rx=0 tx=0).stop 2024-06-06T03:46:15.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.417+0000 7f83cf0feb80 1 -- 172.21.15.40:0/1881872091 shutdown_connections 2024-06-06T03:46:15.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.417+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/1881872091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3678095c0 0x55a3678099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.417+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/1881872091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a36780ee70 0x55a36780f250 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.417+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/1881872091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a367809080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.417+0000 7f83cf0feb80 1 -- 172.21.15.40:0/1881872091 >> 172.21.15.40:0/1881872091 conn(0x55a367733e50 msgr2=0x55a367804370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:15.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.417+0000 7f83cf0feb80 1 -- 172.21.15.40:0/1881872091 shutdown_connections 2024-06-06T03:46:15.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.417+0000 7f83cf0feb80 1 -- 172.21.15.40:0/1881872091 wait complete. 2024-06-06T03:46:15.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.418+0000 7f83cf0feb80 1 -- start start 2024-06-06T03:46:15.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.418+0000 7f83cf0feb80 1 Processor -- start 2024-06-06T03:46:15.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.419+0000 7f83cf0feb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a3678155d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.419+0000 7f83be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a3678155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.419+0000 7f83be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a3678155d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49630/0 (socket says 172.21.15.40:49630) 2024-06-06T03:46:15.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.419+0000 7f83be59c700 1 -- 172.21.15.40:0/2445307945 learned_addr learned my addr 172.21.15.40:0/2445307945 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:15.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.420+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 0x55a3678128d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.420+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a367812e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.420+0000 7f83bdd9b700 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 0x55a3678128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.420+0000 7f83bed9d700 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a367812e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.420+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2445307945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a3678161a0 con 0x55a3678095c0 2024-06-06T03:46:15.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.420+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2445307945 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a3676fad80 con 0x55a36780f790 2024-06-06T03:46:15.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.420+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2445307945 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a367816320 con 0x55a36780ee70 2024-06-06T03:46:15.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.421+0000 7f83cf0feb80 10 client.0 _ll_register_callbacks cb 0x55a3676f4ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:46:15.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.422+0000 7f83be59c700 1 -- 172.21.15.40:0/2445307945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 msgr2=0x55a3678128d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:15.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.422+0000 7f83be59c700 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 0x55a3678128d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.422+0000 7f83be59c700 1 -- 172.21.15.40:0/2445307945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 msgr2=0x55a367812e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:15.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.422+0000 7f83be59c700 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a367812e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.422+0000 7f83be59c700 1 -- 172.21.15.40:0/2445307945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3676a9a60 con 0x55a3678095c0 2024-06-06T03:46:15.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.423+0000 7f83be59c700 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a3678155d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f83b0010da0 tx=0x7f83b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306ce09f366affee server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:15.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.423+0000 7f83a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:46:15.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.423+0000 7f83a5ffb700 1 -- 172.21.15.40:0/2445307945 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0023070 con 0x55a3678095c0 2024-06-06T03:46:15.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.423+0000 7f83a5ffb700 1 -- 172.21.15.40:0/2445307945 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8426+0+0 (secure 0 0 0) 0x7f83b0012df0 con 0x55a3678095c0 2024-06-06T03:46:15.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.424+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2445307945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b8004d50 con 0x55a3678095c0 2024-06-06T03:46:15.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.424+0000 7f83a5ffb700 1 -- 172.21.15.40:0/2445307945 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f83b00098a0 con 0x55a3678095c0 2024-06-06T03:46:15.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.424+0000 7f83cf0feb80 10 client.15681 fetch_fsmap learned FSMap version 39 2024-06-06T03:46:15.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.424+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2445307945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a3678209b0 con 0x55a3678095c0 2024-06-06T03:46:15.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.425+0000 7f83a5ffb700 1 -- 172.21.15.40:0/2445307945 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b0004d10 con 0x55a3678095c0 2024-06-06T03:46:15.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.425+0000 7f83a5ffb700 1 -- 172.21.15.40:0/2445307945 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 39) ==== 8386+0+0 (secure 0 0 0) 0x7f83b000f040 con 0x55a3678095c0 2024-06-06T03:46:15.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.426+0000 7f83cf0feb80 10 client.15681 fetch_fsmap finished waiting for FSMap version 39 2024-06-06T03:46:15.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.427+0000 7f83cf0feb80 10 client.15681 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:46:15.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.427+0000 7f83cf0feb80 20 client.15681 populate_metadata read hostname 'smithi040' 2024-06-06T03:46:15.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.427+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a367822f40 0x55a367825330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.427+0000 7f83cf0feb80 4 client.15681 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:46:15.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.427+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2445307945 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f83b8004d50 con 0x55a367822f40 2024-06-06T03:46:15.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.429+0000 7f83bdd9b700 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a367822f40 0x55a367825330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.431+0000 7f83bdd9b700 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a367822f40 0x55a367825330 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c5ee16ba0c88082 server_cookie=cd17b85cec3644bf in_seq=0 out_seq=0 2024-06-06T03:46:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.431+0000 7f83a5ffb700 10 client.15681 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:46:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.432+0000 7f83a5ffb700 1 -- 172.21.15.40:0/2445307945 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f83b8004d50 con 0x55a367822f40 2024-06-06T03:46:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.432+0000 7f83a5ffb700 10 client.15681 handle_command_reply: tid=0 2024-06-06T03:46:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.432+0000 7f83cf0feb80 1 client.15681 shutdown 2024-06-06T03:46:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.433+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2445307945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 msgr2=0x55a3678155d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.433+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a3678155d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f83b0010da0 tx=0x7f83b0004750 comp rx=0 tx=0).stop 2024-06-06T03:46:15.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.433+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2445307945 shutdown_connections 2024-06-06T03:46:15.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.433+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a367822f40 0x55a367825330 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.433+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a367812e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.433+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a3678155d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.433+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/2445307945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 0x55a3678128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.433+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2445307945 >> 172.21.15.40:0/2445307945 conn(0x55a367733e50 msgr2=0x55a367808070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:15.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.434+0000 7f83a5ffb700 0 client.15681 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:46:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.437+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2445307945 shutdown_connections 2024-06-06T03:46:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.437+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2445307945 wait complete. 2024-06-06T03:46:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.438+0000 7f83cf0feb80 20 client.15681 trim_cache size 0 max 16384 2024-06-06T03:46:15.465 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/keys/request_key.o 2024-06-06T03:46:15.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.484+0000 7f83cf0feb80 1 Processor -- start 2024-06-06T03:46:15.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.488+0000 7f83cf0feb80 1 -- start start 2024-06-06T03:46:15.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.488+0000 7f83cf0feb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a367809080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.488+0000 7f83cf0feb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 0x55a367804520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.488+0000 7f83cf0feb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a367810400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.488+0000 7f83cf0feb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f83a000d570 con 0x55a3678095c0 2024-06-06T03:46:15.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.488+0000 7f83cf0feb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f83a000f170 con 0x55a36780f790 2024-06-06T03:46:15.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.488+0000 7f83cf0feb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f83a000dc40 con 0x55a36780ee70 2024-06-06T03:46:15.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.488+0000 7f83be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a367809080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.488+0000 7f83bdd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 0x55a367804520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.488+0000 7f83be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a367809080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49642/0 (socket says 172.21.15.40:49642) 2024-06-06T03:46:15.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.488+0000 7f83be59c700 1 -- 172.21.15.40:0/2965639750 learned_addr learned my addr 172.21.15.40:0/2965639750 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:15.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.489+0000 7f83bed9d700 1 --2- 172.21.15.40:0/2965639750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a367810400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.489+0000 7f83bed9d700 1 -- 172.21.15.40:0/2965639750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 msgr2=0x55a367804520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:15.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.489+0000 7f83bed9d700 1 --2- 172.21.15.40:0/2965639750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 0x55a367804520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.489+0000 7f83bed9d700 1 -- 172.21.15.40:0/2965639750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 msgr2=0x55a367809080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:15.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.489+0000 7f83bed9d700 1 --2- 172.21.15.40:0/2965639750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a367809080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.489+0000 7f83bed9d700 1 -- 172.21.15.40:0/2965639750 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a367810e40 con 0x55a36780f790 2024-06-06T03:46:15.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.489+0000 7f83bed9d700 1 --2- 172.21.15.40:0/2965639750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a367810400 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009550 tx=0x7f83b80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10c0690bcd5fd44e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:15.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.490+0000 7f83a7fff700 1 -- 172.21.15.40:0/2965639750 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8009320 con 0x55a36780f790 2024-06-06T03:46:15.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.490+0000 7f83a7fff700 1 -- 172.21.15.40:0/2965639750 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b8003c10 con 0x55a36780f790 2024-06-06T03:46:15.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.490+0000 7f83a7fff700 1 -- 172.21.15.40:0/2965639750 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8007630 con 0x55a36780f790 2024-06-06T03:46:15.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.491+0000 7f83be59c700 1 --2- 172.21.15.40:0/2965639750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a367809080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:46:15.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.492+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2965639750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 msgr2=0x55a367810400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:15.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.493+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/2965639750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a367810400 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009550 tx=0x7f83b80040c0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.493+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2965639750 shutdown_connections 2024-06-06T03:46:15.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.493+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/2965639750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a367810400 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.493+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/2965639750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a367809080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.493+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/2965639750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 0x55a367804520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.493+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2965639750 >> 172.21.15.40:0/2965639750 conn(0x55a367733e50 msgr2=0x55a3677fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.493+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2965639750 shutdown_connections 2024-06-06T03:46:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.493+0000 7f83cf0feb80 1 -- 172.21.15.40:0/2965639750 wait complete. 2024-06-06T03:46:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.494+0000 7f83cf0feb80 1 -- start start 2024-06-06T03:46:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.494+0000 7f83cf0feb80 1 Processor -- start 2024-06-06T03:46:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.495+0000 7f83cf0feb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a36781b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.495+0000 7f83be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a36781b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.495+0000 7f83be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a36781b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49648/0 (socket says 172.21.15.40:49648) 2024-06-06T03:46:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.495+0000 7f83be59c700 1 -- 172.21.15.40:0/648405090 learned_addr learned my addr 172.21.15.40:0/648405090 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.495+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 0x55a36781be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.496+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a3678186e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.496+0000 7f83cf0feb80 1 -- 172.21.15.40:0/648405090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f83a000e9e0 con 0x55a3678095c0 2024-06-06T03:46:15.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.496+0000 7f83cf0feb80 1 -- 172.21.15.40:0/648405090 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f83a000e310 con 0x55a36780f790 2024-06-06T03:46:15.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.496+0000 7f83cf0feb80 1 -- 172.21.15.40:0/648405090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f83a000b530 con 0x55a36780ee70 2024-06-06T03:46:15.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.496+0000 7f83cf0feb80 10 client.0 _ll_register_callbacks cb 0x55a3676f4ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:46:15.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.496+0000 7f83bdd9b700 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 0x55a36781be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.496+0000 7f83bed9d700 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a3678186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.497+0000 7f83be59c700 1 -- 172.21.15.40:0/648405090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 msgr2=0x55a36781be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.497+0000 7f83be59c700 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 0x55a36781be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.497+0000 7f83be59c700 1 -- 172.21.15.40:0/648405090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 msgr2=0x55a3678186e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.497+0000 7f83be59c700 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a3678186e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.497+0000 7f83be59c700 1 -- 172.21.15.40:0/648405090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a367818f20 con 0x55a3678095c0 2024-06-06T03:46:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.497+0000 7f83bed9d700 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a3678186e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:46:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.497+0000 7f83be59c700 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a36781b950 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f83b0012010 tx=0x7f83b0012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab22601e0f25f9d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.497+0000 7f83a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:46:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.498+0000 7f83a5ffb700 1 -- 172.21.15.40:0/648405090 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0023040 con 0x55a3678095c0 2024-06-06T03:46:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.498+0000 7f83a5ffb700 1 -- 172.21.15.40:0/648405090 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8426+0+0 (secure 0 0 0) 0x7f83b0005450 con 0x55a3678095c0 2024-06-06T03:46:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.498+0000 7f83cf0feb80 1 -- 172.21.15.40:0/648405090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b8003c10 con 0x55a3678095c0 2024-06-06T03:46:15.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.498+0000 7f83a5ffb700 1 -- 172.21.15.40:0/648405090 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f83b0005690 con 0x55a3678095c0 2024-06-06T03:46:15.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.499+0000 7f83cf0feb80 10 client.15687 fetch_fsmap learned FSMap version 39 2024-06-06T03:46:15.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.499+0000 7f83cf0feb80 1 -- 172.21.15.40:0/648405090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a36788e7a0 con 0x55a3678095c0 2024-06-06T03:46:15.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.499+0000 7f83a5ffb700 1 -- 172.21.15.40:0/648405090 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b8003c10 con 0x55a3678095c0 2024-06-06T03:46:15.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.499+0000 7f83a5ffb700 1 -- 172.21.15.40:0/648405090 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 39) ==== 8386+0+0 (secure 0 0 0) 0x7f83b00289e0 con 0x55a3678095c0 2024-06-06T03:46:15.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.500+0000 7f83cf0feb80 10 client.15687 fetch_fsmap finished waiting for FSMap version 39 2024-06-06T03:46:15.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.500+0000 7f83cf0feb80 10 client.15687 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:46:15.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.500+0000 7f83cf0feb80 20 client.15687 populate_metadata read hostname 'smithi040' 2024-06-06T03:46:15.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.500+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a36788f010 0x55a367891430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:15.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.500+0000 7f83cf0feb80 4 client.15687 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.6, "await_for": 45.0}] 2024-06-06T03:46:15.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.500+0000 7f83cf0feb80 1 -- 172.21.15.40:0/648405090 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.6, "await_for": 45.0}) -- 0x55a367767ea0 con 0x55a36788f010 2024-06-06T03:46:15.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.500+0000 7f83bdd9b700 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a36788f010 0x55a367891430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:15.503 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/swap.o 2024-06-06T03:46:15.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.506+0000 7f83bdd9b700 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a36788f010 0x55a367891430 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2061aaa95690745d server_cookie=72033a369bba7472 in_seq=0 out_seq=0 2024-06-06T03:46:15.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:15.506+0000 7f83a5ffb700 10 client.15687 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:46:17.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:16 smithi131 ceph-mon[36687]: pgmap v236: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 232 op/s 2024-06-06T03:46:17.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:16 smithi040 ceph-mon[32220]: pgmap v236: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 232 op/s 2024-06-06T03:46:17.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:16 smithi104 ceph-mon[36569]: pgmap v236: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 232 op/s 2024-06-06T03:46:19.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:18 smithi131 ceph-mon[36687]: pgmap v237: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 248 op/s 2024-06-06T03:46:19.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:18 smithi040 ceph-mon[32220]: pgmap v237: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 248 op/s 2024-06-06T03:46:19.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:18 smithi104 ceph-mon[36569]: pgmap v237: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 248 op/s 2024-06-06T03:46:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:20 smithi040 ceph-mon[32220]: pgmap v238: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-06-06T03:46:21.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:20 smithi104 ceph-mon[36569]: pgmap v238: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-06-06T03:46:21.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:20 smithi131 ceph-mon[36687]: pgmap v238: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-06-06T03:46:22.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:22 smithi040 ceph-mon[32220]: pgmap v239: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-06-06T03:46:23.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:22 smithi104 ceph-mon[36569]: pgmap v239: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-06-06T03:46:23.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:22 smithi131 ceph-mon[36687]: pgmap v239: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-06-06T03:46:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:24 smithi040 ceph-mon[32220]: pgmap v240: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-06-06T03:46:25.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:24 smithi104 ceph-mon[36569]: pgmap v240: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-06-06T03:46:25.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:24 smithi131 ceph-mon[36687]: pgmap v240: 97 pgs: 97 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-06-06T03:46:27.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:26 smithi040 ceph-mon[32220]: pgmap v241: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-06-06T03:46:27.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:26 smithi104 ceph-mon[36569]: pgmap v241: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-06-06T03:46:27.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:26 smithi131 ceph-mon[36687]: pgmap v241: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-06-06T03:46:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:28 smithi040 ceph-mon[32220]: pgmap v242: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 1.4 MiB/s wr, 223 op/s 2024-06-06T03:46:29.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:28 smithi104 ceph-mon[36569]: pgmap v242: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 1.4 MiB/s wr, 223 op/s 2024-06-06T03:46:29.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:28 smithi131 ceph-mon[36687]: pgmap v242: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 1.4 MiB/s wr, 223 op/s 2024-06-06T03:46:31.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:30 smithi040 ceph-mon[32220]: pgmap v243: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 1.3 MiB/s wr, 214 op/s 2024-06-06T03:46:31.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:30 smithi104 ceph-mon[36569]: pgmap v243: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 1.3 MiB/s wr, 214 op/s 2024-06-06T03:46:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:30 smithi131 ceph-mon[36687]: pgmap v243: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 1.3 MiB/s wr, 214 op/s 2024-06-06T03:46:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:32 smithi040 ceph-mon[32220]: pgmap v244: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 529 KiB/s wr, 162 op/s 2024-06-06T03:46:33.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:32 smithi104 ceph-mon[36569]: pgmap v244: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 529 KiB/s wr, 162 op/s 2024-06-06T03:46:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:32 smithi131 ceph-mon[36687]: pgmap v244: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 529 KiB/s wr, 162 op/s 2024-06-06T03:46:35.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:34 smithi040 ceph-mon[32220]: pgmap v245: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 528 KiB/s wr, 160 op/s 2024-06-06T03:46:35.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:34 smithi131 ceph-mon[36687]: pgmap v245: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 528 KiB/s wr, 160 op/s 2024-06-06T03:46:35.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:34 smithi104 ceph-mon[36569]: pgmap v245: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 528 KiB/s wr, 160 op/s 2024-06-06T03:46:37.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:36 smithi040 ceph-mon[32220]: pgmap v246: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 575 KiB/s wr, 172 op/s 2024-06-06T03:46:37.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:36 smithi104 ceph-mon[36569]: pgmap v246: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 575 KiB/s wr, 172 op/s 2024-06-06T03:46:37.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:36 smithi131 ceph-mon[36687]: pgmap v246: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 575 KiB/s wr, 172 op/s 2024-06-06T03:46:38.541 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:46:38.541 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 39, 2024-06-06T03:46:38.541 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:46:38.541 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 22, 2024-06-06T03:46:38.541 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": { 2024-06-06T03:46:38.541 INFO:teuthology.orchestra.run.smithi040.stdout: "61b36b60": { 2024-06-06T03:46:38.542 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 22, 2024-06-06T03:46:38.542 INFO:teuthology.orchestra.run.smithi040.stdout: "age_ref": 0.0, 2024-06-06T03:46:38.542 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:46:38.542 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "QUIESCED", 2024-06-06T03:46:38.542 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:46:38.542 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:46:38.542 INFO:teuthology.orchestra.run.smithi040.stdout: "timeout": 90.0, 2024-06-06T03:46:38.542 INFO:teuthology.orchestra.run.smithi040.stdout: "expiration": 145.6, 2024-06-06T03:46:38.542 INFO:teuthology.orchestra.run.smithi040.stdout: "members": { 2024-06-06T03:46:38.542 INFO:teuthology.orchestra.run.smithi040.stdout: "file:/": { 2024-06-06T03:46:38.542 INFO:teuthology.orchestra.run.smithi040.stdout: "excluded": false, 2024-06-06T03:46:38.542 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:46:38.542 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "QUIESCED", 2024-06-06T03:46:38.543 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:46:38.543 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:46:38.543 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:46:38.543 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:46:38.543 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:46:38.543 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:46:38.543 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:46:38.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.539+0000 7f83a5ffb700 1 -- 172.21.15.40:0/648405090 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a367767ea0 con 0x55a36788f010 2024-06-06T03:46:38.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.539+0000 7f83a5ffb700 10 client.15687 handle_command_reply: tid=0 2024-06-06T03:46:38.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.539+0000 7f83cf0feb80 1 client.15687 shutdown 2024-06-06T03:46:38.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.539+0000 7f83cf0feb80 1 -- 172.21.15.40:0/648405090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 msgr2=0x55a36781b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:38.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.539+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a36781b950 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f83b0012010 tx=0x7f83b0012d70 comp rx=0 tx=0).stop 2024-06-06T03:46:38.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.539+0000 7f83cf0feb80 1 -- 172.21.15.40:0/648405090 shutdown_connections 2024-06-06T03:46:38.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.539+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55a36788f010 0x55a367891430 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:38.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.539+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a36780f790 0x55a3678186e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:38.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.539+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a3678095c0 0x55a36781b950 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:38.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.539+0000 7f83cf0feb80 1 --2- 172.21.15.40:0/648405090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a36780ee70 0x55a36781be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:38.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.539+0000 7f83a5ffb700 0 client.15687 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:46:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.539+0000 7f83cf0feb80 1 -- 172.21.15.40:0/648405090 >> 172.21.15.40:0/648405090 conn(0x55a367733e50 msgr2=0x55a367810400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.540+0000 7f83cf0feb80 1 -- 172.21.15.40:0/648405090 shutdown_connections 2024-06-06T03:46:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.540+0000 7f83cf0feb80 1 -- 172.21.15.40:0/648405090 wait complete. 2024-06-06T03:46:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.540+0000 7f83cf0feb80 20 client.15687 trim_cache size 0 max 16384 2024-06-06T03:46:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.541+0000 7f83c487b700 1 -- 172.21.15.40:0/1743879842 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f83a80a3050 msgr2=0x7f83a80a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.541+0000 7f83c487b700 1 --2- 172.21.15.40:0/1743879842 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f83a80a3050 0x7f83a80a5510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f83c0077ba0 tx=0x7f83b0009310 comp rx=0 tx=0).stop 2024-06-06T03:46:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.541+0000 7f83c487b700 1 -- 172.21.15.40:0/1743879842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102790 msgr2=0x7f83c010cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.541+0000 7f83c487b700 1 --2- 172.21.15.40:0/1743879842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102790 0x7f83c010cf70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f83b40052a0 tx=0x7f83b4005380 comp rx=0 tx=0).stop 2024-06-06T03:46:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.541+0000 7f83c487b700 1 -- 172.21.15.40:0/1743879842 shutdown_connections 2024-06-06T03:46:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.541+0000 7f83c487b700 1 --2- 172.21.15.40:0/1743879842 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f83a80a3050 0x7f83a80a5510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.541+0000 7f83c487b700 1 --2- 172.21.15.40:0/1743879842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83c0101dd0 0x7f83c010ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.541+0000 7f83c487b700 1 --2- 172.21.15.40:0/1743879842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83c0106250 0x7f83c010d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.541+0000 7f83c487b700 1 --2- 172.21.15.40:0/1743879842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102790 0x7f83c010cf70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:38.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.541+0000 7f83c487b700 1 -- 172.21.15.40:0/1743879842 >> 172.21.15.40:0/1743879842 conn(0x7f83c00765a0 msgr2=0x7f83c0111ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.541+0000 7f83c487b700 1 -- 172.21.15.40:0/1743879842 shutdown_connections 2024-06-06T03:46:38.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:38.541+0000 7f83c487b700 1 -- 172.21.15.40:0/1743879842 wait complete. 2024-06-06T03:46:39.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:38 smithi040 ceph-mon[32220]: pgmap v247: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 26 op/s 2024-06-06T03:46:39.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:38 smithi104 ceph-mon[36569]: pgmap v247: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 26 op/s 2024-06-06T03:46:39.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:38 smithi131 ceph-mon[36687]: pgmap v247: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 26 op/s 2024-06-06T03:46:39.518 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '61b36b60', quiesce took 31.8 seconds. Will release after: -6.199999999999999 2024-06-06T03:46:39.519 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:46:39.774 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:46:41.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:40 smithi040 ceph-mon[32220]: pgmap v248: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 187 KiB/s wr, 45 op/s 2024-06-06T03:46:41.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:40 smithi104 ceph-mon[36569]: pgmap v248: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 187 KiB/s wr, 45 op/s 2024-06-06T03:46:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:40 smithi131 ceph-mon[36687]: pgmap v248: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 187 KiB/s wr, 45 op/s 2024-06-06T03:46:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.805+0000 7f01435ab700 1 -- 172.21.15.40:0/3274041274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013c106720 msgr2=0x7f013c10e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.805+0000 7f01435ab700 1 --2- 172.21.15.40:0/3274041274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013c106720 0x7f013c10e2c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f013400b0d0 tx=0x7f013400b3e0 comp rx=0 tx=0).stop 2024-06-06T03:46:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.805+0000 7f01435ab700 1 -- 172.21.15.40:0/3274041274 shutdown_connections 2024-06-06T03:46:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.805+0000 7f01435ab700 1 --2- 172.21.15.40:0/3274041274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013c106720 0x7f013c10e2c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.805+0000 7f01435ab700 1 --2- 172.21.15.40:0/3274041274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f013c1019d0 0x7f013c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.805+0000 7f01435ab700 1 --2- 172.21.15.40:0/3274041274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013c102380 0x7f013c105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.805+0000 7f01435ab700 1 -- 172.21.15.40:0/3274041274 >> 172.21.15.40:0/3274041274 conn(0x7f013c078340 msgr2=0x7f013c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.805+0000 7f01435ab700 1 -- 172.21.15.40:0/3274041274 shutdown_connections 2024-06-06T03:46:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.805+0000 7f01435ab700 1 -- 172.21.15.40:0/3274041274 wait complete. 2024-06-06T03:46:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.805+0000 7f01435ab700 1 Processor -- start 2024-06-06T03:46:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f01435ab700 1 -- start start 2024-06-06T03:46:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f01435ab700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013c1019d0 0x7f013c10eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f01435ab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013c102380 0x7f013c10f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f01435ab700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f013c106720 0x7f013c10f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f01435ab700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f013c112830 con 0x7f013c102380 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f01435ab700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1126b0 con 0x7f013c106720 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f01435ab700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1129b0 con 0x7f013c1019d0 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f0140b46700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013c102380 0x7f013c10f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f0140b46700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013c102380 0x7f013c10f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36572/0 (socket says 172.21.15.40:36572) 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f0140b46700 1 -- 172.21.15.40:0/926359619 learned_addr learned my addr 172.21.15.40:0/926359619 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f0141b48700 1 --2- 172.21.15.40:0/926359619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f013c106720 0x7f013c10f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f0141347700 1 --2- 172.21.15.40:0/926359619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013c1019d0 0x7f013c10eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f0140b46700 1 -- 172.21.15.40:0/926359619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013c1019d0 msgr2=0x7f013c10eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f0140b46700 1 --2- 172.21.15.40:0/926359619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013c1019d0 0x7f013c10eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f0140b46700 1 -- 172.21.15.40:0/926359619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f013c106720 msgr2=0x7f013c10f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f0140b46700 1 --2- 172.21.15.40:0/926359619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f013c106720 0x7f013c10f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f0140b46700 1 -- 172.21.15.40:0/926359619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c1a70f0 con 0x7f013c102380 2024-06-06T03:46:41.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.806+0000 7f0140b46700 1 --2- 172.21.15.40:0/926359619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013c102380 0x7f013c10f1e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f012c00c9b0 tx=0x7f012c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:41.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.807+0000 7f01327fc700 1 -- 172.21.15.40:0/926359619 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c007aa0 con 0x7f013c102380 2024-06-06T03:46:41.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.807+0000 7f01435ab700 1 -- 172.21.15.40:0/926359619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013c1a73e0 con 0x7f013c102380 2024-06-06T03:46:41.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.807+0000 7f01327fc700 1 -- 172.21.15.40:0/926359619 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f012c00ce50 con 0x7f013c102380 2024-06-06T03:46:41.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.807+0000 7f01435ab700 1 -- 172.21.15.40:0/926359619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013c1a7970 con 0x7f013c102380 2024-06-06T03:46:41.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.807+0000 7f01327fc700 1 -- 172.21.15.40:0/926359619 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c018660 con 0x7f013c102380 2024-06-06T03:46:41.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.808+0000 7f01435ab700 1 -- 172.21.15.40:0/926359619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f013c102ad0 con 0x7f013c102380 2024-06-06T03:46:41.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.809+0000 7f01327fc700 1 -- 172.21.15.40:0/926359619 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f012c00f450 con 0x7f013c102380 2024-06-06T03:46:41.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.810+0000 7f01327fc700 1 --2- 172.21.15.40:0/926359619 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f01280a2e10 0x7f01280a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:41.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.810+0000 7f01327fc700 1 -- 172.21.15.40:0/926359619 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8426+0+0 (secure 0 0 0) 0x7f012c0cf2b0 con 0x7f013c102380 2024-06-06T03:46:41.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.810+0000 7f0141347700 1 --2- 172.21.15.40:0/926359619 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f01280a2e10 0x7f01280a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:41.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.810+0000 7f0141347700 1 --2- 172.21.15.40:0/926359619 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f01280a2e10 0x7f01280a52d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0138005950 tx=0x7f013800a300 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:41.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:41.813+0000 7f01327fc700 1 -- 172.21.15.40:0/926359619 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f012c014030 con 0x7f013c102380 2024-06-06T03:46:42.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:42.068+0000 7f01435ab700 1 -- 172.21.15.40:0/926359619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f013c02dca0 con 0x7f013c102380 2024-06-06T03:46:42.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:42.069+0000 7f01327fc700 1 -- 172.21.15.40:0/926359619 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11947 (secure 0 0 0) 0x7f012c09bfb0 con 0x7f013c102380 2024-06-06T03:46:42.070 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:46:42.070 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:46:03.426078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24443,24470,24425,24403,24398]},"id":1}]} 2024-06-06T03:46:42.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:42.071+0000 7f01435ab700 1 -- 172.21.15.40:0/926359619 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f01280a2e10 msgr2=0x7f01280a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:42.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:42.071+0000 7f01435ab700 1 --2- 172.21.15.40:0/926359619 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f01280a2e10 0x7f01280a52d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0138005950 tx=0x7f013800a300 comp rx=0 tx=0).stop 2024-06-06T03:46:42.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:42.071+0000 7f01435ab700 1 -- 172.21.15.40:0/926359619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013c102380 msgr2=0x7f013c10f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:42.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:42.071+0000 7f01435ab700 1 --2- 172.21.15.40:0/926359619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013c102380 0x7f013c10f1e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f012c00c9b0 tx=0x7f012c00ccc0 comp rx=0 tx=0).stop 2024-06-06T03:46:42.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:42.071+0000 7f01435ab700 1 -- 172.21.15.40:0/926359619 shutdown_connections 2024-06-06T03:46:42.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:42.071+0000 7f01435ab700 1 --2- 172.21.15.40:0/926359619 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f01280a2e10 0x7f01280a52d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:42.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:42.071+0000 7f01435ab700 1 --2- 172.21.15.40:0/926359619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013c1019d0 0x7f013c10eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:42.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:42.071+0000 7f01435ab700 1 --2- 172.21.15.40:0/926359619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f013c106720 0x7f013c10f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:42.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:42.071+0000 7f01435ab700 1 --2- 172.21.15.40:0/926359619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013c102380 0x7f013c10f1e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:42.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:42.071+0000 7f01435ab700 1 -- 172.21.15.40:0/926359619 >> 172.21.15.40:0/926359619 conn(0x7f013c078340 msgr2=0x7f013c0ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:42.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:42.072+0000 7f01435ab700 1 -- 172.21.15.40:0/926359619 shutdown_connections 2024-06-06T03:46:42.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:42.072+0000 7f01435ab700 1 -- 172.21.15.40:0/926359619 wait complete. 2024-06-06T03:46:42.073 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 39 2024-06-06T03:46:42.988 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 61b36b60 --release --await' 2024-06-06T03:46:42.989 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 61b36b60 --release --await 2024-06-06T03:46:43.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:42 smithi040 ceph-mon[32220]: pgmap v249: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 164 KiB/s wr, 38 op/s 2024-06-06T03:46:43.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:42 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/926359619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:46:43.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:42 smithi104 ceph-mon[36569]: pgmap v249: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 164 KiB/s wr, 38 op/s 2024-06-06T03:46:43.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:42 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/926359619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:46:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:46:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:42 smithi131 ceph-mon[36687]: pgmap v249: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 164 KiB/s wr, 38 op/s 2024-06-06T03:46:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:42 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/926359619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:46:44.959 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:44 smithi040 ceph-mon[32220]: pgmap v250: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 163 KiB/s wr, 36 op/s 2024-06-06T03:46:45.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:44 smithi104 ceph-mon[36569]: pgmap v250: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 163 KiB/s wr, 36 op/s 2024-06-06T03:46:45.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.208+0000 7f591d773700 1 -- 172.21.15.40:0/320328068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f591806a170 msgr2=0x7f591806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:45.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.208+0000 7f591d773700 1 --2- 172.21.15.40:0/320328068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f591806a170 0x7f591806a5f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5900009990 tx=0x7f5900009ca0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.208+0000 7f591d773700 1 -- 172.21.15.40:0/320328068 shutdown_connections 2024-06-06T03:46:45.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.208+0000 7f591d773700 1 --2- 172.21.15.40:0/320328068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5918108500 0x7f59181088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.208+0000 7f591d773700 1 --2- 172.21.15.40:0/320328068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f591806ab30 0x7f591810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.208+0000 7f591d773700 1 --2- 172.21.15.40:0/320328068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f591806a170 0x7f591806a5f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.208+0000 7f591d773700 1 -- 172.21.15.40:0/320328068 >> 172.21.15.40:0/320328068 conn(0x7f5918078340 msgr2=0x7f5918078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:45.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.208+0000 7f591d773700 1 -- 172.21.15.40:0/320328068 shutdown_connections 2024-06-06T03:46:45.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.208+0000 7f591d773700 1 -- 172.21.15.40:0/320328068 wait complete. 2024-06-06T03:46:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.209+0000 7f591d773700 1 Processor -- start 2024-06-06T03:46:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.209+0000 7f591d773700 1 -- start start 2024-06-06T03:46:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.209+0000 7f591d773700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f591806a170 0x7f591819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.209+0000 7f591d773700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f591806ab30 0x7f591819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.209+0000 7f591d773700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5918108500 0x7f5918198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f591d773700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59181127b0 con 0x7f5918108500 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f591d773700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5918112630 con 0x7f591806ab30 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f591d773700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5918112930 con 0x7f591806a170 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f59177fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5918108500 0x7f5918198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f59177fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5918108500 0x7f5918198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36588/0 (socket says 172.21.15.40:36588) 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f59177fe700 1 -- 172.21.15.40:0/2555723093 learned_addr learned my addr 172.21.15.40:0/2555723093 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f590e7fc700 1 --2- 172.21.15.40:0/2555723093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f591806ab30 0x7f591819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f5916ffd700 1 --2- 172.21.15.40:0/2555723093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f591806a170 0x7f591819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f59177fe700 1 -- 172.21.15.40:0/2555723093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f591806a170 msgr2=0x7f591819e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f59177fe700 1 --2- 172.21.15.40:0/2555723093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f591806a170 0x7f591819e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f59177fe700 1 -- 172.21.15.40:0/2555723093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f591806ab30 msgr2=0x7f591819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f59177fe700 1 --2- 172.21.15.40:0/2555723093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f591806ab30 0x7f591819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f59177fe700 1 -- 172.21.15.40:0/2555723093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5918199210 con 0x7f5918108500 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f59177fe700 1 --2- 172.21.15.40:0/2555723093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5918108500 0x7f5918198ab0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5910007ee0 tx=0x7f591000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f5914ff9700 1 -- 172.21.15.40:0/2555723093 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910004bb0 con 0x7f5918108500 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f5914ff9700 1 -- 172.21.15.40:0/2555723093 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5910004d50 con 0x7f5918108500 2024-06-06T03:46:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f591d773700 1 -- 172.21.15.40:0/2555723093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5918199500 con 0x7f5918108500 2024-06-06T03:46:45.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.210+0000 7f5914ff9700 1 -- 172.21.15.40:0/2555723093 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910005960 con 0x7f5918108500 2024-06-06T03:46:45.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.211+0000 7f591d773700 1 -- 172.21.15.40:0/2555723093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59181a57c0 con 0x7f5918108500 2024-06-06T03:46:45.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.213+0000 7f5914ff9700 1 -- 172.21.15.40:0/2555723093 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f591001a020 con 0x7f5918108500 2024-06-06T03:46:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.214+0000 7f5914ff9700 1 --2- 172.21.15.40:0/2555723093 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f59040a2d90 0x7f59040a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.214+0000 7f5916ffd700 1 --2- 172.21.15.40:0/2555723093 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f59040a2d90 0x7f59040a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.214+0000 7f5914ff9700 1 -- 172.21.15.40:0/2555723093 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8426+0+0 (secure 0 0 0) 0x7f59100ceae0 con 0x7f5918108500 2024-06-06T03:46:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.215+0000 7f5916ffd700 1 --2- 172.21.15.40:0/2555723093 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f59040a2d90 0x7f59040a5250 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5918199bf0 tx=0x7f59080073d0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.215+0000 7f5927ff6b80 1 Processor -- start 2024-06-06T03:46:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.215+0000 7f5927ff6b80 1 -- start start 2024-06-06T03:46:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.215+0000 7f5927ff6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408e500 0x563d1408e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.215+0000 7f5927ff6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408ee20 0x563d14088710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.215+0000 7f5927ff6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d14089070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.215+0000 7f5927ff6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563d13f7b060 con 0x563d1408ee20 2024-06-06T03:46:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.215+0000 7f5927ff6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563d13fd9950 con 0x563d14088c50 2024-06-06T03:46:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.215+0000 7f5927ff6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563d13f41560 con 0x563d1408e500 2024-06-06T03:46:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.215+0000 7f590e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408ee20 0x563d14088710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.215+0000 7f590e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408ee20 0x563d14088710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36596/0 (socket says 172.21.15.40:36596) 2024-06-06T03:46:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.215+0000 7f590e7fc700 1 -- 172.21.15.40:0/1524460388 learned_addr learned my addr 172.21.15.40:0/1524460388 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.215+0000 7f59177fe700 1 --2- 172.21.15.40:0/1524460388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d14089070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.216+0000 7f5916ffd700 1 --2- 172.21.15.40:0/1524460388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408e500 0x563d1408e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.216+0000 7f590e7fc700 1 -- 172.21.15.40:0/1524460388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408e500 msgr2=0x563d1408e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.216+0000 7f590e7fc700 1 --2- 172.21.15.40:0/1524460388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408e500 0x563d1408e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.216+0000 7f590e7fc700 1 -- 172.21.15.40:0/1524460388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 msgr2=0x563d14089070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.216+0000 7f590e7fc700 1 --2- 172.21.15.40:0/1524460388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d14089070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.216+0000 7f590e7fc700 1 -- 172.21.15.40:0/1524460388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d13f289d0 con 0x563d1408ee20 2024-06-06T03:46:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.216+0000 7f59177fe700 1 --2- 172.21.15.40:0/1524460388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d14089070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:46:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.216+0000 7f590e7fc700 1 --2- 172.21.15.40:0/1524460388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408ee20 0x563d14088710 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5900015000 tx=0x7f5900009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd998e7ce24ff6ce server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.216+0000 7f590d7fa700 1 -- 172.21.15.40:0/1524460388 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900028070 con 0x563d1408ee20 2024-06-06T03:46:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.216+0000 7f590d7fa700 1 -- 172.21.15.40:0/1524460388 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5900003780 con 0x563d1408ee20 2024-06-06T03:46:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.216+0000 7f590d7fa700 1 -- 172.21.15.40:0/1524460388 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590000bb50 con 0x563d1408ee20 2024-06-06T03:46:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.217+0000 7f5927ff6b80 1 -- 172.21.15.40:0/1524460388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408ee20 msgr2=0x563d14088710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.217+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/1524460388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408ee20 0x563d14088710 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5900015000 tx=0x7f5900009f90 comp rx=0 tx=0).stop 2024-06-06T03:46:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.217+0000 7f5927ff6b80 1 -- 172.21.15.40:0/1524460388 shutdown_connections 2024-06-06T03:46:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.217+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/1524460388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408e500 0x563d1408e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.217+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/1524460388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d14089070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.217+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/1524460388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408ee20 0x563d14088710 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.217+0000 7f5927ff6b80 1 -- 172.21.15.40:0/1524460388 >> 172.21.15.40:0/1524460388 conn(0x563d13fb5180 msgr2=0x563d14083a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.217+0000 7f5927ff6b80 1 -- 172.21.15.40:0/1524460388 shutdown_connections 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.217+0000 7f5927ff6b80 1 -- 172.21.15.40:0/1524460388 wait complete. 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.217+0000 7f5927ff6b80 1 -- start start 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.217+0000 7f5927ff6b80 1 Processor -- start 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f5927ff6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d14094b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f5927ff6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408e500 0x563d14091e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f5927ff6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408ee20 0x563d14092360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f5927ff6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563d13f7a0c0 con 0x563d1408ee20 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f5927ff6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563d13fd5910 con 0x563d14088c50 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f5927ff6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563d140957a0 con 0x563d1408e500 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f5927ff6b80 10 client.0 _ll_register_callbacks cb 0x563d13f6f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f59177fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408ee20 0x563d14092360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f59177fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408ee20 0x563d14092360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36610/0 (socket says 172.21.15.40:36610) 2024-06-06T03:46:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f59177fe700 1 -- 172.21.15.40:0/294408672 learned_addr learned my addr 172.21.15.40:0/294408672 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f5916ffd700 1 --2- 172.21.15.40:0/294408672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d14094b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f590e7fc700 1 --2- 172.21.15.40:0/294408672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408e500 0x563d14091e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f59177fe700 1 -- 172.21.15.40:0/294408672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408e500 msgr2=0x563d14091e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f59177fe700 1 --2- 172.21.15.40:0/294408672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408e500 0x563d14091e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f59177fe700 1 -- 172.21.15.40:0/294408672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 msgr2=0x563d14094b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f59177fe700 1 --2- 172.21.15.40:0/294408672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d14094b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f59177fe700 1 -- 172.21.15.40:0/294408672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d13f9f850 con 0x563d1408ee20 2024-06-06T03:46:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f59177fe700 1 --2- 172.21.15.40:0/294408672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408ee20 0x563d14092360 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f59100cc340 tx=0x7f591009aa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be675cc6ee5d7705 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f58fe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:46:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f58fe7fc700 1 -- 172.21.15.40:0/294408672 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591009b060 con 0x563d1408ee20 2024-06-06T03:46:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f58fe7fc700 1 -- 172.21.15.40:0/294408672 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8426+0+0 (secure 0 0 0) 0x7f59100976d0 con 0x563d1408ee20 2024-06-06T03:46:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.218+0000 7f5927ff6b80 1 -- 172.21.15.40:0/294408672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5900003780 con 0x563d1408ee20 2024-06-06T03:46:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.219+0000 7f58fe7fc700 1 -- 172.21.15.40:0/294408672 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f591009f410 con 0x563d1408ee20 2024-06-06T03:46:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.219+0000 7f5927ff6b80 10 client.15705 fetch_fsmap learned FSMap version 39 2024-06-06T03:46:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.219+0000 7f5927ff6b80 1 -- 172.21.15.40:0/294408672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563d1409fee0 con 0x563d1408ee20 2024-06-06T03:46:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.219+0000 7f58fe7fc700 1 -- 172.21.15.40:0/294408672 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5900003780 con 0x563d1408ee20 2024-06-06T03:46:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.219+0000 7f58fe7fc700 1 -- 172.21.15.40:0/294408672 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 39) ==== 8386+0+0 (secure 0 0 0) 0x7f591009b350 con 0x563d1408ee20 2024-06-06T03:46:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.219+0000 7f5927ff6b80 10 client.15705 fetch_fsmap finished waiting for FSMap version 39 2024-06-06T03:46:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.219+0000 7f5927ff6b80 10 client.15705 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:46:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.219+0000 7f5927ff6b80 20 client.15705 populate_metadata read hostname 'smithi040' 2024-06-06T03:46:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.219+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/294408672 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563d140a2570 0x563d140a4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.219+0000 7f5927ff6b80 4 client.15705 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:46:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.219+0000 7f5927ff6b80 1 -- 172.21.15.40:0/294408672 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d13fe6e50 con 0x563d140a2570 2024-06-06T03:46:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.219+0000 7f5916ffd700 1 --2- 172.21.15.40:0/294408672 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563d140a2570 0x563d140a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.220+0000 7f5916ffd700 1 --2- 172.21.15.40:0/294408672 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563d140a2570 0x563d140a4960 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9c839a7099d0b server_cookie=a4bfac8a34b53412 in_seq=0 out_seq=0 2024-06-06T03:46:45.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.220+0000 7f58fe7fc700 10 client.15705 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:46:45.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f58fe7fc700 1 -- 172.21.15.40:0/294408672 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563d13fe6e50 con 0x563d140a2570 2024-06-06T03:46:45.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f58fe7fc700 10 client.15705 handle_command_reply: tid=0 2024-06-06T03:46:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f5927ff6b80 1 client.15705 shutdown 2024-06-06T03:46:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f5927ff6b80 1 -- 172.21.15.40:0/294408672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408ee20 msgr2=0x563d14092360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/294408672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408ee20 0x563d14092360 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f59100cc340 tx=0x7f591009aa40 comp rx=0 tx=0).stop 2024-06-06T03:46:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f5927ff6b80 1 -- 172.21.15.40:0/294408672 shutdown_connections 2024-06-06T03:46:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/294408672 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563d140a2570 0x563d140a4960 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/294408672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408ee20 0x563d14092360 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/294408672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d14094b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/294408672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408e500 0x563d14091e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f5927ff6b80 1 -- 172.21.15.40:0/294408672 >> 172.21.15.40:0/294408672 conn(0x563d13fb5180 msgr2=0x563d14082bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f58fe7fc700 0 client.15705 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:46:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f5927ff6b80 1 -- 172.21.15.40:0/294408672 shutdown_connections 2024-06-06T03:46:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f5927ff6b80 1 -- 172.21.15.40:0/294408672 wait complete. 2024-06-06T03:46:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.221+0000 7f5927ff6b80 20 client.15705 trim_cache size 0 max 16384 2024-06-06T03:46:45.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.241+0000 7f5927ff6b80 1 Processor -- start 2024-06-06T03:46:45.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.241+0000 7f5927ff6b80 1 -- start start 2024-06-06T03:46:45.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f5927ff6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d14094660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f5927ff6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408e500 0x563d14083330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f5927ff6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408ee20 0x563d14083870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f5927ff6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59181acd40 con 0x563d1408e500 2024-06-06T03:46:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f5927ff6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f59181ae940 con 0x563d14088c50 2024-06-06T03:46:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f5927ff6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59181ad410 con 0x563d1408ee20 2024-06-06T03:46:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f590e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408e500 0x563d14083330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f590e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408e500 0x563d14083330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36618/0 (socket says 172.21.15.40:36618) 2024-06-06T03:46:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f590e7fc700 1 -- 172.21.15.40:0/684742815 learned_addr learned my addr 172.21.15.40:0/684742815 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f590e7fc700 1 -- 172.21.15.40:0/684742815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408ee20 msgr2=0x563d14083870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:45.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f59177fe700 1 --2- 172.21.15.40:0/684742815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408ee20 0x563d14083870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f5916ffd700 1 --2- 172.21.15.40:0/684742815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d14094660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f590e7fc700 1 --2- 172.21.15.40:0/684742815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408ee20 0x563d14083870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f590e7fc700 1 -- 172.21.15.40:0/684742815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 msgr2=0x563d14094660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f590e7fc700 1 --2- 172.21.15.40:0/684742815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d14094660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f590e7fc700 1 -- 172.21.15.40:0/684742815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d140a4f40 con 0x563d1408e500 2024-06-06T03:46:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f59177fe700 1 --2- 172.21.15.40:0/684742815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408ee20 0x563d14083870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:46:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f590e7fc700 1 --2- 172.21.15.40:0/684742815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408e500 0x563d14083330 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5900009990 tx=0x7f590001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886907962d404ad3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f590cff9700 1 -- 172.21.15.40:0/684742815 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900032070 con 0x563d1408e500 2024-06-06T03:46:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.242+0000 7f590cff9700 1 -- 172.21.15.40:0/684742815 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5900029040 con 0x563d1408e500 2024-06-06T03:46:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.243+0000 7f590cff9700 1 -- 172.21.15.40:0/684742815 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900028040 con 0x563d1408e500 2024-06-06T03:46:45.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.243+0000 7f5927ff6b80 1 -- 172.21.15.40:0/684742815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408e500 msgr2=0x563d14083330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:45.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.243+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/684742815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408e500 0x563d14083330 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5900009990 tx=0x7f590001a040 comp rx=0 tx=0).stop 2024-06-06T03:46:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.243+0000 7f5927ff6b80 1 -- 172.21.15.40:0/684742815 shutdown_connections 2024-06-06T03:46:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.243+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/684742815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408ee20 0x563d14083870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.243+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/684742815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d14094660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.243+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/684742815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408e500 0x563d14083330 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.243+0000 7f5927ff6b80 1 -- 172.21.15.40:0/684742815 >> 172.21.15.40:0/684742815 conn(0x563d13fb5180 msgr2=0x563d1408d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.243+0000 7f5927ff6b80 1 -- 172.21.15.40:0/684742815 shutdown_connections 2024-06-06T03:46:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.243+0000 7f5927ff6b80 1 -- 172.21.15.40:0/684742815 wait complete. 2024-06-06T03:46:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f5927ff6b80 1 -- start start 2024-06-06T03:46:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f5927ff6b80 1 Processor -- start 2024-06-06T03:46:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f5927ff6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d1409b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f5927ff6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408e500 0x563d14097a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f5927ff6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408ee20 0x563d14097fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f5927ff6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59181ae1b0 con 0x563d1408e500 2024-06-06T03:46:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f5927ff6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f59181adae0 con 0x563d14088c50 2024-06-06T03:46:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f5927ff6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59181aad00 con 0x563d1408ee20 2024-06-06T03:46:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f5927ff6b80 10 client.0 _ll_register_callbacks cb 0x563d13f6f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:46:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f590e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408e500 0x563d14097a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f590e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408e500 0x563d14097a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36630/0 (socket says 172.21.15.40:36630) 2024-06-06T03:46:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f590e7fc700 1 -- 172.21.15.40:0/2323562422 learned_addr learned my addr 172.21.15.40:0/2323562422 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f59177fe700 1 --2- 172.21.15.40:0/2323562422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408ee20 0x563d14097fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f5916ffd700 1 --2- 172.21.15.40:0/2323562422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d1409b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f590e7fc700 1 -- 172.21.15.40:0/2323562422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408ee20 msgr2=0x563d14097fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f590e7fc700 1 --2- 172.21.15.40:0/2323562422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408ee20 0x563d14097fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f590e7fc700 1 -- 172.21.15.40:0/2323562422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 msgr2=0x563d1409b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f590e7fc700 1 --2- 172.21.15.40:0/2323562422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d1409b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f590e7fc700 1 -- 172.21.15.40:0/2323562422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d140987e0 con 0x563d1408e500 2024-06-06T03:46:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.244+0000 7f5916ffd700 1 --2- 172.21.15.40:0/2323562422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d1409b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f590e7fc700 1 --2- 172.21.15.40:0/2323562422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408e500 0x563d14097a60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5900009350 tx=0x7f5900005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8364177232f06f45 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f58fe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f58fe7fc700 1 -- 172.21.15.40:0/2323562422 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900028070 con 0x563d1408e500 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f58fe7fc700 1 -- 172.21.15.40:0/2323562422 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8426+0+0 (secure 0 0 0) 0x7f5900005600 con 0x563d1408e500 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f5927ff6b80 1 -- 172.21.15.40:0/2323562422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5900029040 con 0x563d1408e500 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f58fe7fc700 1 -- 172.21.15.40:0/2323562422 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f590003e050 con 0x563d1408e500 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f5927ff6b80 10 client.15711 fetch_fsmap learned FSMap version 39 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f5927ff6b80 1 -- 172.21.15.40:0/2323562422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563d1410e170 con 0x563d1408e500 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f58fe7fc700 1 -- 172.21.15.40:0/2323562422 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5900029040 con 0x563d1408e500 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f58fe7fc700 1 -- 172.21.15.40:0/2323562422 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 39) ==== 8386+0+0 (secure 0 0 0) 0x7f59000238e0 con 0x563d1408e500 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f5927ff6b80 10 client.15711 fetch_fsmap finished waiting for FSMap version 39 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f5927ff6b80 10 client.15711 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f5927ff6b80 20 client.15711 populate_metadata read hostname 'smithi040' 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/2323562422 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563d1410e3a0 0x563d14110790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f5927ff6b80 4 client.15711 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "61b36b60", "release": true, "await": true}] 2024-06-06T03:46:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.245+0000 7f5927ff6b80 1 -- 172.21.15.40:0/2323562422 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "set_id": "61b36b60", "release": true, "await": true}) -- 0x563d13fe13d0 con 0x563d1410e3a0 2024-06-06T03:46:45.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.246+0000 7f5916ffd700 1 --2- 172.21.15.40:0/2323562422 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563d1410e3a0 0x563d14110790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:45.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.246+0000 7f58fe7fc700 10 client.15711 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:46:45.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:45.246+0000 7f5916ffd700 1 --2- 172.21.15.40:0/2323562422 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563d1410e3a0 0x563d14110790 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49aafe594b9662e3 server_cookie=8df67f39ac5d9048 in_seq=0 out_seq=0 2024-06-06T03:46:45.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:44 smithi131 ceph-mon[36687]: pgmap v250: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 163 KiB/s wr, 36 op/s 2024-06-06T03:46:47.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:46 smithi040 ceph-mon[32220]: pgmap v251: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 178 KiB/s wr, 43 op/s 2024-06-06T03:46:47.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:46 smithi104 ceph-mon[36569]: pgmap v251: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 178 KiB/s wr, 43 op/s 2024-06-06T03:46:47.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:46 smithi131 ceph-mon[36687]: pgmap v251: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 178 KiB/s wr, 43 op/s 2024-06-06T03:46:47.515 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/entry/entry_64_compat.o 2024-06-06T03:46:47.529 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/keys/request_key_auth.o 2024-06-06T03:46:47.531 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/iomap/iter.o 2024-06-06T03:46:48.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.205+0000 7f58fe7fc700 1 -- 172.21.15.40:0/2323562422 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563d13fe13d0 con 0x563d1410e3a0 2024-06-06T03:46:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.205+0000 7f58fe7fc700 10 client.15711 handle_command_reply: tid=0 2024-06-06T03:46:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.205+0000 7f5927ff6b80 1 client.15711 shutdown 2024-06-06T03:46:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.205+0000 7f5927ff6b80 1 -- 172.21.15.40:0/2323562422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408e500 msgr2=0x563d14097a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.205+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/2323562422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408e500 0x563d14097a60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5900009350 tx=0x7f5900005360 comp rx=0 tx=0).stop 2024-06-06T03:46:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.205+0000 7f5927ff6b80 1 -- 172.21.15.40:0/2323562422 shutdown_connections 2024-06-06T03:46:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.205+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/2323562422 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x563d1410e3a0 0x563d14110790 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.205+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/2323562422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d1408ee20 0x563d14097fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.206+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/2323562422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d14088c50 0x563d1409b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.206+0000 7f5927ff6b80 1 --2- 172.21.15.40:0/2323562422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563d1408e500 0x563d14097a60 secure :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5900009350 tx=0x7f5900005360 comp rx=0 tx=0).stop 2024-06-06T03:46:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.206+0000 7f5927ff6b80 1 -- 172.21.15.40:0/2323562422 >> 172.21.15.40:0/2323562422 conn(0x563d13fb5180 msgr2=0x563d1408cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.206+0000 7f58fe7fc700 0 client.15711 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:46:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.206+0000 7f5927ff6b80 1 -- 172.21.15.40:0/2323562422 shutdown_connections 2024-06-06T03:46:48.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.207+0000 7f5927ff6b80 1 -- 172.21.15.40:0/2323562422 wait complete. 2024-06-06T03:46:48.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.208+0000 7f5927ff6b80 20 client.15711 trim_cache size 0 max 16384 2024-06-06T03:46:48.208 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:46:48.208 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 39, 2024-06-06T03:46:48.208 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:46:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 24, 2024-06-06T03:46:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": { 2024-06-06T03:46:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: "61b36b60": { 2024-06-06T03:46:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 24, 2024-06-06T03:46:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: "age_ref": 0.0, 2024-06-06T03:46:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:46:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RELEASED", 2024-06-06T03:46:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:46:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:46:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: "timeout": 90.0, 2024-06-06T03:46:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: "expiration": 145.6, 2024-06-06T03:46:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: "members": { 2024-06-06T03:46:48.209 INFO:teuthology.orchestra.run.smithi040.stdout: "file:/": { 2024-06-06T03:46:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: "excluded": false, 2024-06-06T03:46:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:46:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RELEASED", 2024-06-06T03:46:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:46:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:46:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:46:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:46:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:46:48.210 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:46:48.210 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:46:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.209+0000 7f591d773700 1 -- 172.21.15.40:0/2555723093 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f59040a2d90 msgr2=0x7f59040a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.209+0000 7f591d773700 1 --2- 172.21.15.40:0/2555723093 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f59040a2d90 0x7f59040a5250 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5918199bf0 tx=0x7f59080073d0 comp rx=0 tx=0).stop 2024-06-06T03:46:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.209+0000 7f591d773700 1 -- 172.21.15.40:0/2555723093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5918108500 msgr2=0x7f5918198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.209+0000 7f591d773700 1 --2- 172.21.15.40:0/2555723093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5918108500 0x7f5918198ab0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5910007ee0 tx=0x7f591000d3b0 comp rx=0 tx=0).stop 2024-06-06T03:46:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.209+0000 7f591d773700 1 -- 172.21.15.40:0/2555723093 shutdown_connections 2024-06-06T03:46:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.209+0000 7f591d773700 1 --2- 172.21.15.40:0/2555723093 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f59040a2d90 0x7f59040a5250 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.209+0000 7f591d773700 1 --2- 172.21.15.40:0/2555723093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f591806a170 0x7f591819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.209+0000 7f591d773700 1 --2- 172.21.15.40:0/2555723093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5918108500 0x7f5918198ab0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.209+0000 7f591d773700 1 --2- 172.21.15.40:0/2555723093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f591806ab30 0x7f591819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.209+0000 7f591d773700 1 -- 172.21.15.40:0/2555723093 >> 172.21.15.40:0/2555723093 conn(0x7f5918078340 msgr2=0x7f5918100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.209+0000 7f591d773700 1 -- 172.21.15.40:0/2555723093 shutdown_connections 2024-06-06T03:46:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:48.209+0000 7f591d773700 1 -- 172.21.15.40:0/2555723093 wait complete. 2024-06-06T03:46:48.221 INFO:tasks.workunit.client.0.smithi040.stdout: CC ipc/ipc_sysctl.o 2024-06-06T03:46:48.239 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/truncate.o 2024-06-06T03:46:48.266 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs_common/grace.o 2024-06-06T03:46:48.273 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/entry/syscall_32.o 2024-06-06T03:46:48.295 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/sched/build_utility.o 2024-06-06T03:46:48.307 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/amd/core.o 2024-06-06T03:46:48.746 INFO:tasks.workunit.client.0.smithi040.stdout: CC ipc/mqueue.o 2024-06-06T03:46:48.780 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/keys/user_defined.o 2024-06-06T03:46:48.857 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/iomap/buffered-io.o 2024-06-06T03:46:48.877 INFO:tasks.workunit.client.0.smithi040.stdout: AR arch/x86/entry/built-in.a 2024-06-06T03:46:48.930 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/nfs_common/built-in.a 2024-06-06T03:46:48.938 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/realmode/init.o 2024-06-06T03:46:48.962 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/iomap/direct-io.o 2024-06-06T03:46:49.020 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:48 smithi040 ceph-mon[32220]: pgmap v252: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 131 KiB/s wr, 33 op/s 2024-06-06T03:46:49.084 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '61b36b60', total seconds elapsed: 41.3 2024-06-06T03:46:49.084 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.6 2024-06-06T03:46:49.084 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:46:49.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:48 smithi104 ceph-mon[36569]: pgmap v252: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 131 KiB/s wr, 33 op/s 2024-06-06T03:46:49.136 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/vmscan.o 2024-06-06T03:46:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:48 smithi131 ceph-mon[36687]: pgmap v252: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 131 KiB/s wr, 33 op/s 2024-06-06T03:46:49.301 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/amd/lbr.o 2024-06-06T03:46:49.370 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/keys/compat.o 2024-06-06T03:46:49.557 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/realmode/rm/header.o 2024-06-06T03:46:49.626 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/realmode/rm/trampoline_64.o 2024-06-06T03:46:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:46:49.747 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/realmode/rm/stack.o 2024-06-06T03:46:49.843 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/realmode/rm/reboot.o 2024-06-06T03:46:49.946 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/realmode/rm/wakeup_asm.o 2024-06-06T03:46:49.948 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/amd/ibs.o 2024-06-06T03:46:50.028 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/realmode/rm/wakemain.o 2024-06-06T03:46:50.060 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/realmode/rm/video-mode.o 2024-06-06T03:46:50.069 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/keys/proc.o 2024-06-06T03:46:50.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:49 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:46:50.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:49 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:46:50.219 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/realmode/rm/copy.o 2024-06-06T03:46:50.232 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/quota/dquot.o 2024-06-06T03:46:50.267 INFO:tasks.workunit.client.0.smithi040.stdout: CC ipc/namespace.o 2024-06-06T03:46:50.280 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/realmode/rm/bioscall.o 2024-06-06T03:46:50.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:49 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:46:50.390 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/realmode/rm/regs.o 2024-06-06T03:46:50.489 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/realmode/rm/video-vga.o 2024-06-06T03:46:50.503 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/iomap/fiemap.o 2024-06-06T03:46:50.600 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/realmode/rm/video-vesa.o 2024-06-06T03:46:50.716 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/realmode/rm/video-bios.o 2024-06-06T03:46:50.723 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/keys/sysctl.o 2024-06-06T03:46:50.872 INFO:tasks.workunit.client.0.smithi040.stdout: PASYMS arch/x86/realmode/rm/pasyms.h 2024-06-06T03:46:50.882 INFO:tasks.workunit.client.0.smithi040.stdout: CC ipc/mq_sysctl.o 2024-06-06T03:46:50.915 INFO:tasks.workunit.client.0.smithi040.stdout: LDS arch/x86/realmode/rm/realmode.lds 2024-06-06T03:46:50.942 INFO:tasks.workunit.client.0.smithi040.stdout: LD arch/x86/realmode/rm/realmode.elf 2024-06-06T03:46:50.961 INFO:tasks.workunit.client.0.smithi040.stdout: RELOCS arch/x86/realmode/rm/realmode.relocs 2024-06-06T03:46:50.968 INFO:tasks.workunit.client.0.smithi040.stdout: OBJCOPY arch/x86/realmode/rm/realmode.bin 2024-06-06T03:46:50.980 INFO:tasks.workunit.client.0.smithi040.stdout: AS arch/x86/realmode/rmpiggy.o 2024-06-06T03:46:51.005 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/amd/uncore.o 2024-06-06T03:46:51.034 INFO:tasks.workunit.client.0.smithi040.stdout: AR arch/x86/realmode/built-in.a 2024-06-06T03:46:51.114 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/iomap/seek.o 2024-06-06T03:46:51.161 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/quota/quota_v2.o 2024-06-06T03:46:51.212 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:50 smithi040 ceph-mon[32220]: pgmap v253: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 151 KiB/s wr, 41 op/s 2024-06-06T03:46:51.273 INFO:tasks.workunit.client.0.smithi040.stdout: AR ipc/built-in.a 2024-06-06T03:46:51.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:50 smithi131 ceph-mon[36687]: pgmap v253: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 151 KiB/s wr, 41 op/s 2024-06-06T03:46:51.317 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/keys/keyctl_pkey.o 2024-06-06T03:46:51.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:50 smithi104 ceph-mon[36569]: pgmap v253: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 151 KiB/s wr, 41 op/s 2024-06-06T03:46:51.424 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/sched/built-in.a 2024-06-06T03:46:51.477 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/locking/mutex.o 2024-06-06T03:46:51.502 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/task_mmu.o 2024-06-06T03:46:51.781 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/iomap/swapfile.o 2024-06-06T03:46:51.816 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/kernfs/mount.o 2024-06-06T03:46:51.848 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/shmem.o 2024-06-06T03:46:51.878 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/amd/iommu.o 2024-06-06T03:46:51.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.916+0000 7f58f88c5700 1 -- 172.21.15.40:0/896392080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f4109ff0 msgr2=0x7f58f411dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:51.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.916+0000 7f58f88c5700 1 --2- 172.21.15.40:0/896392080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f4109ff0 0x7f58f411dc80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00b0a0 tx=0x7f58ec00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:46:51.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.917+0000 7f58f88c5700 1 -- 172.21.15.40:0/896392080 shutdown_connections 2024-06-06T03:46:51.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.917+0000 7f58f88c5700 1 --2- 172.21.15.40:0/896392080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f4110b90 0x7f58f4109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:51.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.917+0000 7f58f88c5700 1 --2- 172.21.15.40:0/896392080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f4109ff0 0x7f58f411dc80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:51.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.917+0000 7f58f88c5700 1 --2- 172.21.15.40:0/896392080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f4110f70 0x7f58f41099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:51.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.917+0000 7f58f88c5700 1 -- 172.21.15.40:0/896392080 >> 172.21.15.40:0/896392080 conn(0x7f58f406d4c0 msgr2=0x7f58f406d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:51.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.918+0000 7f58f88c5700 1 -- 172.21.15.40:0/896392080 shutdown_connections 2024-06-06T03:46:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.918+0000 7f58f88c5700 1 -- 172.21.15.40:0/896392080 wait complete. 2024-06-06T03:46:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.919+0000 7f58f88c5700 1 Processor -- start 2024-06-06T03:46:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.919+0000 7f58f88c5700 1 -- start start 2024-06-06T03:46:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.919+0000 7f58f88c5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f4109ff0 0x7f58f411d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.919+0000 7f58f88c5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f4110b90 0x7f58f4116810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.919+0000 7f58f88c5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f4110f70 0x7f58f4116d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.919+0000 7f58f88c5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4121620 con 0x7f58f4110b90 2024-06-06T03:46:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.919+0000 7f58f88c5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41214a0 con 0x7f58f4110f70 2024-06-06T03:46:51.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.919+0000 7f58f88c5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41217a0 con 0x7f58f4109ff0 2024-06-06T03:46:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.920+0000 7f58f259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f4110b90 0x7f58f4116810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.920+0000 7f58f259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f4110b90 0x7f58f4116810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36256/0 (socket says 172.21.15.40:36256) 2024-06-06T03:46:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.920+0000 7f58f259c700 1 -- 172.21.15.40:0/4054114276 learned_addr learned my addr 172.21.15.40:0/4054114276 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.920+0000 7f58f259c700 1 -- 172.21.15.40:0/4054114276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f4109ff0 msgr2=0x7f58f411d780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:46:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.920+0000 7f58f259c700 1 --2- 172.21.15.40:0/4054114276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f4109ff0 0x7f58f411d780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.920+0000 7f58f259c700 1 -- 172.21.15.40:0/4054114276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f4110f70 msgr2=0x7f58f4116d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T03:46:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.920+0000 7f58f259c700 1 --2- 172.21.15.40:0/4054114276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f4110f70 0x7f58f4116d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.920+0000 7f58f259c700 1 -- 172.21.15.40:0/4054114276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f4117390 con 0x7f58f4110b90 2024-06-06T03:46:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.921+0000 7f58f259c700 1 --2- 172.21.15.40:0/4054114276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f4110b90 0x7f58f4116810 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f58e800b780 tx=0x7f58e800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.922+0000 7f58dbfff700 1 -- 172.21.15.40:0/4054114276 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e800d660 con 0x7f58f4110b90 2024-06-06T03:46:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.922+0000 7f58f88c5700 1 -- 172.21.15.40:0/4054114276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f411b2b0 con 0x7f58f4110b90 2024-06-06T03:46:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.922+0000 7f58f88c5700 1 -- 172.21.15.40:0/4054114276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f411b840 con 0x7f58f4110b90 2024-06-06T03:46:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.922+0000 7f58dbfff700 1 -- 172.21.15.40:0/4054114276 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e800dce0 con 0x7f58f4110b90 2024-06-06T03:46:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.922+0000 7f58dbfff700 1 -- 172.21.15.40:0/4054114276 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8017400 con 0x7f58f4110b90 2024-06-06T03:46:51.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.926+0000 7f58f88c5700 1 -- 172.21.15.40:0/4054114276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e0005330 con 0x7f58f4110b90 2024-06-06T03:46:51.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.927+0000 7f58dbfff700 1 -- 172.21.15.40:0/4054114276 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f58e80175a0 con 0x7f58f4110b90 2024-06-06T03:46:51.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.928+0000 7f58dbfff700 1 --2- 172.21.15.40:0/4054114276 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f58dc0a2e90 0x7f58dc0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:51.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.929+0000 7f58f2d9d700 1 --2- 172.21.15.40:0/4054114276 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f58dc0a2e90 0x7f58dc0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:51.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.930+0000 7f58f2d9d700 1 --2- 172.21.15.40:0/4054114276 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f58dc0a2e90 0x7f58dc0a5350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009760 tx=0x7f58e4006c60 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:51.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.930+0000 7f58dbfff700 1 -- 172.21.15.40:0/4054114276 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8426+0+0 (secure 0 0 0) 0x7f58e80ce7e0 con 0x7f58f4110b90 2024-06-06T03:46:51.937 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/quota/quota_tree.o 2024-06-06T03:46:51.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:51.937+0000 7f58dbfff700 1 -- 172.21.15.40:0/4054114276 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f58e8013030 con 0x7f58f4110b90 2024-06-06T03:46:51.939 INFO:tasks.workunit.client.0.smithi040.stdout: AR security/keys/built-in.a 2024-06-06T03:46:52.066 INFO:tasks.workunit.client.0.smithi040.stdout: GEN security/selinux/flask.h security/selinux/av_permissions.h 2024-06-06T03:46:52.130 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/avc.o 2024-06-06T03:46:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:52.486+0000 7f58f88c5700 1 -- 172.21.15.40:0/4054114276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58e00058f0 con 0x7f58f4110b90 2024-06-06T03:46:52.494 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:46:52.494 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:46:03.426078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24443,24470,24425,24403,24398]},"id":1}]} 2024-06-06T03:46:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:52.489+0000 7f58dbfff700 1 -- 172.21.15.40:0/4054114276 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11947 (secure 0 0 0) 0x7f58e800de90 con 0x7f58f4110b90 2024-06-06T03:46:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:52.493+0000 7f58d9ffb700 1 -- 172.21.15.40:0/4054114276 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f58dc0a2e90 msgr2=0x7f58dc0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:52.494+0000 7f58d9ffb700 1 --2- 172.21.15.40:0/4054114276 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f58dc0a2e90 0x7f58dc0a5350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009760 tx=0x7f58e4006c60 comp rx=0 tx=0).stop 2024-06-06T03:46:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:52.494+0000 7f58d9ffb700 1 -- 172.21.15.40:0/4054114276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f4110b90 msgr2=0x7f58f4116810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:52.494+0000 7f58d9ffb700 1 --2- 172.21.15.40:0/4054114276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f4110b90 0x7f58f4116810 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f58e800b780 tx=0x7f58e800bb40 comp rx=0 tx=0).stop 2024-06-06T03:46:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:52.494+0000 7f58d9ffb700 1 -- 172.21.15.40:0/4054114276 shutdown_connections 2024-06-06T03:46:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:52.494+0000 7f58d9ffb700 1 --2- 172.21.15.40:0/4054114276 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f58dc0a2e90 0x7f58dc0a5350 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:52.494+0000 7f58d9ffb700 1 --2- 172.21.15.40:0/4054114276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f4109ff0 0x7f58f411d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:52.494+0000 7f58d9ffb700 1 --2- 172.21.15.40:0/4054114276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f4110f70 0x7f58f4116d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:52.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:52.494+0000 7f58d9ffb700 1 --2- 172.21.15.40:0/4054114276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f4110b90 0x7f58f4116810 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:52.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:52.494+0000 7f58d9ffb700 1 -- 172.21.15.40:0/4054114276 >> 172.21.15.40:0/4054114276 conn(0x7f58f406d4c0 msgr2=0x7f58f40722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:52.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:52.494+0000 7f58d9ffb700 1 -- 172.21.15.40:0/4054114276 shutdown_connections 2024-06-06T03:46:52.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:52.495+0000 7f58d9ffb700 1 -- 172.21.15.40:0/4054114276 wait complete. 2024-06-06T03:46:52.503 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/kernfs/inode.o 2024-06-06T03:46:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 39 2024-06-06T03:46:52.562 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/locking/semaphore.o 2024-06-06T03:46:52.567 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/iomap/built-in.a 2024-06-06T03:46:52.602 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/kernfs/dir.o 2024-06-06T03:46:52.725 INFO:tasks.workunit.client.0.smithi040.stdout: AR arch/x86/events/amd/built-in.a 2024-06-06T03:46:52.759 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/quota/quota.o 2024-06-06T03:46:52.776 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/intel/core.o 2024-06-06T03:46:52.896 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/inode.o 2024-06-06T03:46:53.147 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/intel/bts.o 2024-06-06T03:46:53.180 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/locking/rwsem.o 2024-06-06T03:46:53.206 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:52 smithi040 ceph-mon[32220]: pgmap v254: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 117 KiB/s wr, 27 op/s 2024-06-06T03:46:53.206 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:52 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/4054114276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:46:53.261 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 145.6 --await-for 45.0' 2024-06-06T03:46:53.262 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 145.6 --await-for 45.0 2024-06-06T03:46:53.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:52 smithi131 ceph-mon[36687]: pgmap v254: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 117 KiB/s wr, 27 op/s 2024-06-06T03:46:53.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:52 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/4054114276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:46:53.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:52 smithi104 ceph-mon[36569]: pgmap v254: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 117 KiB/s wr, 27 op/s 2024-06-06T03:46:53.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:52 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/4054114276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:46:53.618 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/root.o 2024-06-06T03:46:53.666 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/kernfs/file.o 2024-06-06T03:46:53.721 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/hooks.o 2024-06-06T03:46:53.776 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:46:53.863 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/quota/kqid.o 2024-06-06T03:46:53.993 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/util.o 2024-06-06T03:46:54.034 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/locking/percpu-rwsem.o 2024-06-06T03:46:54.140 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/intel/ds.o 2024-06-06T03:46:55.210 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:54 smithi040 ceph-mon[32220]: pgmap v255: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 117 KiB/s wr, 24 op/s 2024-06-06T03:46:55.210 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:54 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:46:55.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:54 smithi131 ceph-mon[36687]: pgmap v255: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 117 KiB/s wr, 24 op/s 2024-06-06T03:46:55.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:54 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:46:55.336 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/kernfs/symlink.o 2024-06-06T03:46:55.339 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/base.o 2024-06-06T03:46:55.340 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/locking/spinlock.o 2024-06-06T03:46:55.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:54 smithi104 ceph-mon[36569]: pgmap v255: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 117 KiB/s wr, 24 op/s 2024-06-06T03:46:55.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:54 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:46:55.522 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/quota/netlink.o 2024-06-06T03:46:55.597 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/generic.o 2024-06-06T03:46:55.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.915+0000 7f82df676700 1 -- 172.21.15.40:0/1060990383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d81094a0 msgr2=0x7f82d81159b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:55.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.915+0000 7f82df676700 1 --2- 172.21.15.40:0/1060990383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d81094a0 0x7f82d81159b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f82d000b0a0 tx=0x7f82d000b3b0 comp rx=0 tx=0).stop 2024-06-06T03:46:55.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.919+0000 7f82df676700 1 -- 172.21.15.40:0/1060990383 shutdown_connections 2024-06-06T03:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.919+0000 7f82df676700 1 --2- 172.21.15.40:0/1060990383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d81094a0 0x7f82d81159b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.919+0000 7f82df676700 1 --2- 172.21.15.40:0/1060990383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d810eb70 0x7f82d810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.919+0000 7f82df676700 1 --2- 172.21.15.40:0/1060990383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d8108ae0 0x7f82d8108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.919+0000 7f82df676700 1 -- 172.21.15.40:0/1060990383 >> 172.21.15.40:0/1060990383 conn(0x7f82d8075ca0 msgr2=0x7f82d81072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.919+0000 7f82df676700 1 -- 172.21.15.40:0/1060990383 shutdown_connections 2024-06-06T03:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.919+0000 7f82df676700 1 -- 172.21.15.40:0/1060990383 wait complete. 2024-06-06T03:46:55.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.920+0000 7f82df676700 1 Processor -- start 2024-06-06T03:46:55.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.923+0000 7f82df676700 1 -- start start 2024-06-06T03:46:55.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.939+0000 7f82df676700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d8108ae0 0x7f82d819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.941+0000 7f82df676700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d81094a0 0x7f82d819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:55.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.942+0000 7f82df676700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d810eb70 0x7f82d81a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:55.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.944+0000 7f82ddc13700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d810eb70 0x7f82d81a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:55.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.945+0000 7f82ddc13700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d810eb70 0x7f82d81a39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.40:35586/0 (socket says 172.21.15.40:35586) 2024-06-06T03:46:55.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.946+0000 7f82dd412700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d8108ae0 0x7f82d819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:55.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.946+0000 7f82dd412700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d8108ae0 0x7f82d819f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36274/0 (socket says 172.21.15.40:36274) 2024-06-06T03:46:55.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.947+0000 7f82dcc11700 1 --2- 172.21.15.40:0/1891692933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d81094a0 0x7f82d819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:55.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.946+0000 7f82ddc13700 1 -- 172.21.15.40:0/1891692933 learned_addr learned my addr 172.21.15.40:0/1891692933 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:55.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.947+0000 7f82df676700 1 -- 172.21.15.40:0/1891692933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8118eb0 con 0x7f82d8108ae0 2024-06-06T03:46:55.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.947+0000 7f82df676700 1 -- 172.21.15.40:0/1891692933 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8118d30 con 0x7f82d81094a0 2024-06-06T03:46:55.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.947+0000 7f82df676700 1 -- 172.21.15.40:0/1891692933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8119030 con 0x7f82d810eb70 2024-06-06T03:46:55.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.950+0000 7f82dd412700 1 -- 172.21.15.40:0/1891692933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d810eb70 msgr2=0x7f82d81a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:55.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.951+0000 7f82dd412700 1 --2- 172.21.15.40:0/1891692933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d810eb70 0x7f82d81a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:55.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.951+0000 7f82dd412700 1 -- 172.21.15.40:0/1891692933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d81094a0 msgr2=0x7f82d819f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:55.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.951+0000 7f82dd412700 1 --2- 172.21.15.40:0/1891692933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d81094a0 0x7f82d819f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:55.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.951+0000 7f82dd412700 1 -- 172.21.15.40:0/1891692933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d81a40f0 con 0x7f82d8108ae0 2024-06-06T03:46:55.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.951+0000 7f82dcc11700 1 --2- 172.21.15.40:0/1891692933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d81094a0 0x7f82d819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:46:55.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.955+0000 7f82dd412700 1 --2- 172.21.15.40:0/1891692933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d8108ae0 0x7f82d819f0e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f82c800e9f0 tx=0x7f82c800edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:55.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.956+0000 7f82ce7fc700 1 -- 172.21.15.40:0/1891692933 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c800cb70 con 0x7f82d8108ae0 2024-06-06T03:46:55.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.957+0000 7f82df676700 1 -- 172.21.15.40:0/1891692933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d81a43e0 con 0x7f82d8108ae0 2024-06-06T03:46:55.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.957+0000 7f82ce7fc700 1 -- 172.21.15.40:0/1891692933 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c8004500 con 0x7f82d8108ae0 2024-06-06T03:46:55.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.957+0000 7f82ce7fc700 1 -- 172.21.15.40:0/1891692933 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8010430 con 0x7f82d8108ae0 2024-06-06T03:46:55.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.957+0000 7f82df676700 1 -- 172.21.15.40:0/1891692933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d81abc60 con 0x7f82d8108ae0 2024-06-06T03:46:55.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.962+0000 7f82e9ef9b80 1 Processor -- start 2024-06-06T03:46:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.962+0000 7f82ce7fc700 1 -- 172.21.15.40:0/1891692933 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f82c8004000 con 0x7f82d8108ae0 2024-06-06T03:46:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.962+0000 7f82e9ef9b80 1 -- start start 2024-06-06T03:46:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.962+0000 7f82e9ef9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703bab590 0x557703bab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.962+0000 7f82e9ef9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703babeb0 0x557703ba57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.962+0000 7f82e9ef9b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703ba6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.962+0000 7f82e9ef9b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557703a980f0 con 0x557703ba5ce0 2024-06-06T03:46:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.962+0000 7f82e9ef9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557703af69e0 con 0x557703babeb0 2024-06-06T03:46:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.962+0000 7f82e9ef9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557703a5e5f0 con 0x557703bab590 2024-06-06T03:46:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.963+0000 7f82ddc13700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703bab590 0x557703bab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.963+0000 7f82dcc11700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703ba6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.963+0000 7f82ddc13700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703bab590 0x557703bab970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.40:35594/0 (socket says 172.21.15.40:35594) 2024-06-06T03:46:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.963+0000 7f82dcc11700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703ba6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36290/0 (socket says 172.21.15.40:36290) 2024-06-06T03:46:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.963+0000 7f82ddc13700 1 -- 172.21.15.40:0/1787792129 learned_addr learned my addr 172.21.15.40:0/1787792129 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:55.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.965+0000 7f82ddc13700 1 -- 172.21.15.40:0/1787792129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703babeb0 msgr2=0x557703ba57a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T03:46:55.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.965+0000 7f82ddc13700 1 --2- 172.21.15.40:0/1787792129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703babeb0 0x557703ba57a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:55.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.965+0000 7f82ddc13700 1 -- 172.21.15.40:0/1787792129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 msgr2=0x557703ba6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:55.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.965+0000 7f82ddc13700 1 --2- 172.21.15.40:0/1787792129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703ba6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:55.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.965+0000 7f82ddc13700 1 -- 172.21.15.40:0/1787792129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557703a45a60 con 0x557703bab590 2024-06-06T03:46:55.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.965+0000 7f82dcc11700 1 --2- 172.21.15.40:0/1787792129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703ba6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:46:55.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.966+0000 7f82ce7fc700 1 --2- 172.21.15.40:0/1891692933 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f82c40a2f00 0x7f82c40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:55.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.966+0000 7f82dd412700 1 --2- 172.21.15.40:0/1891692933 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f82c40a2f00 0x7f82c40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:55.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.966+0000 7f82ce7fc700 1 -- 172.21.15.40:0/1891692933 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8426+0+0 (secure 0 0 0) 0x7f82c80d3060 con 0x7f82d8108ae0 2024-06-06T03:46:55.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.966+0000 7f82ddc13700 1 --2- 172.21.15.40:0/1787792129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703bab590 0x557703bab970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f82d00060b0 tx=0x7f82d0009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e20e9a305122dbe3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:55.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.967+0000 7f82dd412700 1 --2- 172.21.15.40:0/1891692933 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f82c40a2f00 0x7f82c40a53c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f82d81a4970 tx=0x7f82c80908e0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:55.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.968+0000 7f82c37fe700 1 -- 172.21.15.40:0/1787792129 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d000bab0 con 0x557703bab590 2024-06-06T03:46:55.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.968+0000 7f82c37fe700 1 -- 172.21.15.40:0/1787792129 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d000bc50 con 0x557703bab590 2024-06-06T03:46:55.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.968+0000 7f82c37fe700 1 -- 172.21.15.40:0/1787792129 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d00074e0 con 0x557703bab590 2024-06-06T03:46:55.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.969+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/1787792129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703bab590 msgr2=0x557703bab970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:55.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.969+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/1787792129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703bab590 0x557703bab970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f82d00060b0 tx=0x7f82d0009e00 comp rx=0 tx=0).stop 2024-06-06T03:46:55.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.969+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/1787792129 shutdown_connections 2024-06-06T03:46:55.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.969+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/1787792129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703bab590 0x557703bab970 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:55.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.969+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/1787792129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703ba6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:55.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.969+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/1787792129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703babeb0 0x557703ba57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:55.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.969+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/1787792129 >> 172.21.15.40:0/1787792129 conn(0x557703ad2210 msgr2=0x557703ba0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:55.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.973+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/1787792129 shutdown_connections 2024-06-06T03:46:55.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.973+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/1787792129 wait complete. 2024-06-06T03:46:55.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.973+0000 7f82e9ef9b80 1 -- start start 2024-06-06T03:46:55.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.973+0000 7f82e9ef9b80 1 Processor -- start 2024-06-06T03:46:55.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.974+0000 7f82e9ef9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703ba5ce0 0x557703bb1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:55.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.974+0000 7f82e9ef9b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703bab590 0x557703baefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:55.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.974+0000 7f82e9ef9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703babeb0 0x557703baf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:55.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.974+0000 7f82e9ef9b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557703a97150 con 0x557703bab590 2024-06-06T03:46:55.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.974+0000 7f82e9ef9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557703af29a0 con 0x557703ba5ce0 2024-06-06T03:46:55.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.974+0000 7f82e9ef9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557703bb2830 con 0x557703babeb0 2024-06-06T03:46:55.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.974+0000 7f82e9ef9b80 10 client.0 _ll_register_callbacks cb 0x557703a8c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:46:55.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.975+0000 7f82ddc13700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703ba5ce0 0x557703bb1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:55.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.975+0000 7f82ddc13700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703babeb0 0x557703baf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:55.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.975+0000 7f82ddc13700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703ba5ce0 0x557703bb1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.40:39922/0 (socket says 172.21.15.40:39922) 2024-06-06T03:46:55.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.975+0000 7f82ddc13700 1 -- 172.21.15.40:0/251593760 learned_addr learned my addr 172.21.15.40:0/251593760 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:55.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.975+0000 7f82ddc13700 1 -- 172.21.15.40:0/251593760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703babeb0 msgr2=0x557703baf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:55.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.975+0000 7f82ddc13700 1 --2- 172.21.15.40:0/251593760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703babeb0 0x557703baf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:55.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.975+0000 7f82ddc13700 1 -- 172.21.15.40:0/251593760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703bab590 msgr2=0x557703baefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:55.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.975+0000 7f82ddc13700 1 --2- 172.21.15.40:0/251593760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703bab590 0x557703baefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:55.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.975+0000 7f82ddc13700 1 -- 172.21.15.40:0/251593760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557703abc8e0 con 0x557703ba5ce0 2024-06-06T03:46:55.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.976+0000 7f82ddc13700 1 --2- 172.21.15.40:0/251593760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703ba5ce0 0x557703bb1cd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82d000b070 tx=0x7f82d0012b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf1c0f4eb9f4fd1f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:55.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.976+0000 7f82c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-06-06T03:46:55.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.976+0000 7f82c0ff9700 1 -- 172.21.15.40:0/251593760 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0004e50 con 0x557703ba5ce0 2024-06-06T03:46:55.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.976+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/251593760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82d000bc50 con 0x557703ba5ce0 2024-06-06T03:46:55.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.976+0000 7f82c0ff9700 1 -- 172.21.15.40:0/251593760 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8426+0+0 (secure 0 0 0) 0x7f82d000be00 con 0x557703ba5ce0 2024-06-06T03:46:55.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.976+0000 7f82c0ff9700 1 -- 172.21.15.40:0/251593760 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f82d00076c0 con 0x557703ba5ce0 2024-06-06T03:46:55.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.976+0000 7f82e9ef9b80 10 client.25361 fetch_fsmap learned FSMap version 41 2024-06-06T03:46:55.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.976+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/251593760 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557703bce0a0 con 0x557703ba5ce0 2024-06-06T03:46:55.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.977+0000 7f82c0ff9700 1 -- 172.21.15.40:0/251593760 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d000bc50 con 0x557703ba5ce0 2024-06-06T03:46:55.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.977+0000 7f82c0ff9700 1 -- 172.21.15.40:0/251593760 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 41) ==== 8394+0+0 (secure 0 0 0) 0x7f82d0011070 con 0x557703ba5ce0 2024-06-06T03:46:55.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.977+0000 7f82e9ef9b80 10 client.25361 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T03:46:55.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.977+0000 7f82e9ef9b80 10 client.25361 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:46:55.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.977+0000 7f82e9ef9b80 20 client.25361 populate_metadata read hostname 'smithi040' 2024-06-06T03:46:55.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.977+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/251593760 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x557703bd0860 0x557703bd2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:55.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.977+0000 7f82e9ef9b80 4 client.25361 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:46:55.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.977+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/251593760 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557703b03ee0 con 0x557703bd0860 2024-06-06T03:46:55.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.978+0000 7f82dcc11700 1 --2- 172.21.15.40:0/251593760 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x557703bd0860 0x557703bd2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:55.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.979+0000 7f82dcc11700 1 --2- 172.21.15.40:0/251593760 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x557703bd0860 0x557703bd2cc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4087d1d627d08e3 server_cookie=a6704f7115208194 in_seq=0 out_seq=0 2024-06-06T03:46:55.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:55.979+0000 7f82c0ff9700 10 client.25361 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:46:56.005 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/locking/osq_lock.o 2024-06-06T03:46:56.015 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:55 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:46:56.112 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/kernfs/built-in.a 2024-06-06T03:46:56.208 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/locking/qspinlock.o 2024-06-06T03:46:56.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.281+0000 7f82c0ff9700 1 -- 172.21.15.40:0/251593760 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557703b03ee0 con 0x557703bd0860 2024-06-06T03:46:56.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.281+0000 7f82c0ff9700 10 client.25361 handle_command_reply: tid=0 2024-06-06T03:46:56.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.281+0000 7f82e9ef9b80 1 client.25361 shutdown 2024-06-06T03:46:56.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.281+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/251593760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703ba5ce0 msgr2=0x557703bb1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:56.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.281+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/251593760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703ba5ce0 0x557703bb1cd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82d000b070 tx=0x7f82d0012b70 comp rx=0 tx=0).stop 2024-06-06T03:46:56.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.281+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/251593760 shutdown_connections 2024-06-06T03:46:56.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.281+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/251593760 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x557703bd0860 0x557703bd2cc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:56.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.281+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/251593760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703babeb0 0x557703baf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:56.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.281+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/251593760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703ba5ce0 0x557703bb1cd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:56.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.281+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/251593760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703bab590 0x557703baefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:56.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.281+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/251593760 >> 172.21.15.40:0/251593760 conn(0x557703ad2210 msgr2=0x557703b9fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:56.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.281+0000 7f82c0ff9700 0 client.25361 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:46:56.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.281+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/251593760 shutdown_connections 2024-06-06T03:46:56.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.281+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/251593760 wait complete. 2024-06-06T03:46:56.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.282+0000 7f82e9ef9b80 20 client.25361 trim_cache size 0 max 16384 2024-06-06T03:46:56.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:55 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:46:56.307 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/intel/knc.o 2024-06-06T03:46:56.310 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/mmzone.o 2024-06-06T03:46:56.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.310+0000 7f82e9ef9b80 1 Processor -- start 2024-06-06T03:46:56.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.310+0000 7f82e9ef9b80 1 -- start start 2024-06-06T03:46:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82e9ef9b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703bb1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82e9ef9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703bab590 0x557703ba03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82e9ef9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703babeb0 0x557703ba0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82e9ef9b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f82d80731e0 con 0x557703ba5ce0 2024-06-06T03:46:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82e9ef9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f82d81ac680 con 0x557703bab590 2024-06-06T03:46:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82e9ef9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f82d80738e0 con 0x557703babeb0 2024-06-06T03:46:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82ddc13700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703bb1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82ddc13700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703bb1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36314/0 (socket says 172.21.15.40:36314) 2024-06-06T03:46:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82ddc13700 1 -- 172.21.15.40:0/3384110438 learned_addr learned my addr 172.21.15.40:0/3384110438 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82dcc11700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703bab590 0x557703ba03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82ddc13700 1 --2- 172.21.15.40:0/3384110438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703babeb0 0x557703ba0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82dcc11700 1 -- 172.21.15.40:0/3384110438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703babeb0 msgr2=0x557703ba0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82dcc11700 1 --2- 172.21.15.40:0/3384110438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703babeb0 0x557703ba0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:56.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82dcc11700 1 -- 172.21.15.40:0/3384110438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 msgr2=0x557703bb1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:56.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82dcc11700 1 --2- 172.21.15.40:0/3384110438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703bb1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:56.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82dcc11700 1 -- 172.21.15.40:0/3384110438 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557703ba2150 con 0x557703bab590 2024-06-06T03:46:56.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.311+0000 7f82ddc13700 1 --2- 172.21.15.40:0/3384110438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703bb1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:46:56.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.312+0000 7f82dcc11700 1 --2- 172.21.15.40:0/3384110438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703bab590 0x557703ba03c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x557703bd3500 tx=0x7f82d4005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9701ae09828d91af server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:56.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.312+0000 7f82c2ffd700 1 -- 172.21.15.40:0/3384110438 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4013d20 con 0x557703bab590 2024-06-06T03:46:56.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.312+0000 7f82c2ffd700 1 -- 172.21.15.40:0/3384110438 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d4004530 con 0x557703bab590 2024-06-06T03:46:56.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.312+0000 7f82c2ffd700 1 -- 172.21.15.40:0/3384110438 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d401b380 con 0x557703bab590 2024-06-06T03:46:56.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.313+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/3384110438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703bab590 msgr2=0x557703ba03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:46:56.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.313+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/3384110438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703bab590 0x557703ba03c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x557703bd3500 tx=0x7f82d4005b00 comp rx=0 tx=0).stop 2024-06-06T03:46:56.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.313+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/3384110438 shutdown_connections 2024-06-06T03:46:56.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.313+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/3384110438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703babeb0 0x557703ba0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:56.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.313+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/3384110438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703bb1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:56.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.313+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/3384110438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703bab590 0x557703ba03c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:56.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.313+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/3384110438 >> 172.21.15.40:0/3384110438 conn(0x557703ad2210 msgr2=0x557703baf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:46:56.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.313+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/3384110438 shutdown_connections 2024-06-06T03:46:56.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.313+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/3384110438 wait complete. 2024-06-06T03:46:56.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.313+0000 7f82e9ef9b80 1 -- start start 2024-06-06T03:46:56.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.314+0000 7f82e9ef9b80 1 Processor -- start 2024-06-06T03:46:56.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.314+0000 7f82e9ef9b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703bb7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:56.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.314+0000 7f82e9ef9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703bab590 0x557703bb8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:56.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.314+0000 7f82e9ef9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703babeb0 0x557703bb4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:56.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.314+0000 7f82e9ef9b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f82d81abef0 con 0x557703ba5ce0 2024-06-06T03:46:56.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.314+0000 7f82e9ef9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8073fb0 con 0x557703babeb0 2024-06-06T03:46:56.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.314+0000 7f82e9ef9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f82d80711e0 con 0x557703bab590 2024-06-06T03:46:56.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.314+0000 7f82e9ef9b80 10 client.0 _ll_register_callbacks cb 0x557703a8c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:46:56.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.314+0000 7f82ddc13700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703bb7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:56.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.314+0000 7f82ddc13700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703bb7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36324/0 (socket says 172.21.15.40:36324) 2024-06-06T03:46:56.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.314+0000 7f82ddc13700 1 -- 172.21.15.40:0/1152997882 learned_addr learned my addr 172.21.15.40:0/1152997882 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:46:56.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.314+0000 7f82ddc13700 1 --2- 172.21.15.40:0/1152997882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703babeb0 0x557703bb4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:56.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.314+0000 7f82dcc11700 1 --2- 172.21.15.40:0/1152997882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703bab590 0x557703bb8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:56.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.315+0000 7f82ddc13700 1 -- 172.21.15.40:0/1152997882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703bab590 msgr2=0x557703bb8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:56.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.315+0000 7f82ddc13700 1 --2- 172.21.15.40:0/1152997882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703bab590 0x557703bb8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:56.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.315+0000 7f82ddc13700 1 -- 172.21.15.40:0/1152997882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703babeb0 msgr2=0x557703bb4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:46:56.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.315+0000 7f82ddc13700 1 --2- 172.21.15.40:0/1152997882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703babeb0 0x557703bb4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:46:56.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.315+0000 7f82ddc13700 1 -- 172.21.15.40:0/1152997882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557703bb51b0 con 0x557703ba5ce0 2024-06-06T03:46:56.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.315+0000 7f82ddc13700 1 --2- 172.21.15.40:0/1152997882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703bb7d00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f82d000b070 tx=0x7f82d0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b596a64a95b7a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:46:56.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.315+0000 7f82c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:46:56.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.315+0000 7f82c0ff9700 1 -- 172.21.15.40:0/1152997882 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0007a70 con 0x557703ba5ce0 2024-06-06T03:46:56.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.315+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/1152997882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82d4004530 con 0x557703ba5ce0 2024-06-06T03:46:56.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.315+0000 7f82c0ff9700 1 -- 172.21.15.40:0/1152997882 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8426+0+0 (secure 0 0 0) 0x7f82d000f040 con 0x557703ba5ce0 2024-06-06T03:46:56.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.315+0000 7f82c0ff9700 1 -- 172.21.15.40:0/1152997882 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f82d0007c10 con 0x557703ba5ce0 2024-06-06T03:46:56.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.315+0000 7f82e9ef9b80 10 client.15732 fetch_fsmap learned FSMap version 41 2024-06-06T03:46:56.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.316+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/1152997882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557703c3be90 con 0x557703ba5ce0 2024-06-06T03:46:56.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.316+0000 7f82c0ff9700 1 -- 172.21.15.40:0/1152997882 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d4004530 con 0x557703ba5ce0 2024-06-06T03:46:56.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.316+0000 7f82c0ff9700 1 -- 172.21.15.40:0/1152997882 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 41) ==== 8394+0+0 (secure 0 0 0) 0x7f82d0011040 con 0x557703ba5ce0 2024-06-06T03:46:56.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.316+0000 7f82e9ef9b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T03:46:56.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.316+0000 7f82e9ef9b80 10 client.15732 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:46:56.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.316+0000 7f82e9ef9b80 20 client.15732 populate_metadata read hostname 'smithi040' 2024-06-06T03:46:56.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.316+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/1152997882 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x557703c3c540 0x557703c3e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:46:56.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.316+0000 7f82e9ef9b80 4 client.15732 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.6, "await_for": 45.0}] 2024-06-06T03:46:56.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.316+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/1152997882 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.6, "await_for": 45.0}) -- 0x557703afe460 con 0x557703c3c540 2024-06-06T03:46:56.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.317+0000 7f82dcc11700 1 --2- 172.21.15.40:0/1152997882 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x557703c3c540 0x557703c3e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:46:56.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.317+0000 7f82dcc11700 1 --2- 172.21.15.40:0/1152997882 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x557703c3c540 0x557703c3e960 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dec237d9c6bc2800 server_cookie=6fe64c39b1953025 in_seq=0 out_seq=0 2024-06-06T03:46:56.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:46:56.317+0000 7f82c0ff9700 10 client.15732 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:46:56.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:55 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:46:56.357 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/vmstat.o 2024-06-06T03:46:57.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:56 smithi131 ceph-mon[36687]: pgmap v256: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 774 KiB/s wr, 57 op/s 2024-06-06T03:46:57.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:56 smithi040 ceph-mon[32220]: pgmap v256: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 774 KiB/s wr, 57 op/s 2024-06-06T03:46:57.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:56 smithi104 ceph-mon[36569]: pgmap v256: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 774 KiB/s wr, 57 op/s 2024-06-06T03:46:59.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:58 smithi131 ceph-mon[36687]: pgmap v257: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 58 op/s 2024-06-06T03:46:59.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:58 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:46:59.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:58 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:46:59.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:58 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:46:59.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:58 smithi040 ceph-mon[32220]: pgmap v257: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 58 op/s 2024-06-06T03:46:59.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:58 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:46:59.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:58 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:46:59.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:58 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:46:59.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:58 smithi104 ceph-mon[36569]: pgmap v257: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 58 op/s 2024-06-06T03:46:59.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:58 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:46:59.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:58 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:46:59.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:58 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:47:00.231 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:46:59 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:47:00.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:46:59 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:47:00.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:46:59 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:47:01.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:00 smithi131 ceph-mon[36687]: pgmap v258: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-06-06T03:47:01.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:00 smithi040 ceph-mon[32220]: pgmap v258: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-06-06T03:47:01.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:00 smithi104 ceph-mon[36569]: pgmap v258: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-06-06T03:47:03.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:02 smithi131 ceph-mon[36687]: pgmap v259: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 72 op/s 2024-06-06T03:47:03.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:02 smithi040 ceph-mon[32220]: pgmap v259: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 72 op/s 2024-06-06T03:47:03.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:02 smithi104 ceph-mon[36569]: pgmap v259: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 72 op/s 2024-06-06T03:47:05.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:04 smithi131 ceph-mon[36687]: pgmap v260: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 65 op/s 2024-06-06T03:47:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:04 smithi040 ceph-mon[32220]: pgmap v260: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 65 op/s 2024-06-06T03:47:05.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:04 smithi104 ceph-mon[36569]: pgmap v260: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 65 op/s 2024-06-06T03:47:07.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:06 smithi131 ceph-mon[36687]: pgmap v261: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 99 op/s 2024-06-06T03:47:07.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:06 smithi040 ceph-mon[32220]: pgmap v261: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 99 op/s 2024-06-06T03:47:07.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:06 smithi104 ceph-mon[36569]: pgmap v261: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 99 op/s 2024-06-06T03:47:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.683+0000 7f82c0ff9700 1 -- 172.21.15.40:0/1152997882 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557703afe460 con 0x557703c3c540 2024-06-06T03:47:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.683+0000 7f82c0ff9700 10 client.15732 handle_command_reply: tid=0 2024-06-06T03:47:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.683+0000 7f82e9ef9b80 1 client.15732 shutdown 2024-06-06T03:47:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.684+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/1152997882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 msgr2=0x557703bb7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.684+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/1152997882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703bb7d00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f82d000b070 tx=0x7f82d0000f40 comp rx=0 tx=0).stop 2024-06-06T03:47:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.684+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/1152997882 shutdown_connections 2024-06-06T03:47:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.684+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/1152997882 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x557703c3c540 0x557703c3e960 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.684+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/1152997882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557703babeb0 0x557703bb4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.684+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/1152997882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557703ba5ce0 0x557703bb7d00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.684+0000 7f82e9ef9b80 1 --2- 172.21.15.40:0/1152997882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557703bab590 0x557703bb8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.684+0000 7f82c0ff9700 0 client.15732 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:47:07.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.684+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/1152997882 >> 172.21.15.40:0/1152997882 conn(0x557703ad2210 msgr2=0x557703bba9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:47:07.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.684+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/1152997882 shutdown_connections 2024-06-06T03:47:07.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.684+0000 7f82e9ef9b80 1 -- 172.21.15.40:0/1152997882 wait complete. 2024-06-06T03:47:07.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.684+0000 7f82e9ef9b80 20 client.15732 trim_cache size 0 max 16384 2024-06-06T03:47:07.686 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:47:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 42, 2024-06-06T03:47:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:47:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 26, 2024-06-06T03:47:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": { 2024-06-06T03:47:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "9cacfcc8": { 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 26, 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "age_ref": 0.0, 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "QUIESCED", 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "timeout": 90.0, 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "expiration": 145.6, 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "members": { 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "file:/": { 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "excluded": false, 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "QUIESCED", 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:47:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:47:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:47:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:47:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:47:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:47:07.688 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:47:07.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.685+0000 7f82df676700 1 -- 172.21.15.40:0/1891692933 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f82c40a2f00 msgr2=0x7f82c40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:07.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.685+0000 7f82df676700 1 --2- 172.21.15.40:0/1891692933 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f82c40a2f00 0x7f82c40a53c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f82d81a4970 tx=0x7f82c80908e0 comp rx=0 tx=0).stop 2024-06-06T03:47:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.685+0000 7f82df676700 1 -- 172.21.15.40:0/1891692933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d8108ae0 msgr2=0x7f82d819f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.685+0000 7f82df676700 1 --2- 172.21.15.40:0/1891692933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d8108ae0 0x7f82d819f0e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f82c800e9f0 tx=0x7f82c800edb0 comp rx=0 tx=0).stop 2024-06-06T03:47:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.685+0000 7f82df676700 1 -- 172.21.15.40:0/1891692933 shutdown_connections 2024-06-06T03:47:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.685+0000 7f82df676700 1 --2- 172.21.15.40:0/1891692933 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f82c40a2f00 0x7f82c40a53c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.685+0000 7f82df676700 1 --2- 172.21.15.40:0/1891692933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d810eb70 0x7f82d81a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.685+0000 7f82df676700 1 --2- 172.21.15.40:0/1891692933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d8108ae0 0x7f82d819f0e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.685+0000 7f82df676700 1 --2- 172.21.15.40:0/1891692933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d81094a0 0x7f82d819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.685+0000 7f82df676700 1 -- 172.21.15.40:0/1891692933 >> 172.21.15.40:0/1891692933 conn(0x7f82d8075ca0 msgr2=0x7f82d810a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:47:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.685+0000 7f82df676700 1 -- 172.21.15.40:0/1891692933 shutdown_connections 2024-06-06T03:47:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:07.685+0000 7f82df676700 1 -- 172.21.15.40:0/1891692933 wait complete. 2024-06-06T03:47:08.021 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:07 smithi040 ceph-mon[32220]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T03:47:08.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:07 smithi131 ceph-mon[36687]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T03:47:08.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:07 smithi104 ceph-mon[36569]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T03:47:08.343 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9cacfcc8', quiesce took 19.3 seconds. Will release after: 6.300000000000001 2024-06-06T03:47:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:08 smithi131 ceph-mon[36687]: pgmap v263: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 835 KiB/s wr, 82 op/s 2024-06-06T03:47:09.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:08 smithi040 ceph-mon[32220]: pgmap v263: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 835 KiB/s wr, 82 op/s 2024-06-06T03:47:09.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:08 smithi104 ceph-mon[36569]: pgmap v263: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 835 KiB/s wr, 82 op/s 2024-06-06T03:47:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:10 smithi131 ceph-mon[36687]: pgmap v264: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 530 KiB/s wr, 115 op/s 2024-06-06T03:47:11.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:10 smithi040 ceph-mon[32220]: pgmap v264: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 530 KiB/s wr, 115 op/s 2024-06-06T03:47:11.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:10 smithi104 ceph-mon[36569]: pgmap v264: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 530 KiB/s wr, 115 op/s 2024-06-06T03:47:13.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:12 smithi131 ceph-mon[36687]: pgmap v265: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 517 KiB/s wr, 128 op/s 2024-06-06T03:47:13.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:12 smithi040 ceph-mon[32220]: pgmap v265: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 517 KiB/s wr, 128 op/s 2024-06-06T03:47:13.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:12 smithi104 ceph-mon[36569]: pgmap v265: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 517 KiB/s wr, 128 op/s 2024-06-06T03:47:14.644 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:47:14.913 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:47:15.035 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:14 smithi040 ceph-mon[32220]: pgmap v266: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 517 KiB/s wr, 128 op/s 2024-06-06T03:47:15.036 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:14 smithi040 ceph-mon[32220]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T03:47:15.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:14 smithi104 ceph-mon[36569]: pgmap v266: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 517 KiB/s wr, 128 op/s 2024-06-06T03:47:15.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:14 smithi104 ceph-mon[36569]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T03:47:15.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:14 smithi131 ceph-mon[36687]: pgmap v266: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 517 KiB/s wr, 128 op/s 2024-06-06T03:47:15.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:14 smithi131 ceph-mon[36687]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T03:47:16.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.667+0000 7fe948aa2700 1 -- 172.21.15.40:0/1522525517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe944101d50 msgr2=0x7fe94410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:16.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.667+0000 7fe948aa2700 1 --2- 172.21.15.40:0/1522525517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe944101d50 0x7fe94410e240 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00b0a0 tx=0x7fe93c00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:47:16.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.667+0000 7fe948aa2700 1 -- 172.21.15.40:0/1522525517 shutdown_connections 2024-06-06T03:47:16.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.667+0000 7fe948aa2700 1 --2- 172.21.15.40:0/1522525517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe944107400 0x7fe9441077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:16.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.667+0000 7fe948aa2700 1 --2- 172.21.15.40:0/1522525517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe944101d50 0x7fe94410e240 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:16.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.667+0000 7fe948aa2700 1 --2- 172.21.15.40:0/1522525517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe944101390 0x7fe944101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:16.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.667+0000 7fe948aa2700 1 -- 172.21.15.40:0/1522525517 >> 172.21.15.40:0/1522525517 conn(0x7fe944078340 msgr2=0x7fe944078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:47:16.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.667+0000 7fe948aa2700 1 -- 172.21.15.40:0/1522525517 shutdown_connections 2024-06-06T03:47:16.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.667+0000 7fe948aa2700 1 -- 172.21.15.40:0/1522525517 wait complete. 2024-06-06T03:47:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.668+0000 7fe948aa2700 1 Processor -- start 2024-06-06T03:47:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.668+0000 7fe948aa2700 1 -- start start 2024-06-06T03:47:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe948aa2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe944101390 0x7fe94419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe948aa2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe944101d50 0x7fe94419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe948aa2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe944107400 0x7fe94419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe948aa2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe944111670 con 0x7fe944107400 2024-06-06T03:47:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe948aa2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe9441114f0 con 0x7fe944101d50 2024-06-06T03:47:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe948aa2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe9441117f0 con 0x7fe944101390 2024-06-06T03:47:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe942d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe944107400 0x7fe94419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe942d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe944107400 0x7fe94419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57238/0 (socket says 172.21.15.40:57238) 2024-06-06T03:47:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe942d9d700 1 -- 172.21.15.40:0/3414748387 learned_addr learned my addr 172.21.15.40:0/3414748387 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:47:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe939d9b700 1 --2- 172.21.15.40:0/3414748387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe944101d50 0x7fe94419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe94259c700 1 --2- 172.21.15.40:0/3414748387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe944101390 0x7fe94419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe942d9d700 1 -- 172.21.15.40:0/3414748387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe944101390 msgr2=0x7fe94419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe942d9d700 1 --2- 172.21.15.40:0/3414748387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe944101390 0x7fe94419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe942d9d700 1 -- 172.21.15.40:0/3414748387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe944101d50 msgr2=0x7fe94419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe942d9d700 1 --2- 172.21.15.40:0/3414748387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe944101d50 0x7fe94419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe942d9d700 1 -- 172.21.15.40:0/3414748387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94419fda0 con 0x7fe944107400 2024-06-06T03:47:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe942d9d700 1 --2- 172.21.15.40:0/3414748387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe944107400 0x7fe94419f660 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0062a0 tx=0x7fe93c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:47:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe93bfff700 1 -- 172.21.15.40:0/3414748387 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c021070 con 0x7fe944107400 2024-06-06T03:47:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe948aa2700 1 -- 172.21.15.40:0/3414748387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9441a0030 con 0x7fe944107400 2024-06-06T03:47:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.669+0000 7fe93bfff700 1 -- 172.21.15.40:0/3414748387 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe93c007d00 con 0x7fe944107400 2024-06-06T03:47:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.670+0000 7fe93bfff700 1 -- 172.21.15.40:0/3414748387 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c005460 con 0x7fe944107400 2024-06-06T03:47:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.670+0000 7fe948aa2700 1 -- 172.21.15.40:0/3414748387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9441a7910 con 0x7fe944107400 2024-06-06T03:47:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.671+0000 7fe948aa2700 1 -- 172.21.15.40:0/3414748387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe944069180 con 0x7fe944107400 2024-06-06T03:47:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.672+0000 7fe93bfff700 1 -- 172.21.15.40:0/3414748387 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe93c024070 con 0x7fe944107400 2024-06-06T03:47:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.673+0000 7fe93bfff700 1 --2- 172.21.15.40:0/3414748387 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fe9300a2d90 0x7fe9300a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.673+0000 7fe94259c700 1 --2- 172.21.15.40:0/3414748387 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fe9300a2d90 0x7fe9300a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.673+0000 7fe93bfff700 1 -- 172.21.15.40:0/3414748387 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8426+0+0 (secure 0 0 0) 0x7fe93c0d06a0 con 0x7fe944107400 2024-06-06T03:47:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.674+0000 7fe94259c700 1 --2- 172.21.15.40:0/3414748387 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fe9300a2d90 0x7fe9300a5250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe934005fd0 tx=0x7fe934005db0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:47:16.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.677+0000 7fe93bfff700 1 -- 172.21.15.40:0/3414748387 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe93c09d350 con 0x7fe944107400 2024-06-06T03:47:16.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.932+0000 7fe948aa2700 1 -- 172.21.15.40:0/3414748387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe94419c0f0 con 0x7fe944107400 2024-06-06T03:47:16.933 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:16 smithi040 ceph-mon[32220]: pgmap v268: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 234 KiB/s wr, 93 op/s 2024-06-06T03:47:16.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.933+0000 7fe93bfff700 1 -- 172.21.15.40:0/3414748387 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11961 (secure 0 0 0) 0x7fe93c09cc50 con 0x7fe944107400 2024-06-06T03:47:16.934 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:47:16.934 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:46:58.889752+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24403,24398,24425,24470,24443]},"id":1}]} 2024-06-06T03:47:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.935+0000 7fe948aa2700 1 -- 172.21.15.40:0/3414748387 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fe9300a2d90 msgr2=0x7fe9300a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.935+0000 7fe948aa2700 1 --2- 172.21.15.40:0/3414748387 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fe9300a2d90 0x7fe9300a5250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe934005fd0 tx=0x7fe934005db0 comp rx=0 tx=0).stop 2024-06-06T03:47:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.935+0000 7fe948aa2700 1 -- 172.21.15.40:0/3414748387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe944107400 msgr2=0x7fe94419f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.935+0000 7fe948aa2700 1 --2- 172.21.15.40:0/3414748387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe944107400 0x7fe94419f660 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0062a0 tx=0x7fe93c00be60 comp rx=0 tx=0).stop 2024-06-06T03:47:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.935+0000 7fe948aa2700 1 -- 172.21.15.40:0/3414748387 shutdown_connections 2024-06-06T03:47:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.935+0000 7fe948aa2700 1 --2- 172.21.15.40:0/3414748387 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fe9300a2d90 0x7fe9300a5250 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.935+0000 7fe948aa2700 1 --2- 172.21.15.40:0/3414748387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe944101390 0x7fe94419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.935+0000 7fe948aa2700 1 --2- 172.21.15.40:0/3414748387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe944107400 0x7fe94419f660 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.935+0000 7fe948aa2700 1 --2- 172.21.15.40:0/3414748387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe944101d50 0x7fe94419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.935+0000 7fe948aa2700 1 -- 172.21.15.40:0/3414748387 >> 172.21.15.40:0/3414748387 conn(0x7fe944078340 msgr2=0x7fe94406a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:47:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.936+0000 7fe948aa2700 1 -- 172.21.15.40:0/3414748387 shutdown_connections 2024-06-06T03:47:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:16.936+0000 7fe948aa2700 1 -- 172.21.15.40:0/3414748387 wait complete. 2024-06-06T03:47:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 42 2024-06-06T03:47:17.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:16 smithi104 ceph-mon[36569]: pgmap v268: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 234 KiB/s wr, 93 op/s 2024-06-06T03:47:17.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:16 smithi131 ceph-mon[36687]: pgmap v268: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 234 KiB/s wr, 93 op/s 2024-06-06T03:47:17.617 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 9cacfcc8 --release --await' 2024-06-06T03:47:17.618 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 9cacfcc8 --release --await 2024-06-06T03:47:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:47:17.926 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:17 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3414748387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:47:18.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:17 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3414748387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:47:18.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:17 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3414748387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:47:19.033 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:18 smithi040 ceph-mon[32220]: pgmap v269: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 218 KiB/s wr, 85 op/s 2024-06-06T03:47:19.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:18 smithi104 ceph-mon[36569]: pgmap v269: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 218 KiB/s wr, 85 op/s 2024-06-06T03:47:19.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:18 smithi131 ceph-mon[36687]: pgmap v269: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 218 KiB/s wr, 85 op/s 2024-06-06T03:47:19.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.557+0000 7f978ad99700 1 -- 172.21.15.40:0/3060106376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9784101d40 msgr2=0x7f978410e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:19.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.557+0000 7f978ad99700 1 --2- 172.21.15.40:0/3060106376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9784101d40 0x7f978410e230 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f977c00b0a0 tx=0x7f977c00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.557+0000 7f978ad99700 1 -- 172.21.15.40:0/3060106376 shutdown_connections 2024-06-06T03:47:19.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.557+0000 7f978ad99700 1 --2- 172.21.15.40:0/3060106376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97841073f0 0x7f97841077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.557+0000 7f978ad99700 1 --2- 172.21.15.40:0/3060106376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9784101d40 0x7f978410e230 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.557+0000 7f978ad99700 1 --2- 172.21.15.40:0/3060106376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9784101380 0x7f9784101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.557+0000 7f978ad99700 1 -- 172.21.15.40:0/3060106376 >> 172.21.15.40:0/3060106376 conn(0x7f9784078300 msgr2=0x7f9784078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:47:19.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.557+0000 7f978ad99700 1 -- 172.21.15.40:0/3060106376 shutdown_connections 2024-06-06T03:47:19.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.557+0000 7f978ad99700 1 -- 172.21.15.40:0/3060106376 wait complete. 2024-06-06T03:47:19.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.558+0000 7f978ad99700 1 Processor -- start 2024-06-06T03:47:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.558+0000 7f978ad99700 1 -- start start 2024-06-06T03:47:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f978ad99700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9784101380 0x7f978419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f978ad99700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9784101d40 0x7f978419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f978ad99700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97841073f0 0x7f9784198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f978ad99700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f97841127e0 con 0x7f97841073f0 2024-06-06T03:47:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f978ad99700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9784112660 con 0x7f9784101380 2024-06-06T03:47:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f978ad99700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9784112960 con 0x7f9784101d40 2024-06-06T03:47:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f9789336700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97841073f0 0x7f9784198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f9789336700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97841073f0 0x7f9784198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32868/0 (socket says 172.21.15.40:32868) 2024-06-06T03:47:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f9789336700 1 -- 172.21.15.40:0/4030605519 learned_addr learned my addr 172.21.15.40:0/4030605519 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:47:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f9788b35700 1 --2- 172.21.15.40:0/4030605519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9784101380 0x7f978419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f9783fff700 1 --2- 172.21.15.40:0/4030605519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9784101d40 0x7f978419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f9789336700 1 -- 172.21.15.40:0/4030605519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9784101d40 msgr2=0x7f978419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f9789336700 1 --2- 172.21.15.40:0/4030605519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9784101d40 0x7f978419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f9789336700 1 -- 172.21.15.40:0/4030605519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9784101380 msgr2=0x7f978419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f9789336700 1 --2- 172.21.15.40:0/4030605519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9784101380 0x7f978419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f9789336700 1 -- 172.21.15.40:0/4030605519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97841991c0 con 0x7f97841073f0 2024-06-06T03:47:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f9789336700 1 --2- 172.21.15.40:0/4030605519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97841073f0 0x7f9784198a60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f977c0062a0 tx=0x7f977c004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:47:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f9781ffb700 1 -- 172.21.15.40:0/4030605519 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c021070 con 0x7f97841073f0 2024-06-06T03:47:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f9781ffb700 1 -- 172.21.15.40:0/4030605519 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f977c007c80 con 0x7f97841073f0 2024-06-06T03:47:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.559+0000 7f978ad99700 1 -- 172.21.15.40:0/4030605519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9784199450 con 0x7f97841073f0 2024-06-06T03:47:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.560+0000 7f9781ffb700 1 -- 172.21.15.40:0/4030605519 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c01d450 con 0x7f97841073f0 2024-06-06T03:47:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.560+0000 7f978ad99700 1 -- 172.21.15.40:0/4030605519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978410ebf0 con 0x7f97841073f0 2024-06-06T03:47:19.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.562+0000 7f9781ffb700 1 -- 172.21.15.40:0/4030605519 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f977c004020 con 0x7f97841073f0 2024-06-06T03:47:19.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.563+0000 7f9781ffb700 1 --2- 172.21.15.40:0/4030605519 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f976c0a2e90 0x7f976c0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.563+0000 7f9781ffb700 1 -- 172.21.15.40:0/4030605519 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8426+0+0 (secure 0 0 0) 0x7f977c00f020 con 0x7f97841073f0 2024-06-06T03:47:19.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.563+0000 7f9788b35700 1 --2- 172.21.15.40:0/4030605519 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f976c0a2e90 0x7f976c0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.563+0000 7f9788b35700 1 --2- 172.21.15.40:0/4030605519 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f976c0a2e90 0x7f976c0a5350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9784199ba0 tx=0x7f977400b5b0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:47:19.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.563+0000 7f979561cb80 1 Processor -- start 2024-06-06T03:47:19.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.563+0000 7f979561cb80 1 -- start start 2024-06-06T03:47:19.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f979561cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a9c40 0x55cfa28aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f979561cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28aa560 0x55cfa28a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f979561cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a43d0 0x55cfa28a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f979561cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55cfa27f5910 con 0x55cfa28a9c40 2024-06-06T03:47:19.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f979561cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cfa279d820 con 0x55cfa28a43d0 2024-06-06T03:47:19.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f979561cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cfa27f18d0 con 0x55cfa28aa560 2024-06-06T03:47:19.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f9788b35700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a9c40 0x55cfa28aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f9788b35700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a9c40 0x55cfa28aa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32872/0 (socket says 172.21.15.40:32872) 2024-06-06T03:47:19.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f9788b35700 1 -- 172.21.15.40:0/3589935465 learned_addr learned my addr 172.21.15.40:0/3589935465 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:47:19.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f9783fff700 1 --2- 172.21.15.40:0/3589935465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28aa560 0x55cfa28a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f9788b35700 1 -- 172.21.15.40:0/3589935465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28aa560 msgr2=0x55cfa28a3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f9789336700 1 --2- 172.21.15.40:0/3589935465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a43d0 0x55cfa28a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f9788b35700 1 --2- 172.21.15.40:0/3589935465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28aa560 0x55cfa28a3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f9788b35700 1 -- 172.21.15.40:0/3589935465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a43d0 msgr2=0x55cfa28a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f9788b35700 1 --2- 172.21.15.40:0/3589935465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a43d0 0x55cfa28a4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f9788b35700 1 -- 172.21.15.40:0/3589935465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfa2794050 con 0x55cfa28a9c40 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f9788b35700 1 --2- 172.21.15.40:0/3589935465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a9c40 0x55cfa28aa020 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f977400fcc0 tx=0x7f9774012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3c6f2bca5d600a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f976affd700 1 -- 172.21.15.40:0/3589935465 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774022070 con 0x55cfa28a9c40 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f976affd700 1 -- 172.21.15.40:0/3589935465 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9774011b80 con 0x55cfa28a9c40 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.564+0000 7f976affd700 1 -- 172.21.15.40:0/3589935465 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977401d680 con 0x55cfa28a9c40 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.565+0000 7f979561cb80 1 -- 172.21.15.40:0/3589935465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a9c40 msgr2=0x55cfa28aa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.565+0000 7f979561cb80 1 --2- 172.21.15.40:0/3589935465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a9c40 0x55cfa28aa020 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f977400fcc0 tx=0x7f9774012000 comp rx=0 tx=0).stop 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.565+0000 7f979561cb80 1 -- 172.21.15.40:0/3589935465 shutdown_connections 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.565+0000 7f979561cb80 1 --2- 172.21.15.40:0/3589935465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a43d0 0x55cfa28a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.565+0000 7f979561cb80 1 --2- 172.21.15.40:0/3589935465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a9c40 0x55cfa28aa020 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.565+0000 7f979561cb80 1 --2- 172.21.15.40:0/3589935465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28aa560 0x55cfa28a3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.565+0000 7f979561cb80 1 -- 172.21.15.40:0/3589935465 >> 172.21.15.40:0/3589935465 conn(0x55cfa27cedc0 msgr2=0x55cfa289f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:47:19.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.565+0000 7f979561cb80 1 -- 172.21.15.40:0/3589935465 shutdown_connections 2024-06-06T03:47:19.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.565+0000 7f979561cb80 1 -- 172.21.15.40:0/3589935465 wait complete. 2024-06-06T03:47:19.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.566+0000 7f979561cb80 1 -- start start 2024-06-06T03:47:19.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.566+0000 7f979561cb80 1 Processor -- start 2024-06-06T03:47:19.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.566+0000 7f979561cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a43d0 0x55cfa28b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.566+0000 7f979561cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a9c40 0x55cfa28ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.566+0000 7f979561cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28aa560 0x55cfa28adb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.566+0000 7f979561cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55cfa28b0fd0 con 0x55cfa28a43d0 2024-06-06T03:47:19.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.566+0000 7f979561cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cfa2795cf0 con 0x55cfa28a9c40 2024-06-06T03:47:19.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.566+0000 7f979561cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cfa28b1150 con 0x55cfa28aa560 2024-06-06T03:47:19.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.566+0000 7f979561cb80 10 client.0 _ll_register_callbacks cb 0x55cfa278fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:47:19.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.566+0000 7f9788b35700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a43d0 0x55cfa28b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.566+0000 7f9788b35700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a43d0 0x55cfa28b0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32876/0 (socket says 172.21.15.40:32876) 2024-06-06T03:47:19.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.566+0000 7f9788b35700 1 -- 172.21.15.40:0/241005485 learned_addr learned my addr 172.21.15.40:0/241005485 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:47:19.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.566+0000 7f9783fff700 1 --2- 172.21.15.40:0/241005485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a9c40 0x55cfa28ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.566+0000 7f9789336700 1 --2- 172.21.15.40:0/241005485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28aa560 0x55cfa28adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.567+0000 7f9788b35700 1 -- 172.21.15.40:0/241005485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28aa560 msgr2=0x55cfa28adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:47:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.567+0000 7f9788b35700 1 --2- 172.21.15.40:0/241005485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28aa560 0x55cfa28adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.567+0000 7f9788b35700 1 -- 172.21.15.40:0/241005485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a9c40 msgr2=0x55cfa28ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:47:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.567+0000 7f9788b35700 1 --2- 172.21.15.40:0/241005485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a9c40 0x55cfa28ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.567+0000 7f9788b35700 1 -- 172.21.15.40:0/241005485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfa27449d0 con 0x55cfa28a43d0 2024-06-06T03:47:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.567+0000 7f9788b35700 1 --2- 172.21.15.40:0/241005485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a43d0 0x55cfa28b0270 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f977400ba70 tx=0x7f9774011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1e94e5a5091e0f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:47:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.567+0000 7f9763fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:47:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.567+0000 7f9763fff700 1 -- 172.21.15.40:0/241005485 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774036060 con 0x55cfa28a43d0 2024-06-06T03:47:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.567+0000 7f9763fff700 1 -- 172.21.15.40:0/241005485 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8426+0+0 (secure 0 0 0) 0x7f9774005250 con 0x55cfa28a43d0 2024-06-06T03:47:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.567+0000 7f979561cb80 1 -- 172.21.15.40:0/241005485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9774011b80 con 0x55cfa28a43d0 2024-06-06T03:47:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.567+0000 7f9763fff700 1 -- 172.21.15.40:0/241005485 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9774022070 con 0x55cfa28a43d0 2024-06-06T03:47:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.567+0000 7f979561cb80 10 client.15750 fetch_fsmap learned FSMap version 42 2024-06-06T03:47:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.567+0000 7f979561cb80 1 -- 172.21.15.40:0/241005485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cfa28bb870 con 0x55cfa28a43d0 2024-06-06T03:47:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.567+0000 7f9763fff700 1 -- 172.21.15.40:0/241005485 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9774011b80 con 0x55cfa28a43d0 2024-06-06T03:47:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.568+0000 7f9763fff700 1 -- 172.21.15.40:0/241005485 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 42) ==== 8418+0+0 (secure 0 0 0) 0x7f977401e610 con 0x55cfa28a43d0 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.568+0000 7f979561cb80 10 client.15750 fetch_fsmap finished waiting for FSMap version 42 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.568+0000 7f979561cb80 10 client.15750 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.568+0000 7f979561cb80 20 client.15750 populate_metadata read hostname 'smithi040' 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.568+0000 7f979561cb80 1 --2- 172.21.15.40:0/241005485 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55cfa28bddf0 0x55cfa28c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.568+0000 7f979561cb80 4 client.15750 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.568+0000 7f979561cb80 1 -- 172.21.15.40:0/241005485 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfa2802e10 con 0x55cfa28bddf0 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.568+0000 7f9783fff700 1 --2- 172.21.15.40:0/241005485 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55cfa28bddf0 0x55cfa28c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.568+0000 7f9783fff700 1 --2- 172.21.15.40:0/241005485 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55cfa28bddf0 0x55cfa28c01e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5561c77ab758f8bc server_cookie=68e03a628eb2fbcb in_seq=0 out_seq=0 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.568+0000 7f9763fff700 10 client.15750 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.569+0000 7f9763fff700 1 -- 172.21.15.40:0/241005485 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cfa2802e10 con 0x55cfa28bddf0 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.569+0000 7f9763fff700 10 client.15750 handle_command_reply: tid=0 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.569+0000 7f979561cb80 1 client.15750 shutdown 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.569+0000 7f979561cb80 1 -- 172.21.15.40:0/241005485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a43d0 msgr2=0x55cfa28b0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.569+0000 7f979561cb80 1 --2- 172.21.15.40:0/241005485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a43d0 0x55cfa28b0270 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f977400ba70 tx=0x7f9774011980 comp rx=0 tx=0).stop 2024-06-06T03:47:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.569+0000 7f979561cb80 1 -- 172.21.15.40:0/241005485 shutdown_connections 2024-06-06T03:47:19.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.569+0000 7f979561cb80 1 --2- 172.21.15.40:0/241005485 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55cfa28bddf0 0x55cfa28c01e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.569+0000 7f979561cb80 1 --2- 172.21.15.40:0/241005485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28aa560 0x55cfa28adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.569+0000 7f979561cb80 1 --2- 172.21.15.40:0/241005485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a43d0 0x55cfa28b0270 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.569+0000 7f979561cb80 1 --2- 172.21.15.40:0/241005485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a9c40 0x55cfa28ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.569+0000 7f979561cb80 1 -- 172.21.15.40:0/241005485 >> 172.21.15.40:0/241005485 conn(0x55cfa27cedc0 msgr2=0x55cfa289ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:47:19.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.570+0000 7f9763fff700 0 client.15750 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:47:19.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.570+0000 7f979561cb80 1 -- 172.21.15.40:0/241005485 shutdown_connections 2024-06-06T03:47:19.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.570+0000 7f979561cb80 1 -- 172.21.15.40:0/241005485 wait complete. 2024-06-06T03:47:19.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.570+0000 7f979561cb80 20 client.15750 trim_cache size 0 max 16384 2024-06-06T03:47:19.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.590+0000 7f979561cb80 1 Processor -- start 2024-06-06T03:47:19.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.590+0000 7f979561cb80 1 -- start start 2024-06-06T03:47:19.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.590+0000 7f979561cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a43d0 0x55cfa28a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.590+0000 7f979561cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28a9c40 0x55cfa28ab050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.590+0000 7f979561cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28aa560 0x55cfa28c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.590+0000 7f979561cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f97841aeee0 con 0x55cfa28a43d0 2024-06-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.590+0000 7f979561cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f97841b0c60 con 0x55cfa28aa560 2024-06-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.590+0000 7f979561cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f97841af670 con 0x55cfa28a9c40 2024-06-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.590+0000 7f9788b35700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a43d0 0x55cfa28a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.590+0000 7f9788b35700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a43d0 0x55cfa28a3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32880/0 (socket says 172.21.15.40:32880) 2024-06-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.590+0000 7f9788b35700 1 -- 172.21.15.40:0/2837526118 learned_addr learned my addr 172.21.15.40:0/2837526118 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.591+0000 7f9783fff700 1 --2- 172.21.15.40:0/2837526118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28a9c40 0x55cfa28ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.591+0000 7f9789336700 1 --2- 172.21.15.40:0/2837526118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28aa560 0x55cfa28c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.591+0000 7f9788b35700 1 -- 172.21.15.40:0/2837526118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28a9c40 msgr2=0x55cfa28ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.591+0000 7f9788b35700 1 --2- 172.21.15.40:0/2837526118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28a9c40 0x55cfa28ab050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.591+0000 7f9788b35700 1 -- 172.21.15.40:0/2837526118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28aa560 msgr2=0x55cfa28c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:47:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.591+0000 7f9788b35700 1 --2- 172.21.15.40:0/2837526118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28aa560 0x55cfa28c57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.591+0000 7f9788b35700 1 -- 172.21.15.40:0/2837526118 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfa28ab590 con 0x55cfa28a43d0 2024-06-06T03:47:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.591+0000 7f9788b35700 1 --2- 172.21.15.40:0/2837526118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a43d0 0x55cfa28a3e90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9774000c00 tx=0x7f9774005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e4926112faa14f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:47:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.591+0000 7f976a7fc700 1 -- 172.21.15.40:0/2837526118 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774036030 con 0x55cfa28a43d0 2024-06-06T03:47:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.591+0000 7f976a7fc700 1 -- 172.21.15.40:0/2837526118 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f977401ebb0 con 0x55cfa28a43d0 2024-06-06T03:47:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.591+0000 7f976a7fc700 1 -- 172.21.15.40:0/2837526118 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774015040 con 0x55cfa28a43d0 2024-06-06T03:47:19.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.592+0000 7f979561cb80 1 -- 172.21.15.40:0/2837526118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a43d0 msgr2=0x55cfa28a3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:19.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.592+0000 7f979561cb80 1 --2- 172.21.15.40:0/2837526118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a43d0 0x55cfa28a3e90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9774000c00 tx=0x7f9774005d20 comp rx=0 tx=0).stop 2024-06-06T03:47:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.592+0000 7f979561cb80 1 -- 172.21.15.40:0/2837526118 shutdown_connections 2024-06-06T03:47:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.592+0000 7f979561cb80 1 --2- 172.21.15.40:0/2837526118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28aa560 0x55cfa28c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.592+0000 7f979561cb80 1 --2- 172.21.15.40:0/2837526118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28a43d0 0x55cfa28a3e90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.592+0000 7f979561cb80 1 --2- 172.21.15.40:0/2837526118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28a9c40 0x55cfa28ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.592+0000 7f979561cb80 1 -- 172.21.15.40:0/2837526118 >> 172.21.15.40:0/2837526118 conn(0x55cfa27cedc0 msgr2=0x55cfa28aa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:47:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.592+0000 7f979561cb80 1 -- 172.21.15.40:0/2837526118 shutdown_connections 2024-06-06T03:47:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.592+0000 7f979561cb80 1 -- 172.21.15.40:0/2837526118 wait complete. 2024-06-06T03:47:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.592+0000 7f979561cb80 1 -- start start 2024-06-06T03:47:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.592+0000 7f979561cb80 1 Processor -- start 2024-06-06T03:47:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f979561cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a43d0 0x55cfa28b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f979561cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28a9c40 0x55cfa28b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f979561cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28aa560 0x55cfa28b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f979561cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f97841b04d0 con 0x55cfa28aa560 2024-06-06T03:47:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f979561cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f97841afd40 con 0x55cfa28a43d0 2024-06-06T03:47:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f979561cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f97841acf00 con 0x55cfa28a9c40 2024-06-06T03:47:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f979561cb80 10 client.0 _ll_register_callbacks cb 0x55cfa278fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:47:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f9789336700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28aa560 0x55cfa28b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f9789336700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28aa560 0x55cfa28b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32882/0 (socket says 172.21.15.40:32882) 2024-06-06T03:47:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f9789336700 1 -- 172.21.15.40:0/2254293823 learned_addr learned my addr 172.21.15.40:0/2254293823 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:47:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f9788b35700 1 --2- 172.21.15.40:0/2254293823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a43d0 0x55cfa28b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f9783fff700 1 --2- 172.21.15.40:0/2254293823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28a9c40 0x55cfa28b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f9789336700 1 -- 172.21.15.40:0/2254293823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28a9c40 msgr2=0x55cfa28b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:47:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f9789336700 1 --2- 172.21.15.40:0/2254293823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28a9c40 0x55cfa28b6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f9789336700 1 -- 172.21.15.40:0/2254293823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a43d0 msgr2=0x55cfa28b67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:47:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f9789336700 1 --2- 172.21.15.40:0/2254293823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a43d0 0x55cfa28b67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f9789336700 1 -- 172.21.15.40:0/2254293823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfa28b3dd0 con 0x55cfa28aa560 2024-06-06T03:47:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.593+0000 7f9783fff700 1 --2- 172.21.15.40:0/2254293823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28a9c40 0x55cfa28b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:47:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f9789336700 1 --2- 172.21.15.40:0/2254293823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28aa560 0x55cfa28b3590 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f977c0999a0 tx=0x7f977c09cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2ea9c39c5f7dea server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:47:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f9763fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:47:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f9763fff700 1 -- 172.21.15.40:0/2254293823 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c09d6d0 con 0x55cfa28aa560 2024-06-06T03:47:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f979561cb80 1 -- 172.21.15.40:0/2254293823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f977401ebb0 con 0x55cfa28aa560 2024-06-06T03:47:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f9763fff700 1 -- 172.21.15.40:0/2254293823 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8426+0+0 (secure 0 0 0) 0x7f977c09d230 con 0x55cfa28aa560 2024-06-06T03:47:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f9763fff700 1 -- 172.21.15.40:0/2254293823 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f977c09cfd0 con 0x55cfa28aa560 2024-06-06T03:47:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f979561cb80 10 client.15756 fetch_fsmap learned FSMap version 42 2024-06-06T03:47:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f979561cb80 1 -- 172.21.15.40:0/2254293823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cfa2929510 con 0x55cfa28aa560 2024-06-06T03:47:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f9763fff700 1 -- 172.21.15.40:0/2254293823 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f977401ebb0 con 0x55cfa28aa560 2024-06-06T03:47:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f9763fff700 1 -- 172.21.15.40:0/2254293823 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 42) ==== 8418+0+0 (secure 0 0 0) 0x7f977c011070 con 0x55cfa28aa560 2024-06-06T03:47:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f979561cb80 10 client.15756 fetch_fsmap finished waiting for FSMap version 42 2024-06-06T03:47:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f979561cb80 10 client.15756 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:47:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f979561cb80 20 client.15756 populate_metadata read hostname 'smithi040' 2024-06-06T03:47:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f979561cb80 1 --2- 172.21.15.40:0/2254293823 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55cfa2929cb0 0x55cfa292c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:47:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f979561cb80 4 client.15756 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9cacfcc8", "release": true, "await": true}] 2024-06-06T03:47:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.594+0000 7f979561cb80 1 -- 172.21.15.40:0/2254293823 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9cacfcc8", "release": true, "await": true}) -- 0x55cfa27fd390 con 0x55cfa2929cb0 2024-06-06T03:47:19.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.595+0000 7f9788b35700 1 --2- 172.21.15.40:0/2254293823 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55cfa2929cb0 0x55cfa292c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:47:19.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.595+0000 7f9763fff700 10 client.15756 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:47:19.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:19.595+0000 7f9788b35700 1 --2- 172.21.15.40:0/2254293823 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55cfa2929cb0 0x55cfa292c0d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=140937c882360fa9 server_cookie=44d0b57f3135c0e5 in_seq=0 out_seq=0 2024-06-06T03:47:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:20 smithi040 ceph-mon[32220]: pgmap v270: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 50 KiB/s wr, 33 op/s 2024-06-06T03:47:21.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:20 smithi104 ceph-mon[36569]: pgmap v270: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 50 KiB/s wr, 33 op/s 2024-06-06T03:47:21.091 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/intel/lbr.o 2024-06-06T03:47:21.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:20 smithi131 ceph-mon[36687]: pgmap v270: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 50 KiB/s wr, 33 op/s 2024-06-06T03:47:21.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.317+0000 7f9763fff700 1 -- 172.21.15.40:0/2254293823 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cfa27fd390 con 0x55cfa2929cb0 2024-06-06T03:47:21.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.317+0000 7f9763fff700 10 client.15756 handle_command_reply: tid=0 2024-06-06T03:47:21.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.317+0000 7f979561cb80 1 client.15756 shutdown 2024-06-06T03:47:21.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.317+0000 7f979561cb80 1 -- 172.21.15.40:0/2254293823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28aa560 msgr2=0x55cfa28b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.317+0000 7f979561cb80 1 --2- 172.21.15.40:0/2254293823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28aa560 0x55cfa28b3590 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f977c0999a0 tx=0x7f977c09cbe0 comp rx=0 tx=0).stop 2024-06-06T03:47:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.317+0000 7f979561cb80 1 -- 172.21.15.40:0/2254293823 shutdown_connections 2024-06-06T03:47:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.317+0000 7f979561cb80 1 --2- 172.21.15.40:0/2254293823 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x55cfa2929cb0 0x55cfa292c0d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.317+0000 7f979561cb80 1 --2- 172.21.15.40:0/2254293823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cfa28aa560 0x55cfa28b3590 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.317+0000 7f9763fff700 0 client.15756 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:47:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.317+0000 7f979561cb80 1 --2- 172.21.15.40:0/2254293823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfa28a43d0 0x55cfa28b67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.317+0000 7f979561cb80 1 --2- 172.21.15.40:0/2254293823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cfa28a9c40 0x55cfa28b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.317+0000 7f979561cb80 1 -- 172.21.15.40:0/2254293823 >> 172.21.15.40:0/2254293823 conn(0x55cfa27cedc0 msgr2=0x55cfa28a86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:47:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.318+0000 7f979561cb80 1 -- 172.21.15.40:0/2254293823 shutdown_connections 2024-06-06T03:47:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.318+0000 7f979561cb80 1 -- 172.21.15.40:0/2254293823 wait complete. 2024-06-06T03:47:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.318+0000 7f979561cb80 20 client.15756 trim_cache size 0 max 16384 2024-06-06T03:47:21.320 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:47:21.320 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 42, 2024-06-06T03:47:21.320 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:47:21.320 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 28, 2024-06-06T03:47:21.320 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": { 2024-06-06T03:47:21.320 INFO:teuthology.orchestra.run.smithi040.stdout: "9cacfcc8": { 2024-06-06T03:47:21.320 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 28, 2024-06-06T03:47:21.320 INFO:teuthology.orchestra.run.smithi040.stdout: "age_ref": 0.0, 2024-06-06T03:47:21.320 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:47:21.321 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RELEASED", 2024-06-06T03:47:21.321 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:47:21.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:47:21.321 INFO:teuthology.orchestra.run.smithi040.stdout: "timeout": 90.0, 2024-06-06T03:47:21.321 INFO:teuthology.orchestra.run.smithi040.stdout: "expiration": 145.6, 2024-06-06T03:47:21.321 INFO:teuthology.orchestra.run.smithi040.stdout: "members": { 2024-06-06T03:47:21.321 INFO:teuthology.orchestra.run.smithi040.stdout: "file:/": { 2024-06-06T03:47:21.321 INFO:teuthology.orchestra.run.smithi040.stdout: "excluded": false, 2024-06-06T03:47:21.321 INFO:teuthology.orchestra.run.smithi040.stdout: "state": { 2024-06-06T03:47:21.321 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RELEASED", 2024-06-06T03:47:21.321 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0 2024-06-06T03:47:21.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:47:21.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:47:21.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:47:21.322 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:47:21.322 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:47:21.322 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:47:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.318+0000 7f978ad99700 1 -- 172.21.15.40:0/4030605519 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f976c0a2e90 msgr2=0x7f976c0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.318+0000 7f978ad99700 1 --2- 172.21.15.40:0/4030605519 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f976c0a2e90 0x7f976c0a5350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9784199ba0 tx=0x7f977400b5b0 comp rx=0 tx=0).stop 2024-06-06T03:47:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.318+0000 7f978ad99700 1 -- 172.21.15.40:0/4030605519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97841073f0 msgr2=0x7f9784198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:47:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.318+0000 7f978ad99700 1 --2- 172.21.15.40:0/4030605519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97841073f0 0x7f9784198a60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f977c0062a0 tx=0x7f977c004e20 comp rx=0 tx=0).stop 2024-06-06T03:47:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.319+0000 7f978ad99700 1 -- 172.21.15.40:0/4030605519 shutdown_connections 2024-06-06T03:47:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.319+0000 7f978ad99700 1 --2- 172.21.15.40:0/4030605519 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f976c0a2e90 0x7f976c0a5350 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.319+0000 7f978ad99700 1 --2- 172.21.15.40:0/4030605519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97841073f0 0x7f9784198a60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.319+0000 7f978ad99700 1 --2- 172.21.15.40:0/4030605519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9784101380 0x7f978419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.319+0000 7f978ad99700 1 --2- 172.21.15.40:0/4030605519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9784101d40 0x7f978419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:47:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.319+0000 7f978ad99700 1 -- 172.21.15.40:0/4030605519 >> 172.21.15.40:0/4030605519 conn(0x7f9784078300 msgr2=0x7f97841000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:47:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.319+0000 7f978ad99700 1 -- 172.21.15.40:0/4030605519 shutdown_connections 2024-06-06T03:47:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:47:21.319+0000 7f978ad99700 1 -- 172.21.15.40:0/4030605519 wait complete. 2024-06-06T03:47:21.333 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/array.o 2024-06-06T03:47:21.342 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/fd.o 2024-06-06T03:47:21.353 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/locking/rtmutex_api.o 2024-06-06T03:47:21.380 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/quota/built-in.a 2024-06-06T03:47:21.399 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/intel/p4.o 2024-06-06T03:47:21.408 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/locking/qrwlock.o 2024-06-06T03:47:21.476 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/selinuxfs.o 2024-06-06T03:47:21.689 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/backing-dev.o 2024-06-06T03:47:21.886 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/mm_init.o 2024-06-06T03:47:22.089 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9cacfcc8', total seconds elapsed: 33.0 2024-06-06T03:47:22.120 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/proc_tty.o 2024-06-06T03:47:22.239 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/cmdline.o 2024-06-06T03:47:22.255 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/locking/built-in.a 2024-06-06T03:47:22.274 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/intel/p6.o 2024-06-06T03:47:22.319 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/power/qos.o 2024-06-06T03:47:22.376 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/printk/printk.o 2024-06-06T03:47:22.600 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/percpu.o 2024-06-06T03:47:22.684 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/consoles.o 2024-06-06T03:47:22.718 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/cpuinfo.o 2024-06-06T03:47:22.902 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/intel/pt.o 2024-06-06T03:47:23.027 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/slab_common.o 2024-06-06T03:47:23.042 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/netlink.o 2024-06-06T03:47:23.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:22 smithi040 ceph-mon[32220]: pgmap v271: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-06-06T03:47:23.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:22 smithi104 ceph-mon[36569]: pgmap v271: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-06-06T03:47:23.154 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/power/main.o 2024-06-06T03:47:23.185 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/devices.o 2024-06-06T03:47:23.219 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/interrupts.o 2024-06-06T03:47:23.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:22 smithi131 ceph-mon[36687]: pgmap v271: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-06-06T03:47:23.683 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/nlmsgtab.o 2024-06-06T03:47:23.691 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/loadavg.o 2024-06-06T03:47:23.744 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/meminfo.o 2024-06-06T03:47:23.965 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/printk/printk_safe.o 2024-06-06T03:47:23.969 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/intel/uncore.o 2024-06-06T03:47:24.094 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/power/console.o 2024-06-06T03:47:24.194 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/stat.o 2024-06-06T03:47:24.399 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/uptime.o 2024-06-06T03:47:24.401 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/compaction.o 2024-06-06T03:47:24.460 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/show_mem.o 2024-06-06T03:47:24.502 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/printk/printk_ringbuffer.o 2024-06-06T03:47:24.508 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/netif.o 2024-06-06T03:47:24.725 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/power/process.o 2024-06-06T03:47:24.747 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/util.o 2024-06-06T03:47:24.875 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/version.o 2024-06-06T03:47:25.021 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/printk/sysctl.o 2024-06-06T03:47:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:24 smithi040 ceph-mon[32220]: pgmap v272: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-06-06T03:47:25.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:24 smithi104 ceph-mon[36569]: pgmap v272: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-06-06T03:47:25.191 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/softirqs.o 2024-06-06T03:47:25.255 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/intel/uncore_nhmex.o 2024-06-06T03:47:25.268 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/interval_tree.o 2024-06-06T03:47:25.279 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/printk/built-in.a 2024-06-06T03:47:25.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:24 smithi131 ceph-mon[36687]: pgmap v272: 97 pgs: 97 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-06-06T03:47:25.329 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/namespaces.o 2024-06-06T03:47:25.333 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/list_lru.o 2024-06-06T03:47:25.446 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/netnode.o 2024-06-06T03:47:25.566 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/power/suspend.o 2024-06-06T03:47:25.668 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/self.o 2024-06-06T03:47:25.987 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/workingset.o 2024-06-06T03:47:25.994 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/thread_self.o 2024-06-06T03:47:25.997 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/debug.o 2024-06-06T03:47:26.167 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/proc_sysctl.o 2024-06-06T03:47:26.176 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/intel/uncore_snb.o 2024-06-06T03:47:26.419 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/netport.o 2024-06-06T03:47:26.467 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/power/hibernate.o 2024-06-06T03:47:26.468 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/intel/uncore_snbep.o 2024-06-06T03:47:26.506 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/proc_net.o 2024-06-06T03:47:26.700 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/gup.o 2024-06-06T03:47:26.702 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/mmap_lock.o 2024-06-06T03:47:26.986 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/highmem.o 2024-06-06T03:47:27.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:26 smithi040 ceph-mon[32220]: pgmap v273: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 214 KiB/s wr, 45 op/s 2024-06-06T03:47:27.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:26 smithi104 ceph-mon[36569]: pgmap v273: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 214 KiB/s wr, 45 op/s 2024-06-06T03:47:27.221 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/kcore.o 2024-06-06T03:47:27.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:26 smithi131 ceph-mon[36687]: pgmap v273: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 214 KiB/s wr, 45 op/s 2024-06-06T03:47:27.342 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/status.o 2024-06-06T03:47:27.427 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/memory.o 2024-06-06T03:47:27.467 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/vmcore.o 2024-06-06T03:47:27.485 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/power/snapshot.o 2024-06-06T03:47:27.570 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/mincore.o 2024-06-06T03:47:28.026 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/kmsg.o 2024-06-06T03:47:28.213 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/intel/uncore_discovery.o 2024-06-06T03:47:28.235 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/ss/ebitmap.o 2024-06-06T03:47:28.310 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/mlock.o 2024-06-06T03:47:28.340 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/mmap.o 2024-06-06T03:47:28.462 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/mmu_gather.o 2024-06-06T03:47:28.496 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc/page.o 2024-06-06T03:47:28.796 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/power/swap.o 2024-06-06T03:47:28.817 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:28 smithi040 ceph-mon[32220]: pgmap v274: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 346 KiB/s wr, 56 op/s 2024-06-06T03:47:29.068 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/intel/cstate.o 2024-06-06T03:47:29.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:28 smithi104 ceph-mon[36569]: pgmap v274: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 346 KiB/s wr, 56 op/s 2024-06-06T03:47:29.192 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/mprotect.o 2024-06-06T03:47:29.281 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/ss/hashtab.o 2024-06-06T03:47:29.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:28 smithi131 ceph-mon[36687]: pgmap v274: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 346 KiB/s wr, 56 op/s 2024-06-06T03:47:29.295 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/proc/built-in.a 2024-06-06T03:47:29.351 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/sysfs/file.o 2024-06-06T03:47:29.365 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/devpts/inode.o 2024-06-06T03:47:29.677 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/ss/symtab.o 2024-06-06T03:47:29.772 INFO:tasks.workunit.client.0.smithi040.stdout: AR arch/x86/events/intel/built-in.a 2024-06-06T03:47:29.827 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/zhaoxin/core.o 2024-06-06T03:47:29.860 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/power/user.o 2024-06-06T03:47:29.944 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/ss/sidtab.o 2024-06-06T03:47:30.019 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/sysfs/dir.o 2024-06-06T03:47:30.062 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/devpts/built-in.a 2024-06-06T03:47:30.095 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/sysfs/symlink.o 2024-06-06T03:47:30.256 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/mremap.o 2024-06-06T03:47:30.260 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/sysfs/mount.o 2024-06-06T03:47:30.305 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/msync.o 2024-06-06T03:47:30.479 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/sysfs/group.o 2024-06-06T03:47:30.546 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/power/poweroff.o 2024-06-06T03:47:30.555 INFO:tasks.workunit.client.0.smithi040.stdout: AR arch/x86/events/zhaoxin/built-in.a 2024-06-06T03:47:30.583 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/core.o 2024-06-06T03:47:30.625 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/probe.o 2024-06-06T03:47:30.755 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/power/built-in.a 2024-06-06T03:47:30.815 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/irqdesc.o 2024-06-06T03:47:30.891 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/handle.o 2024-06-06T03:47:30.911 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/manage.o 2024-06-06T03:47:30.957 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/spurious.o 2024-06-06T03:47:30.993 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/ss/avtab.o 2024-06-06T03:47:31.039 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/sysfs/built-in.a 2024-06-06T03:47:31.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:30 smithi040 ceph-mon[32220]: pgmap v275: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 354 KiB/s wr, 69 op/s 2024-06-06T03:47:31.086 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/netfs/buffered_read.o 2024-06-06T03:47:31.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:30 smithi104 ceph-mon[36569]: pgmap v275: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 354 KiB/s wr, 69 op/s 2024-06-06T03:47:31.257 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/page_vma_mapped.o 2024-06-06T03:47:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:30 smithi131 ceph-mon[36687]: pgmap v275: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 354 KiB/s wr, 69 op/s 2024-06-06T03:47:31.360 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/pagewalk.o 2024-06-06T03:47:31.440 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/resend.o 2024-06-06T03:47:31.497 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/chip.o 2024-06-06T03:47:31.863 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/pgtable-generic.o 2024-06-06T03:47:31.870 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/netfs/io.o 2024-06-06T03:47:31.996 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/netfs/iterator.o 2024-06-06T03:47:32.007 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/ss/policydb.o 2024-06-06T03:47:32.056 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/netfs/main.o 2024-06-06T03:47:32.195 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/netfs/objects.o 2024-06-06T03:47:32.201 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/utils.o 2024-06-06T03:47:32.490 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/dummychip.o 2024-06-06T03:47:32.715 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/rmap.o 2024-06-06T03:47:32.798 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/vmalloc.o 2024-06-06T03:47:32.913 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/process_vm_access.o 2024-06-06T03:47:32.925 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/rapl.o 2024-06-06T03:47:32.926 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/devres.o 2024-06-06T03:47:33.068 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/events/msr.o 2024-06-06T03:47:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:32 smithi040 ceph-mon[32220]: pgmap v276: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 445 KiB/s wr, 70 op/s 2024-06-06T03:47:33.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:32 smithi104 ceph-mon[36569]: pgmap v276: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 445 KiB/s wr, 70 op/s 2024-06-06T03:47:33.156 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/netfs/built-in.a 2024-06-06T03:47:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:32 smithi131 ceph-mon[36687]: pgmap v276: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 445 KiB/s wr, 70 op/s 2024-06-06T03:47:33.387 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/balloc.o 2024-06-06T03:47:33.458 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/autoprobe.o 2024-06-06T03:47:33.749 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/jbd2/transaction.o 2024-06-06T03:47:33.756 INFO:tasks.workunit.client.0.smithi040.stdout: AR arch/x86/events/built-in.a 2024-06-06T03:47:33.772 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/irqdomain.o 2024-06-06T03:47:33.953 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/fpu/init.o 2024-06-06T03:47:34.063 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/proc.o 2024-06-06T03:47:34.271 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/ss/services.o 2024-06-06T03:47:34.272 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/ss/conditional.o 2024-06-06T03:47:34.364 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/fpu/bugs.o 2024-06-06T03:47:34.523 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/fpu/core.o 2024-06-06T03:47:34.745 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/migration.o 2024-06-06T03:47:34.764 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/cpuhotplug.o 2024-06-06T03:47:34.822 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/bitmap.o 2024-06-06T03:47:35.054 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/jbd2/commit.o 2024-06-06T03:47:35.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:34 smithi040 ceph-mon[32220]: pgmap v277: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 439 KiB/s wr, 68 op/s 2024-06-06T03:47:35.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:34 smithi131 ceph-mon[36687]: pgmap v277: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 439 KiB/s wr, 68 op/s 2024-06-06T03:47:35.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:34 smithi104 ceph-mon[36569]: pgmap v277: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 439 KiB/s wr, 68 op/s 2024-06-06T03:47:35.182 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/block_validity.o 2024-06-06T03:47:35.243 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/pm.o 2024-06-06T03:47:35.286 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/ss/mls.o 2024-06-06T03:47:35.408 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/page_alloc.o 2024-06-06T03:47:35.476 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/dir.o 2024-06-06T03:47:35.552 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/fpu/regset.o 2024-06-06T03:47:35.944 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/msi.o 2024-06-06T03:47:35.988 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/rcu/update.o 2024-06-06T03:47:36.126 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/jbd2/recovery.o 2024-06-06T03:47:36.256 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/fpu/signal.o 2024-06-06T03:47:36.334 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/ext4_jbd2.o 2024-06-06T03:47:36.335 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/init-mm.o 2024-06-06T03:47:36.337 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/ss/context.o 2024-06-06T03:47:36.888 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/selinux/netlabel.o 2024-06-06T03:47:36.934 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/affinity.o 2024-06-06T03:47:36.985 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/fpu/xstate.o 2024-06-06T03:47:37.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:36 smithi040 ceph-mon[32220]: pgmap v278: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-06-06T03:47:37.083 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/jbd2/checkpoint.o 2024-06-06T03:47:37.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:36 smithi104 ceph-mon[36569]: pgmap v278: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-06-06T03:47:37.125 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/jbd2/revoke.o 2024-06-06T03:47:37.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:36 smithi131 ceph-mon[36687]: pgmap v278: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-06-06T03:47:37.366 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq/matrix.o 2024-06-06T03:47:37.460 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/extents.o 2024-06-06T03:47:37.633 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/rcu/sync.o 2024-06-06T03:47:37.877 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/rcu/srcutree.o 2024-06-06T03:47:37.910 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/rcu/tree.o 2024-06-06T03:47:37.914 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/jbd2/journal.o 2024-06-06T03:47:37.992 INFO:tasks.workunit.client.0.smithi040.stdout: AR security/selinux/built-in.a 2024-06-06T03:47:37.994 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/memblock.o 2024-06-06T03:47:38.072 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/integrity/iint.o 2024-06-06T03:47:38.081 INFO:tasks.workunit.client.0.smithi040.stdout: AR arch/x86/kernel/fpu/built-in.a 2024-06-06T03:47:38.184 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/mce/core.o 2024-06-06T03:47:38.385 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/irq/built-in.a 2024-06-06T03:47:38.417 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/mce/severity.o 2024-06-06T03:47:38.608 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/integrity/integrity_audit.o 2024-06-06T03:47:38.840 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/rcu/rcu_segcblist.o 2024-06-06T03:47:39.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:38 smithi104 ceph-mon[36569]: pgmap v279: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-06-06T03:47:39.110 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/madvise.o 2024-06-06T03:47:39.114 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/livepatch/built-in.a 2024-06-06T03:47:39.130 INFO:tasks.workunit.client.0.smithi040.stdout: AR security/integrity/built-in.a 2024-06-06T03:47:39.133 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/page_io.o 2024-06-06T03:47:39.165 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/commoncap.o 2024-06-06T03:47:39.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:38 smithi131 ceph-mon[36687]: pgmap v279: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-06-06T03:47:39.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:38 smithi040 ceph-mon[32220]: pgmap v279: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-06-06T03:47:39.345 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/min_addr.o 2024-06-06T03:47:39.586 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/mce/genpool.o 2024-06-06T03:47:39.835 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/security.o 2024-06-06T03:47:39.847 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/jbd2/built-in.a 2024-06-06T03:47:39.872 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/swap_state.o 2024-06-06T03:47:39.998 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/swapfile.o 2024-06-06T03:47:40.141 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/mce/intel.o 2024-06-06T03:47:40.150 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ramfs/inode.o 2024-06-06T03:47:40.265 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/swap_slots.o 2024-06-06T03:47:40.528 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/extents_status.o 2024-06-06T03:47:40.710 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/mce/amd.o 2024-06-06T03:47:40.760 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ramfs/file-mmu.o 2024-06-06T03:47:40.837 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/dmapool.o 2024-06-06T03:47:40.905 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/hugetlb.o 2024-06-06T03:47:40.933 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/rcu/built-in.a 2024-06-06T03:47:40.989 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/dma/mapping.o 2024-06-06T03:47:41.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:40 smithi104 ceph-mon[36569]: pgmap v280: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.2 MiB/s wr, 100 op/s 2024-06-06T03:47:41.271 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/ramfs/built-in.a 2024-06-06T03:47:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:40 smithi131 ceph-mon[36687]: pgmap v280: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.2 MiB/s wr, 100 op/s 2024-06-06T03:47:41.319 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/dma/direct.o 2024-06-06T03:47:41.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:40 smithi040 ceph-mon[32220]: pgmap v280: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.2 MiB/s wr, 100 op/s 2024-06-06T03:47:41.470 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/entry/common.o 2024-06-06T03:47:41.843 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/module/main.o 2024-06-06T03:47:41.860 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/mce/threshold.o 2024-06-06T03:47:41.865 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/module/strict_rwx.o 2024-06-06T03:47:41.980 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/lsm_audit.o 2024-06-06T03:47:42.088 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/dma/ops_helpers.o 2024-06-06T03:47:42.109 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/file.o 2024-06-06T03:47:42.326 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/entry/syscall_user_dispatch.o 2024-06-06T03:47:42.394 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/module/kmod.o 2024-06-06T03:47:42.600 INFO:tasks.workunit.client.0.smithi040.stdout: AR arch/x86/kernel/cpu/mce/built-in.a 2024-06-06T03:47:42.649 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/dma/dummy.o 2024-06-06T03:47:42.651 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/mtrr/mtrr.o 2024-06-06T03:47:42.861 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/entry/built-in.a 2024-06-06T03:47:42.914 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/time.o 2024-06-06T03:47:42.949 INFO:tasks.workunit.client.0.smithi040.stdout: CC security/device_cgroup.o 2024-06-06T03:47:42.962 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/fsmap.o 2024-06-06T03:47:43.053 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/fsync.o 2024-06-06T03:47:43.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:42 smithi104 ceph-mon[36569]: pgmap v281: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.2 MiB/s wr, 91 op/s 2024-06-06T03:47:43.177 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/dma/swiotlb.o 2024-06-06T03:47:43.249 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:42 smithi040 ceph-mon[32220]: pgmap v281: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.2 MiB/s wr, 91 op/s 2024-06-06T03:47:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:42 smithi131 ceph-mon[36687]: pgmap v281: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.2 MiB/s wr, 91 op/s 2024-06-06T03:47:43.393 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/mtrr/if.o 2024-06-06T03:47:43.574 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/module/tree_lookup.o 2024-06-06T03:47:43.795 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/timer.o 2024-06-06T03:47:43.956 INFO:tasks.workunit.client.0.smithi040.stdout: AR security/built-in.a 2024-06-06T03:47:43.960 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/hugetlb_vmemmap.o 2024-06-06T03:47:43.986 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/mtrr/generic.o 2024-06-06T03:47:44.099 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/hrtimer.o 2024-06-06T03:47:44.113 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/module/kallsyms.o 2024-06-06T03:47:44.205 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/dma/remap.o 2024-06-06T03:47:44.228 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/asymmetric_keys/asymmetric_type.o 2024-06-06T03:47:44.237 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/hash.o 2024-06-06T03:47:44.729 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/mempolicy.o 2024-06-06T03:47:44.749 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/dma/built-in.a 2024-06-06T03:47:44.779 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/module/procfs.o 2024-06-06T03:47:44.782 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/module/sysfs.o 2024-06-06T03:47:44.868 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/mtrr/cleanup.o 2024-06-06T03:47:44.880 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/asymmetric_keys/restrict.o 2024-06-06T03:47:44.958 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/ialloc.o 2024-06-06T03:47:45.147 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/timekeeping.o 2024-06-06T03:47:45.231 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/ntp.o 2024-06-06T03:47:45.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:44 smithi131 ceph-mon[36687]: pgmap v282: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-06-06T03:47:45.283 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/asymmetric_keys/signature.o 2024-06-06T03:47:45.304 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/asymmetric_keys/public_key.o 2024-06-06T03:47:45.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:44 smithi040 ceph-mon[32220]: pgmap v282: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-06-06T03:47:45.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:44 smithi104 ceph-mon[36569]: pgmap v282: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-06-06T03:47:45.411 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/module/built-in.a 2024-06-06T03:47:45.462 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/futex/core.o 2024-06-06T03:47:45.540 INFO:tasks.workunit.client.0.smithi040.stdout: AR arch/x86/kernel/cpu/mtrr/built-in.a 2024-06-06T03:47:45.589 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/microcode/core.o 2024-06-06T03:47:45.731 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/futex/syscalls.o 2024-06-06T03:47:45.894 INFO:tasks.workunit.client.0.smithi040.stdout: ASN.1 crypto/asymmetric_keys/x509.asn1.[ch] 2024-06-06T03:47:45.904 INFO:tasks.workunit.client.0.smithi040.stdout: ASN.1 crypto/asymmetric_keys/x509_akid.asn1.[ch] 2024-06-06T03:47:45.914 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/asymmetric_keys/x509_loader.o 2024-06-06T03:47:45.954 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/asymmetric_keys/x509_public_key.o 2024-06-06T03:47:46.102 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/clocksource.o 2024-06-06T03:47:46.216 INFO:tasks.workunit.client.0.smithi040.stdout: ASN.1 crypto/asymmetric_keys/pkcs7.asn1.[ch] 2024-06-06T03:47:46.225 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/jiffies.o 2024-06-06T03:47:46.235 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/microcode/intel.o 2024-06-06T03:47:46.262 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/futex/pi.o 2024-06-06T03:47:46.340 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/sparse.o 2024-06-06T03:47:46.484 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/asymmetric_keys/pkcs7_trust.o 2024-06-06T03:47:46.485 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/futex/requeue.o 2024-06-06T03:47:46.604 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/timer_list.o 2024-06-06T03:47:46.618 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/indirect.o 2024-06-06T03:47:46.805 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/asymmetric_keys/pkcs7_verify.o 2024-06-06T03:47:46.821 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/inline.o 2024-06-06T03:47:46.891 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/timeconv.o 2024-06-06T03:47:46.919 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/microcode/amd.o 2024-06-06T03:47:46.951 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/futex/waitwake.o 2024-06-06T03:47:47.089 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/sparse-vmemmap.o 2024-06-06T03:47:47.210 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/timecounter.o 2024-06-06T03:47:47.211 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/asymmetric_keys/x509.asn1.o 2024-06-06T03:47:47.252 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/asymmetric_keys/x509_akid.asn1.o 2024-06-06T03:47:47.271 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/alarmtimer.o 2024-06-06T03:47:47.272 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/mmu_notifier.o 2024-06-06T03:47:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:46 smithi131 ceph-mon[36687]: pgmap v283: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-06-06T03:47:47.310 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/asymmetric_keys/x509_cert_parser.o 2024-06-06T03:47:47.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:46 smithi040 ceph-mon[32220]: pgmap v283: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-06-06T03:47:47.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:46 smithi104 ceph-mon[36569]: pgmap v283: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-06-06T03:47:47.473 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/futex/built-in.a 2024-06-06T03:47:47.517 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/slub.o 2024-06-06T03:47:47.673 INFO:tasks.workunit.client.0.smithi040.stdout: AR arch/x86/kernel/cpu/microcode/built-in.a 2024-06-06T03:47:47.706 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/cacheinfo.o 2024-06-06T03:47:47.755 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/migrate.o 2024-06-06T03:47:47.790 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/asymmetric_keys/pkcs7.asn1.o 2024-06-06T03:47:47.825 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/asymmetric_keys/pkcs7_parser.o 2024-06-06T03:47:47.974 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/memory-tiers.o 2024-06-06T03:47:48.029 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/inode.o 2024-06-06T03:47:48.099 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/ioctl.o 2024-06-06T03:47:48.205 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/posix-timers.o 2024-06-06T03:47:48.315 INFO:tasks.workunit.client.0.smithi040.stdout: AR crypto/asymmetric_keys/built-in.a 2024-06-06T03:47:48.345 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/api.o 2024-06-06T03:47:48.646 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/scattered.o 2024-06-06T03:47:48.760 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/page_counter.o 2024-06-06T03:47:49.030 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/topology.o 2024-06-06T03:47:49.060 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/hugetlb_cgroup.o 2024-06-06T03:47:49.075 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/cipher.o 2024-06-06T03:47:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:48 smithi131 ceph-mon[36687]: pgmap v284: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 687 KiB/s wr, 68 op/s 2024-06-06T03:47:49.291 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/early_ioremap.o 2024-06-06T03:47:49.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:48 smithi040 ceph-mon[32220]: pgmap v284: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 687 KiB/s wr, 68 op/s 2024-06-06T03:47:49.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:48 smithi104 ceph-mon[36569]: pgmap v284: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 687 KiB/s wr, 68 op/s 2024-06-06T03:47:49.346 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/posix-cpu-timers.o 2024-06-06T03:47:49.440 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/common.o 2024-06-06T03:47:49.625 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/compress.o 2024-06-06T03:47:49.649 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/mballoc.o 2024-06-06T03:47:49.931 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/secretmem.o 2024-06-06T03:47:50.010 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/migrate.o 2024-06-06T03:47:50.012 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/memfd.o 2024-06-06T03:47:50.146 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/algapi.o 2024-06-06T03:47:50.267 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/posix-clock.o 2024-06-06T03:47:50.626 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/rdrand.o 2024-06-06T03:47:50.653 INFO:tasks.workunit.client.0.smithi040.stdout: CC mm/ptdump.o 2024-06-06T03:47:50.778 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/match.o 2024-06-06T03:47:50.828 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/mmp.o 2024-06-06T03:47:50.851 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/move_extent.o 2024-06-06T03:47:50.916 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/itimer.o 2024-06-06T03:47:51.175 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/namei.o 2024-06-06T03:47:51.175 INFO:tasks.workunit.client.0.smithi040.stdout: AR mm/built-in.a 2024-06-06T03:47:51.178 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/bugs.o 2024-06-06T03:47:51.183 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/scatterwalk.o 2024-06-06T03:47:51.227 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/page-io.o 2024-06-06T03:47:51.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:50 smithi131 ceph-mon[36687]: pgmap v285: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 423 KiB/s wr, 70 op/s 2024-06-06T03:47:51.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:50 smithi040 ceph-mon[32220]: pgmap v285: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 423 KiB/s wr, 70 op/s 2024-06-06T03:47:51.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:50 smithi104 ceph-mon[36569]: pgmap v285: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 423 KiB/s wr, 70 op/s 2024-06-06T03:47:51.589 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/readpage.o 2024-06-06T03:47:51.713 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/clockevents.o 2024-06-06T03:47:51.741 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/proc.o 2024-06-06T03:47:51.746 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/aead.o 2024-06-06T03:47:52.091 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/resize.o 2024-06-06T03:47:52.285 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/geniv.o 2024-06-06T03:47:52.312 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/tick-common.o 2024-06-06T03:47:52.387 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/super.o 2024-06-06T03:47:52.443 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/aperfmperf.o 2024-06-06T03:47:52.454 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/symlink.o 2024-06-06T03:47:52.956 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/cpuid-deps.o 2024-06-06T03:47:52.998 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/skcipher.o 2024-06-06T03:47:53.053 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/seqiv.o 2024-06-06T03:47:53.115 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/tick-broadcast.o 2024-06-06T03:47:53.168 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/partitions/core.o 2024-06-06T03:47:53.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:52 smithi131 ceph-mon[36687]: pgmap v286: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 383 KiB/s wr, 52 op/s 2024-06-06T03:47:53.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:52 smithi040 ceph-mon[32220]: pgmap v286: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 383 KiB/s wr, 52 op/s 2024-06-06T03:47:53.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:52 smithi104 ceph-mon[36569]: pgmap v286: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 383 KiB/s wr, 52 op/s 2024-06-06T03:47:53.471 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/umwait.o 2024-06-06T03:47:53.673 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/bdev.o 2024-06-06T03:47:53.677 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/fops.o 2024-06-06T03:47:53.843 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/tick-broadcast-hrtimer.o 2024-06-06T03:47:53.845 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/echainiv.o 2024-06-06T03:47:54.108 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/partitions/msdos.o 2024-06-06T03:47:54.194 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/proc.o 2024-06-06T03:47:54.254 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/tick-oneshot.o 2024-06-06T03:47:54.266 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/ahash.o 2024-06-06T03:47:54.456 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/shash.o 2024-06-06T03:47:54.623 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/bio.o 2024-06-06T03:47:54.633 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/elevator.o 2024-06-06T03:47:54.689 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/tick-sched.o 2024-06-06T03:47:54.800 INFO:tasks.workunit.client.0.smithi040.stdout: MKCAP arch/x86/kernel/cpu/capflags.c 2024-06-06T03:47:54.901 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/partitions/efi.o 2024-06-06T03:47:55.128 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/akcipher.o 2024-06-06T03:47:55.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:54 smithi131 ceph-mon[36687]: pgmap v287: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 327 KiB/s wr, 46 op/s 2024-06-06T03:47:55.310 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/sig.o 2024-06-06T03:47:55.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:54 smithi040 ceph-mon[32220]: pgmap v287: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 327 KiB/s wr, 46 op/s 2024-06-06T03:47:55.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:54 smithi104 ceph-mon[36569]: pgmap v287: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 327 KiB/s wr, 46 op/s 2024-06-06T03:47:55.504 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/vsyscall.o 2024-06-06T03:47:55.581 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/timekeeping_debug.o 2024-06-06T03:47:55.673 INFO:tasks.workunit.client.0.smithi040.stdout: AR block/partitions/built-in.a 2024-06-06T03:47:55.734 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/time/namespace.o 2024-06-06T03:47:55.823 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/kpp.o 2024-06-06T03:47:55.910 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/powerflags.o 2024-06-06T03:47:55.940 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-core.o 2024-06-06T03:47:55.973 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/feat_ctl.o 2024-06-06T03:47:56.047 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/intel.o 2024-06-06T03:47:56.191 INFO:tasks.workunit.client.0.smithi040.stdout: CC arch/x86/kernel/cpu/intel_pconfig.o 2024-06-06T03:47:56.376 INFO:tasks.workunit.client.0.smithi040.stdout: ASN.1 crypto/rsapubkey.asn1.[ch] 2024-06-06T03:47:56.380 INFO:tasks.workunit.client.0.smithi040.stdout: ASN.1 crypto/rsaprivkey.asn1.[ch] 2024-06-06T03:47:56.397 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/hugetlbfs/inode.o 2024-06-06T03:47:56.413 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/io_uring.o 2024-06-06T03:47:56.415 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/time/built-in.a 2024-06-06T03:47:56.465 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/rsa.o 2024-06-06T03:47:56.475 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/cgroup/cgroup.o 2024-06-06T03:47:57.021 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_clock.o 2024-06-06T03:47:57.047 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/rsa_helper.o 2024-06-06T03:47:57.146 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/rsa-pkcs1pad.o 2024-06-06T03:47:57.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:56 smithi131 ceph-mon[36687]: pgmap v288: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-06-06T03:47:57.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:56 smithi040 ceph-mon[32220]: pgmap v288: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-06-06T03:47:57.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:56 smithi104 ceph-mon[36569]: pgmap v288: 97 pgs: 97 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-06-06T03:47:57.389 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/ring_buffer.o 2024-06-06T03:47:57.490 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/hugetlbfs/built-in.a 2024-06-06T03:47:57.564 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-sysfs.o 2024-06-06T03:47:57.704 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/bpf/core.o 2024-06-06T03:47:57.843 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/acompress.o 2024-06-06T03:47:58.452 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-flush.o 2024-06-06T03:47:58.577 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/scompress.o 2024-06-06T03:47:58.876 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/sysfs.o 2024-06-06T03:47:59.216 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-settings.o 2024-06-06T03:47:59.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:58 smithi131 ceph-mon[36687]: pgmap v289: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-06-06T03:47:59.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:58 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:47:59.305 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/algboss.o 2024-06-06T03:47:59.312 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace.o 2024-06-06T03:47:59.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:58 smithi040 ceph-mon[32220]: pgmap v289: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-06-06T03:47:59.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:58 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:47:59.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:58 smithi104 ceph-mon[36569]: pgmap v289: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-06-06T03:47:59.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:58 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:47:59.639 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/xattr.o 2024-06-06T03:47:59.670 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/cgroup/rstat.o 2024-06-06T03:47:59.684 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/xattr.o 2024-06-06T03:47:59.713 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/bpf/built-in.a 2024-06-06T03:47:59.762 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/events/core.o 2024-06-06T03:48:00.008 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-ioc.o 2024-06-06T03:48:00.090 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/testmgr.o 2024-06-06T03:48:00.235 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:59 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:48:00.235 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:59 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:48:00.235 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:47:59 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:48:00.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:59 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:48:00.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:59 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:48:00.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:47:59 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:48:00.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:59 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:48:00.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:59 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:48:00.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:47:59 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:48:00.423 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/cgroup/namespace.o 2024-06-06T03:48:00.430 INFO:tasks.workunit.client.0.smithi040.stderr:./arch/x86/kernel/cpu/mkcapflags.sh: line 46: printf: write error: Resource temporarily unavailable 2024-06-06T03:48:00.434 INFO:tasks.workunit.client.0.smithi040.stderr:make[5]: *** [arch/x86/kernel/cpu/Makefile:64: arch/x86/kernel/cpu/capflags.c] Error 1 2024-06-06T03:48:00.435 INFO:tasks.workunit.client.0.smithi040.stderr:make[4]: *** [scripts/Makefile.build:480: arch/x86/kernel/cpu] Error 2 2024-06-06T03:48:00.436 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/nop.o 2024-06-06T03:48:00.436 INFO:tasks.workunit.client.0.smithi040.stderr:make[3]: *** [scripts/Makefile.build:480: arch/x86/kernel] Error 2 2024-06-06T03:48:00.436 INFO:tasks.workunit.client.0.smithi040.stderr:make[2]: *** [scripts/Makefile.build:480: arch/x86] Error 2 2024-06-06T03:48:00.436 INFO:tasks.workunit.client.0.smithi040.stderr:make[2]: *** Waiting for unfinished jobs.... 2024-06-06T03:48:00.443 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/events/ring_buffer.o 2024-06-06T03:48:00.649 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-map.o 2024-06-06T03:48:00.667 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/cmac.o 2024-06-06T03:48:00.995 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/cgroup/cgroup-v1.o 2024-06-06T03:48:01.085 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/fs.o 2024-06-06T03:48:01.127 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/hmac.o 2024-06-06T03:48:01.250 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:00 smithi040 ceph-mon[32220]: pgmap v290: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-06-06T03:48:01.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:00 smithi131 ceph-mon[36687]: pgmap v290: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-06-06T03:48:01.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:00 smithi104 ceph-mon[36569]: pgmap v290: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-06-06T03:48:01.349 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/xattr_hurd.o 2024-06-06T03:48:01.352 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/cgroup/freezer.o 2024-06-06T03:48:01.640 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-merge.o 2024-06-06T03:48:01.843 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/crypto_null.o 2024-06-06T03:48:01.914 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/splice.o 2024-06-06T03:48:02.041 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/sync.o 2024-06-06T03:48:02.051 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/xattr_trusted.o 2024-06-06T03:48:02.095 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/cgroup/legacy_freezer.o 2024-06-06T03:48:02.505 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/md5.o 2024-06-06T03:48:02.513 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_output.o 2024-06-06T03:48:02.738 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/xattr_user.o 2024-06-06T03:48:02.758 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/cgroup/pids.o 2024-06-06T03:48:02.764 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/fast_commit.o 2024-06-06T03:48:02.885 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/advise.o 2024-06-06T03:48:02.948 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-timeout.o 2024-06-06T03:48:03.104 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/sha256_generic.o 2024-06-06T03:48:03.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:02 smithi131 ceph-mon[36687]: pgmap v291: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.8 MiB/s wr, 127 op/s 2024-06-06T03:48:03.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:02 smithi040 ceph-mon[32220]: pgmap v291: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.8 MiB/s wr, 127 op/s 2024-06-06T03:48:03.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:02 smithi104 ceph-mon[36569]: pgmap v291: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.8 MiB/s wr, 127 op/s 2024-06-06T03:48:03.363 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/cgroup/rdma.o 2024-06-06T03:48:03.382 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/cgroup/cpuset.o 2024-06-06T03:48:03.584 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-lib.o 2024-06-06T03:48:03.585 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/filetable.o 2024-06-06T03:48:03.752 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/sha512_generic.o 2024-06-06T03:48:03.805 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_seq.o 2024-06-06T03:48:04.040 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/sha3_generic.o 2024-06-06T03:48:04.272 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-mq.o 2024-06-06T03:48:04.324 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_stat.o 2024-06-06T03:48:04.430 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-mq-tag.o 2024-06-06T03:48:04.502 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/cbc.o 2024-06-06T03:48:04.520 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/openclose.o 2024-06-06T03:48:04.538 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/events/callchain.o 2024-06-06T03:48:04.540 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/orphan.o 2024-06-06T03:48:04.917 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/ctr.o 2024-06-06T03:48:05.003 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_printk.o 2024-06-06T03:48:05.101 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/cgroup/misc.o 2024-06-06T03:48:05.194 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/events/hw_breakpoint.o 2024-06-06T03:48:05.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:04 smithi131 ceph-mon[36687]: pgmap v292: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 1.8 MiB/s wr, 123 op/s 2024-06-06T03:48:05.305 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-stat.o 2024-06-06T03:48:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:04 smithi040 ceph-mon[32220]: pgmap v292: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 1.8 MiB/s wr, 123 op/s 2024-06-06T03:48:05.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:04 smithi104 ceph-mon[36569]: pgmap v292: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 1.8 MiB/s wr, 123 op/s 2024-06-06T03:48:05.371 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/gcm.o 2024-06-06T03:48:05.380 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/acl.o 2024-06-06T03:48:05.482 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/uring_cmd.o 2024-06-06T03:48:05.614 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/cgroup/debug.o 2024-06-06T03:48:05.687 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/pid_list.o 2024-06-06T03:48:05.978 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-mq-sysfs.o 2024-06-06T03:48:06.096 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ext4/xattr_security.o 2024-06-06T03:48:06.239 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/cgroup/built-in.a 2024-06-06T03:48:06.263 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/events/uprobes.o 2024-06-06T03:48:06.271 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/fork.o 2024-06-06T03:48:06.272 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/ccm.o 2024-06-06T03:48:06.389 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_sched_switch.o 2024-06-06T03:48:06.406 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/epoll.o 2024-06-06T03:48:06.651 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/statx.o 2024-06-06T03:48:06.748 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/ext4/built-in.a 2024-06-06T03:48:06.792 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-mq-cpumap.o 2024-06-06T03:48:06.820 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fat/cache.o 2024-06-06T03:48:07.082 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/net.o 2024-06-06T03:48:07.132 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/aes_generic.o 2024-06-06T03:48:07.160 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_nop.o 2024-06-06T03:48:07.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:06 smithi131 ceph-mon[36687]: pgmap v293: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-06-06T03:48:07.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:06 smithi040 ceph-mon[32220]: pgmap v293: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-06-06T03:48:07.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:06 smithi104 ceph-mon[36569]: pgmap v293: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-06-06T03:48:07.355 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/msg_ring.o 2024-06-06T03:48:07.404 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-mq-sched.o 2024-06-06T03:48:07.486 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fat/dir.o 2024-06-06T03:48:07.640 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/events/built-in.a 2024-06-06T03:48:07.683 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/ioctl.o 2024-06-06T03:48:07.755 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/blktrace.o 2024-06-06T03:48:07.886 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/crc32c_generic.o 2024-06-06T03:48:08.241 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:07 smithi104 ceph-mon[36569]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T03:48:08.250 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/authenc.o 2024-06-06T03:48:08.272 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_events.o 2024-06-06T03:48:08.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:07 smithi131 ceph-mon[36687]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T03:48:08.284 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_export.o 2024-06-06T03:48:08.297 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/timeout.o 2024-06-06T03:48:08.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:07 smithi040 ceph-mon[32220]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T03:48:08.493 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/genhd.o 2024-06-06T03:48:08.508 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/ioprio.o 2024-06-06T03:48:08.698 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fat/fatent.o 2024-06-06T03:48:08.929 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_event_perf.o 2024-06-06T03:48:09.065 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_events_filter.o 2024-06-06T03:48:09.083 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/authencesn.o 2024-06-06T03:48:09.171 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/sqpoll.o 2024-06-06T03:48:09.224 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/badblocks.o 2024-06-06T03:48:09.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:08 smithi131 ceph-mon[36687]: pgmap v295: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 472 KiB/s wr, 87 op/s 2024-06-06T03:48:09.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:08 smithi040 ceph-mon[32220]: pgmap v295: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 472 KiB/s wr, 87 op/s 2024-06-06T03:48:09.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:08 smithi104 ceph-mon[36569]: pgmap v295: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 472 KiB/s wr, 87 op/s 2024-06-06T03:48:09.498 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-rq-qos.o 2024-06-06T03:48:09.620 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fat/file.o 2024-06-06T03:48:09.674 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_events_trigger.o 2024-06-06T03:48:09.782 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/disk-events.o 2024-06-06T03:48:09.911 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_eprobe.o 2024-06-06T03:48:09.925 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/rng.o 2024-06-06T03:48:10.042 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/fdinfo.o 2024-06-06T03:48:10.166 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-ia-ranges.o 2024-06-06T03:48:10.232 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_kprobe.o 2024-06-06T03:48:10.413 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fat/inode.o 2024-06-06T03:48:10.477 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/early-lookup.o 2024-06-06T03:48:10.614 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/drbg.o 2024-06-06T03:48:10.650 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/error_report-traces.o 2024-06-06T03:48:10.810 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/bsg.o 2024-06-06T03:48:11.005 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/power-traces.o 2024-06-06T03:48:11.014 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/tctx.o 2024-06-06T03:48:11.127 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/rpm-traces.o 2024-06-06T03:48:11.270 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_dynevent.o 2024-06-06T03:48:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:10 smithi131 ceph-mon[36687]: pgmap v296: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 392 KiB/s wr, 69 op/s 2024-06-06T03:48:11.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:10 smithi040 ceph-mon[32220]: pgmap v296: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 392 KiB/s wr, 69 op/s 2024-06-06T03:48:11.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:10 smithi104 ceph-mon[36569]: pgmap v296: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 392 KiB/s wr, 69 op/s 2024-06-06T03:48:11.387 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/jitterentropy.o 2024-06-06T03:48:11.457 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/jitterentropy-kcapi.o 2024-06-06T03:48:11.461 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-cgroup.o 2024-06-06T03:48:11.505 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fat/misc.o 2024-06-06T03:48:11.767 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/poll.o 2024-06-06T03:48:11.805 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_probe.o 2024-06-06T03:48:11.851 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/trace_uprobe.o 2024-06-06T03:48:11.898 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/ghash-generic.o 2024-06-06T03:48:11.933 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/hash_info.o 2024-06-06T03:48:11.985 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/trace/rethook.o 2024-06-06T03:48:12.171 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fat/nfs.o 2024-06-06T03:48:12.278 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fat/namei_vfat.o 2024-06-06T03:48:12.307 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/rsapubkey.asn1.o 2024-06-06T03:48:12.346 INFO:tasks.workunit.client.0.smithi040.stdout: CC crypto/rsaprivkey.asn1.o 2024-06-06T03:48:12.382 INFO:tasks.workunit.client.0.smithi040.stdout: AR crypto/built-in.a 2024-06-06T03:48:12.429 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fat/namei_msdos.o 2024-06-06T03:48:12.620 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/cancel.o 2024-06-06T03:48:12.722 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/kbuf.o 2024-06-06T03:48:12.735 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-ioprio.o 2024-06-06T03:48:12.753 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-iolatency.o 2024-06-06T03:48:12.999 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-iocost.o 2024-06-06T03:48:13.188 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/mq-deadline.o 2024-06-06T03:48:13.265 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/fat/built-in.a 2024-06-06T03:48:13.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:12 smithi131 ceph-mon[36687]: pgmap v297: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 389 KiB/s wr, 68 op/s 2024-06-06T03:48:13.309 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/isofs/namei.o 2024-06-06T03:48:13.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:12 smithi040 ceph-mon[32220]: pgmap v297: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 389 KiB/s wr, 68 op/s 2024-06-06T03:48:13.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:12 smithi104 ceph-mon[36569]: pgmap v297: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 389 KiB/s wr, 68 op/s 2024-06-06T03:48:13.373 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/kyber-iosched.o 2024-06-06T03:48:13.378 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/rsrc.o 2024-06-06T03:48:13.387 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/trace/built-in.a 2024-06-06T03:48:13.416 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/exec_domain.o 2024-06-06T03:48:13.616 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/rw.o 2024-06-06T03:48:13.801 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-mq-pci.o 2024-06-06T03:48:13.939 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/isofs/inode.o 2024-06-06T03:48:14.018 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/panic.o 2024-06-06T03:48:14.415 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-mq-virtio.o 2024-06-06T03:48:14.575 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-mq-debugfs.o 2024-06-06T03:48:14.742 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/blk-pm.o 2024-06-06T03:48:14.938 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/cpu.o 2024-06-06T03:48:14.978 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/opdef.o 2024-06-06T03:48:14.983 INFO:tasks.workunit.client.0.smithi040.stdout: CC block/holder.o 2024-06-06T03:48:15.062 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/client.o 2024-06-06T03:48:15.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:14 smithi040 ceph-mon[32220]: pgmap v298: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 389 KiB/s wr, 68 op/s 2024-06-06T03:48:15.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:14 smithi040 ceph-mon[32220]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T03:48:15.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:14 smithi104 ceph-mon[36569]: pgmap v298: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 389 KiB/s wr, 68 op/s 2024-06-06T03:48:15.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:14 smithi104 ceph-mon[36569]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T03:48:15.097 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/dir.o 2024-06-06T03:48:15.219 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/isofs/dir.o 2024-06-06T03:48:15.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:14 smithi131 ceph-mon[36687]: pgmap v298: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 389 KiB/s wr, 68 op/s 2024-06-06T03:48:15.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:14 smithi131 ceph-mon[36687]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T03:48:15.373 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/isofs/util.o 2024-06-06T03:48:15.425 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/isofs/rock.o 2024-06-06T03:48:15.620 INFO:tasks.workunit.client.0.smithi040.stdout: AR block/built-in.a 2024-06-06T03:48:15.697 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/isofs/export.o 2024-06-06T03:48:15.859 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/isofs/joliet.o 2024-06-06T03:48:15.917 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/file.o 2024-06-06T03:48:15.919 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/notif.o 2024-06-06T03:48:16.161 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/isofs/compress.o 2024-06-06T03:48:16.274 INFO:tasks.workunit.client.0.smithi040.stdout: CC io_uring/io-wq.o 2024-06-06T03:48:16.452 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/getroot.o 2024-06-06T03:48:16.488 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/inode.o 2024-06-06T03:48:16.543 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/exit.o 2024-06-06T03:48:16.828 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/exportfs/expfs.o 2024-06-06T03:48:16.908 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/isofs/built-in.a 2024-06-06T03:48:16.964 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/clntlock.o 2024-06-06T03:48:17.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:16 smithi040 ceph-mon[32220]: pgmap v300: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 801 KiB/s wr, 61 op/s 2024-06-06T03:48:17.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:16 smithi104 ceph-mon[36569]: pgmap v300: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 801 KiB/s wr, 61 op/s 2024-06-06T03:48:17.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:16 smithi131 ceph-mon[36687]: pgmap v300: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 801 KiB/s wr, 61 op/s 2024-06-06T03:48:17.343 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/super.o 2024-06-06T03:48:17.404 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/exportfs/built-in.a 2024-06-06T03:48:17.413 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/io.o 2024-06-06T03:48:17.423 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/direct.o 2024-06-06T03:48:17.441 INFO:tasks.workunit.client.0.smithi040.stdout: AR io_uring/built-in.a 2024-06-06T03:48:17.480 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nls/nls_base.o 2024-06-06T03:48:17.491 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/unicode/built-in.a 2024-06-06T03:48:17.504 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/softirq.o 2024-06-06T03:48:17.931 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nls/nls_cp437.o 2024-06-06T03:48:17.950 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/clntproc.o 2024-06-06T03:48:18.015 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/resource.o 2024-06-06T03:48:18.305 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/pagelist.o 2024-06-06T03:48:18.310 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nls/nls_ascii.o 2024-06-06T03:48:18.417 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/read.o 2024-06-06T03:48:18.549 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/sysctl.o 2024-06-06T03:48:18.579 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/symlink.o 2024-06-06T03:48:18.672 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nls/nls_iso8859-1.o 2024-06-06T03:48:18.966 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/capability.o 2024-06-06T03:48:18.994 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/unlink.o 2024-06-06T03:48:19.029 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nls/nls_utf8.o 2024-06-06T03:48:19.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:18 smithi040 ceph-mon[32220]: pgmap v301: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-06-06T03:48:19.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:18 smithi104 ceph-mon[36569]: pgmap v301: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-06-06T03:48:19.103 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/clntxdr.o 2024-06-06T03:48:19.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:18 smithi131 ceph-mon[36687]: pgmap v301: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-06-06T03:48:19.395 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/nls/built-in.a 2024-06-06T03:48:19.447 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/write.o 2024-06-06T03:48:19.456 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/autofs/init.o 2024-06-06T03:48:19.673 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/ptrace.o 2024-06-06T03:48:19.848 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/namespace.o 2024-06-06T03:48:19.922 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/autofs/inode.o 2024-06-06T03:48:19.967 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/mount_clnt.o 2024-06-06T03:48:19.981 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/user.o 2024-06-06T03:48:19.996 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/host.o 2024-06-06T03:48:20.286 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfstrace.o 2024-06-06T03:48:20.372 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/signal.o 2024-06-06T03:48:20.522 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/autofs/root.o 2024-06-06T03:48:20.678 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/sys.o 2024-06-06T03:48:20.789 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/export.o 2024-06-06T03:48:20.891 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/sysfs.o 2024-06-06T03:48:21.031 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/svc.o 2024-06-06T03:48:21.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:20 smithi040 ceph-mon[32220]: pgmap v302: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-06-06T03:48:21.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:20 smithi104 ceph-mon[36569]: pgmap v302: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-06-06T03:48:21.189 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/autofs/symlink.o 2024-06-06T03:48:21.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:20 smithi131 ceph-mon[36687]: pgmap v302: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-06-06T03:48:21.424 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/fs_context.o 2024-06-06T03:48:21.624 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/autofs/waitq.o 2024-06-06T03:48:21.835 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfsroot.o 2024-06-06T03:48:21.976 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/sysctl.o 2024-06-06T03:48:22.045 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/svclock.o 2024-06-06T03:48:22.199 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/autofs/expire.o 2024-06-06T03:48:22.296 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/umh.o 2024-06-06T03:48:22.616 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/workqueue.o 2024-06-06T03:48:22.764 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs2super.o 2024-06-06T03:48:22.766 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/autofs/dev-ioctl.o 2024-06-06T03:48:22.824 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/proc.o 2024-06-06T03:48:22.965 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs2xdr.o 2024-06-06T03:48:23.032 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/pid.o 2024-06-06T03:48:23.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:22 smithi040 ceph-mon[32220]: pgmap v303: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.0 MiB/s wr, 132 op/s 2024-06-06T03:48:23.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:22 smithi104 ceph-mon[36569]: pgmap v303: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.0 MiB/s wr, 132 op/s 2024-06-06T03:48:23.108 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/svcshare.o 2024-06-06T03:48:23.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:22 smithi131 ceph-mon[36687]: pgmap v303: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.0 MiB/s wr, 132 op/s 2024-06-06T03:48:23.399 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs3super.o 2024-06-06T03:48:23.407 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/autofs/built-in.a 2024-06-06T03:48:23.455 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/9p/vfs_super.o 2024-06-06T03:48:23.620 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs3client.o 2024-06-06T03:48:23.884 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs3proc.o 2024-06-06T03:48:23.976 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/svcproc.o 2024-06-06T03:48:24.089 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/9p/vfs_inode.o 2024-06-06T03:48:24.106 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/task_work.o 2024-06-06T03:48:24.272 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs3xdr.o 2024-06-06T03:48:24.335 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs3acl.o 2024-06-06T03:48:24.502 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs4proc.o 2024-06-06T03:48:24.686 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/extable.o 2024-06-06T03:48:24.927 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/params.o 2024-06-06T03:48:24.956 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/svcsubs.o 2024-06-06T03:48:25.024 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs4xdr.o 2024-06-06T03:48:25.033 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/9p/vfs_inode_dotl.o 2024-06-06T03:48:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:24 smithi040 ceph-mon[32220]: pgmap v304: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.0 MiB/s wr, 132 op/s 2024-06-06T03:48:25.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:24 smithi104 ceph-mon[36569]: pgmap v304: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.0 MiB/s wr, 132 op/s 2024-06-06T03:48:25.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:24 smithi131 ceph-mon[36687]: pgmap v304: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.0 MiB/s wr, 132 op/s 2024-06-06T03:48:25.313 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/9p/vfs_addr.o 2024-06-06T03:48:25.533 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/kthread.o 2024-06-06T03:48:25.727 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/sys_ni.o 2024-06-06T03:48:25.840 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/9p/vfs_file.o 2024-06-06T03:48:25.892 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs4state.o 2024-06-06T03:48:25.906 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/mon.o 2024-06-06T03:48:25.993 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/9p/vfs_dir.o 2024-06-06T03:48:26.310 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/nsproxy.o 2024-06-06T03:48:26.538 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/9p/vfs_dentry.o 2024-06-06T03:48:26.623 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/9p/v9fs.o 2024-06-06T03:48:26.631 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/notifier.o 2024-06-06T03:48:26.865 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/trace.o 2024-06-06T03:48:27.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:26 smithi040 ceph-mon[32220]: pgmap v305: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.8 MiB/s wr, 124 op/s 2024-06-06T03:48:27.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:26 smithi104 ceph-mon[36569]: pgmap v305: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.8 MiB/s wr, 124 op/s 2024-06-06T03:48:27.122 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/ksysfs.o 2024-06-06T03:48:27.133 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/9p/fid.o 2024-06-06T03:48:27.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:26 smithi131 ceph-mon[36687]: pgmap v305: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.8 MiB/s wr, 124 op/s 2024-06-06T03:48:27.290 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/9p/xattr.o 2024-06-06T03:48:27.397 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/cred.o 2024-06-06T03:48:27.575 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs4renewd.o 2024-06-06T03:48:27.682 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/reboot.o 2024-06-06T03:48:27.736 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs4super.o 2024-06-06T03:48:27.787 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs4file.o 2024-06-06T03:48:27.813 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/xdr.o 2024-06-06T03:48:27.902 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/delegation.o 2024-06-06T03:48:27.905 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/9p/built-in.a 2024-06-06T03:48:27.975 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/hostfs/built-in.a 2024-06-06T03:48:27.988 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/clnt4xdr.o 2024-06-06T03:48:28.154 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/async.o 2024-06-06T03:48:28.439 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs4idmap.o 2024-06-06T03:48:28.574 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/xdr4.o 2024-06-06T03:48:28.604 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/range.o 2024-06-06T03:48:28.640 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/callback.o 2024-06-06T03:48:28.705 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/smpboot.o 2024-06-06T03:48:28.817 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/svc4proc.o 2024-06-06T03:48:28.891 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/lockd/procfs.o 2024-06-06T03:48:28.977 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/callback_xdr.o 2024-06-06T03:48:29.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:28 smithi104 ceph-mon[36569]: pgmap v306: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-06-06T03:48:29.234 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/ucount.o 2024-06-06T03:48:29.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:28 smithi131 ceph-mon[36687]: pgmap v306: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-06-06T03:48:29.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:28 smithi040 ceph-mon[32220]: pgmap v306: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-06-06T03:48:29.423 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/regset.o 2024-06-06T03:48:29.522 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/ksyms_common.o 2024-06-06T03:48:29.592 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/groups.o 2024-06-06T03:48:29.613 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/callback_proc.o 2024-06-06T03:48:29.676 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs4namespace.o 2024-06-06T03:48:29.691 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/kcmp.o 2024-06-06T03:48:29.775 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/lockd/built-in.a 2024-06-06T03:48:29.821 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/freezer.o 2024-06-06T03:48:29.821 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/debugfs/inode.o 2024-06-06T03:48:30.028 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/profile.o 2024-06-06T03:48:30.135 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs4getroot.o 2024-06-06T03:48:30.269 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/stacktrace.o 2024-06-06T03:48:30.346 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/dma.o 2024-06-06T03:48:30.488 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/smp.o 2024-06-06T03:48:30.573 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/debugfs/file.o 2024-06-06T03:48:30.702 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs4client.o 2024-06-06T03:48:30.731 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/uid16.o 2024-06-06T03:48:30.736 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/kallsyms.o 2024-06-06T03:48:30.822 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/acct.o 2024-06-06T03:48:30.844 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/crash_core.o 2024-06-06T03:48:30.987 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs4session.o 2024-06-06T03:48:31.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:30 smithi104 ceph-mon[36569]: pgmap v307: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-06-06T03:48:31.133 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:30 smithi040 ceph-mon[32220]: pgmap v307: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-06-06T03:48:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:30 smithi131 ceph-mon[36687]: pgmap v307: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-06-06T03:48:31.432 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/debugfs/built-in.a 2024-06-06T03:48:31.463 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/tracefs/inode.o 2024-06-06T03:48:31.467 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/kexec_core.o 2024-06-06T03:48:31.500 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/kexec.o 2024-06-06T03:48:31.502 INFO:tasks.workunit.client.0.smithi040.stdout: CC [M] fs/efivarfs/inode.o 2024-06-06T03:48:31.594 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/compat.o 2024-06-06T03:48:31.752 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/utsname.o 2024-06-06T03:48:31.792 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/dns_resolve.o 2024-06-06T03:48:32.005 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs4trace.o 2024-06-06T03:48:32.043 INFO:tasks.workunit.client.0.smithi040.stdout: CC [M] fs/efivarfs/file.o 2024-06-06T03:48:32.130 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/tracefs/built-in.a 2024-06-06T03:48:32.162 INFO:tasks.workunit.client.0.smithi040.stdout: CC [M] fs/efivarfs/super.o 2024-06-06T03:48:32.179 INFO:tasks.workunit.client.0.smithi040.stdout: CC [M] fs/efivarfs/vars.o 2024-06-06T03:48:32.190 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/pid_namespace.o 2024-06-06T03:48:32.306 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/stop_machine.o 2024-06-06T03:48:32.498 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/audit.o 2024-06-06T03:48:32.567 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/auditfilter.o 2024-06-06T03:48:32.613 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/open.o 2024-06-06T03:48:32.807 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/read_write.o 2024-06-06T03:48:32.850 INFO:tasks.workunit.client.0.smithi040.stdout: LD [M] fs/efivarfs/efivarfs.o 2024-06-06T03:48:32.880 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/file_table.o 2024-06-06T03:48:32.908 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/auditsc.o 2024-06-06T03:48:33.025 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/audit_watch.o 2024-06-06T03:48:33.090 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:32 smithi104 ceph-mon[36569]: pgmap v308: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 224 KiB/s wr, 41 op/s 2024-06-06T03:48:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:32 smithi131 ceph-mon[36687]: pgmap v308: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 224 KiB/s wr, 41 op/s 2024-06-06T03:48:33.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:32 smithi040 ceph-mon[32220]: pgmap v308: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 224 KiB/s wr, 41 op/s 2024-06-06T03:48:33.643 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/super.o 2024-06-06T03:48:33.792 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/audit_fsnotify.o 2024-06-06T03:48:33.818 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/audit_tree.o 2024-06-06T03:48:33.906 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/char_dev.o 2024-06-06T03:48:33.977 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/kprobes.o 2024-06-06T03:48:34.195 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/stat.o 2024-06-06T03:48:34.456 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/seccomp.o 2024-06-06T03:48:34.460 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nfs/nfs4sysctl.o 2024-06-06T03:48:34.492 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/relay.o 2024-06-06T03:48:34.612 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/exec.o 2024-06-06T03:48:34.676 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/pipe.o 2024-06-06T03:48:34.952 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/utsname_sysctl.o 2024-06-06T03:48:35.253 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/namei.o 2024-06-06T03:48:35.259 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/delayacct.o 2024-06-06T03:48:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:34 smithi131 ceph-mon[36687]: pgmap v309: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 221 KiB/s wr, 34 op/s 2024-06-06T03:48:35.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:34 smithi040 ceph-mon[32220]: pgmap v309: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 221 KiB/s wr, 34 op/s 2024-06-06T03:48:35.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:34 smithi104 ceph-mon[36569]: pgmap v309: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 221 KiB/s wr, 34 op/s 2024-06-06T03:48:35.436 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/taskstats.o 2024-06-06T03:48:35.544 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/nfs/built-in.a 2024-06-06T03:48:35.571 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/tsacct.o 2024-06-06T03:48:35.595 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fcntl.o 2024-06-06T03:48:35.779 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/tracepoint.o 2024-06-06T03:48:35.922 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/ioctl.o 2024-06-06T03:48:35.949 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/irq_work.o 2024-06-06T03:48:36.116 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/static_call.o 2024-06-06T03:48:36.196 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/readdir.o 2024-06-06T03:48:36.266 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/static_call_inline.o 2024-06-06T03:48:36.399 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/crash_dump.o 2024-06-06T03:48:36.422 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/select.o 2024-06-06T03:48:36.487 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/jump_label.o 2024-06-06T03:48:36.505 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/context_tracking.o 2024-06-06T03:48:36.775 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/iomem.o 2024-06-06T03:48:36.778 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/dcache.o 2024-06-06T03:48:36.920 INFO:tasks.workunit.client.0.smithi040.stdout: CC kernel/rseq.o 2024-06-06T03:48:37.032 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/inode.o 2024-06-06T03:48:37.179 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/attr.o 2024-06-06T03:48:37.218 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/bad_inode.o 2024-06-06T03:48:37.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:36 smithi131 ceph-mon[36687]: pgmap v310: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 322 KiB/s wr, 49 op/s 2024-06-06T03:48:37.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:36 smithi040 ceph-mon[32220]: pgmap v310: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 322 KiB/s wr, 49 op/s 2024-06-06T03:48:37.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:36 smithi104 ceph-mon[36569]: pgmap v310: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 322 KiB/s wr, 49 op/s 2024-06-06T03:48:37.354 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/file.o 2024-06-06T03:48:37.691 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/filesystems.o 2024-06-06T03:48:37.707 INFO:tasks.workunit.client.0.smithi040.stdout: AR kernel/built-in.a 2024-06-06T03:48:37.739 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/namespace.o 2024-06-06T03:48:37.810 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/seq_file.o 2024-06-06T03:48:37.823 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/xattr.o 2024-06-06T03:48:37.877 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/libfs.o 2024-06-06T03:48:38.213 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fs-writeback.o 2024-06-06T03:48:38.375 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/pnode.o 2024-06-06T03:48:38.473 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/splice.o 2024-06-06T03:48:38.574 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/sync.o 2024-06-06T03:48:38.665 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/utimes.o 2024-06-06T03:48:38.891 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/d_path.o 2024-06-06T03:48:38.970 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/stack.o 2024-06-06T03:48:39.000 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fs_struct.o 2024-06-06T03:48:39.142 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:38 smithi104 ceph-mon[36569]: pgmap v311: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 583 KiB/s wr, 55 op/s 2024-06-06T03:48:39.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:38 smithi131 ceph-mon[36687]: pgmap v311: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 583 KiB/s wr, 55 op/s 2024-06-06T03:48:39.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:38 smithi040 ceph-mon[32220]: pgmap v311: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 583 KiB/s wr, 55 op/s 2024-06-06T03:48:39.363 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/statfs.o 2024-06-06T03:48:39.407 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fs_pin.o 2024-06-06T03:48:39.418 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/nsfs.o 2024-06-06T03:48:39.507 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fs_types.o 2024-06-06T03:48:39.676 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fs_context.o 2024-06-06T03:48:39.774 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fs_parser.o 2024-06-06T03:48:39.885 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fsopen.o 2024-06-06T03:48:39.913 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/init.o 2024-06-06T03:48:39.963 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/kernel_read_file.o 2024-06-06T03:48:39.988 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/mnt_idmapping.o 2024-06-06T03:48:40.008 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/remap_range.o 2024-06-06T03:48:40.137 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/buffer.o 2024-06-06T03:48:40.369 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/mpage.o 2024-06-06T03:48:40.465 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/proc_namespace.o 2024-06-06T03:48:40.480 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/direct-io.o 2024-06-06T03:48:40.486 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/eventpoll.o 2024-06-06T03:48:40.514 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/anon_inodes.o 2024-06-06T03:48:40.591 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/signalfd.o 2024-06-06T03:48:40.803 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/timerfd.o 2024-06-06T03:48:41.048 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/eventfd.o 2024-06-06T03:48:41.080 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/aio.o 2024-06-06T03:48:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:40 smithi131 ceph-mon[36687]: pgmap v312: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.8 MiB/s wr, 110 op/s 2024-06-06T03:48:41.286 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/locks.o 2024-06-06T03:48:41.291 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/binfmt_misc.o 2024-06-06T03:48:41.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:40 smithi040 ceph-mon[32220]: pgmap v312: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.8 MiB/s wr, 110 op/s 2024-06-06T03:48:41.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:40 smithi104 ceph-mon[36569]: pgmap v312: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.8 MiB/s wr, 110 op/s 2024-06-06T03:48:41.421 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/binfmt_script.o 2024-06-06T03:48:41.605 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/binfmt_elf.o 2024-06-06T03:48:41.752 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/compat_binfmt_elf.o 2024-06-06T03:48:41.923 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/mbcache.o 2024-06-06T03:48:41.926 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/posix_acl.o 2024-06-06T03:48:41.960 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/coredump.o 2024-06-06T03:48:42.100 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/drop_caches.o 2024-06-06T03:48:42.543 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/sysctls.o 2024-06-06T03:48:42.564 INFO:tasks.workunit.client.0.smithi040.stdout: CC fs/fhandle.o 2024-06-06T03:48:43.073 INFO:tasks.workunit.client.0.smithi040.stdout: AR fs/built-in.a 2024-06-06T03:48:43.230 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-06-06T03:48:43.230 INFO:tasks.workunit.client.0.smithi040.stderr:make[1]: *** [/home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/Makefile:2034: .] Error 2 2024-06-06T03:48:43.231 INFO:tasks.workunit.client.0.smithi040.stderr:make: *** [Makefile:234: __sub-make] Error 2 2024-06-06T03:48:43.231 INFO:tasks.workunit:Stopping ['kernel_untar_build.sh'] on client.0... 2024-06-06T03:48:43.231 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-06T03:48:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:42 smithi131 ceph-mon[36687]: pgmap v313: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-06-06T03:48:43.296 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:42 smithi040 ceph-mon[32220]: pgmap v313: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-06-06T03:48:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:42 smithi104 ceph-mon[36569]: pgmap v313: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-06-06T03:48:43.583 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test kernel_untar_build.sh) on smithi040 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d746ca5346f34eef7e826a1eb71e59e02b682ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2024-06-06T03:48:43.931 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=efd308eb8d134c02906dff01dfe59e18 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test kernel_untar_build.sh) on smithi040 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d746ca5346f34eef7e826a1eb71e59e02b682ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2024-06-06T03:48:43.935 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-06-06T03:48:43.948 INFO:tasks.quiescer:joining Quiescers 2024-06-06T03:48:43.948 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-06-06T03:48:43.949 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:48:44.205 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:48:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:44 smithi040 ceph-mon[32220]: pgmap v314: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.7 MiB/s wr, 99 op/s 2024-06-06T03:48:45.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:44 smithi131 ceph-mon[36687]: pgmap v314: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.7 MiB/s wr, 99 op/s 2024-06-06T03:48:45.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:44 smithi104 ceph-mon[36569]: pgmap v314: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.7 MiB/s wr, 99 op/s 2024-06-06T03:48:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.083+0000 7fee751c7700 1 -- 172.21.15.40:0/1007498533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee70107840 msgr2=0x7fee7010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.083+0000 7fee751c7700 1 --2- 172.21.15.40:0/1007498533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee70107840 0x7fee7010f3c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fee6800b0d0 tx=0x7fee6800b3e0 comp rx=0 tx=0).stop 2024-06-06T03:48:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.083+0000 7fee751c7700 1 -- 172.21.15.40:0/1007498533 shutdown_connections 2024-06-06T03:48:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.083+0000 7fee751c7700 1 --2- 172.21.15.40:0/1007498533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee70107840 0x7fee7010f3c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.083+0000 7fee751c7700 1 --2- 172.21.15.40:0/1007498533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee70102740 0x7fee70102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.083+0000 7fee751c7700 1 --2- 172.21.15.40:0/1007498533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee701030f0 0x7fee70107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.083+0000 7fee751c7700 1 -- 172.21.15.40:0/1007498533 >> 172.21.15.40:0/1007498533 conn(0x7fee700795f0 msgr2=0x7fee70079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.083+0000 7fee751c7700 1 -- 172.21.15.40:0/1007498533 shutdown_connections 2024-06-06T03:48:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.083+0000 7fee751c7700 1 -- 172.21.15.40:0/1007498533 wait complete. 2024-06-06T03:48:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.084+0000 7fee751c7700 1 Processor -- start 2024-06-06T03:48:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.084+0000 7fee751c7700 1 -- start start 2024-06-06T03:48:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.084+0000 7fee751c7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee70102740 0x7fee7019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.084+0000 7fee751c7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee701030f0 0x7fee7019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.084+0000 7fee751c7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee70107840 0x7fee70198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.084+0000 7fee751c7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fee701127d0 con 0x7fee70102740 2024-06-06T03:48:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.084+0000 7fee751c7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee70112650 con 0x7fee701030f0 2024-06-06T03:48:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.084+0000 7fee751c7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee70112950 con 0x7fee70107840 2024-06-06T03:48:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.084+0000 7fee6ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee70102740 0x7fee7019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.084+0000 7fee6ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee70102740 0x7fee7019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36404/0 (socket says 172.21.15.40:36404) 2024-06-06T03:48:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.084+0000 7fee6ed9d700 1 -- 172.21.15.40:0/1847827447 learned_addr learned my addr 172.21.15.40:0/1847827447 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:48:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.084+0000 7fee6e59c700 1 --2- 172.21.15.40:0/1847827447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee701030f0 0x7fee7019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.085+0000 7fee6f59e700 1 --2- 172.21.15.40:0/1847827447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee70107840 0x7fee70198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.085+0000 7fee6ed9d700 1 -- 172.21.15.40:0/1847827447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee70107840 msgr2=0x7fee70198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.085+0000 7fee6ed9d700 1 --2- 172.21.15.40:0/1847827447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee70107840 0x7fee70198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.085+0000 7fee6ed9d700 1 -- 172.21.15.40:0/1847827447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee701030f0 msgr2=0x7fee7019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.085+0000 7fee6ed9d700 1 --2- 172.21.15.40:0/1847827447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee701030f0 0x7fee7019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.085+0000 7fee6ed9d700 1 -- 172.21.15.40:0/1847827447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee70199170 con 0x7fee70102740 2024-06-06T03:48:46.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.085+0000 7fee6ed9d700 1 --2- 172.21.15.40:0/1847827447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee70102740 0x7fee7019e800 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fee6000d980 tx=0x7fee6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:46.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.085+0000 7fee57fff700 1 -- 172.21.15.40:0/1847827447 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee60016070 con 0x7fee70102740 2024-06-06T03:48:46.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.085+0000 7fee751c7700 1 -- 172.21.15.40:0/1847827447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee70199460 con 0x7fee70102740 2024-06-06T03:48:46.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.085+0000 7fee57fff700 1 -- 172.21.15.40:0/1847827447 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee60004d10 con 0x7fee70102740 2024-06-06T03:48:46.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.085+0000 7fee57fff700 1 -- 172.21.15.40:0/1847827447 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee600055f0 con 0x7fee70102740 2024-06-06T03:48:46.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.085+0000 7fee751c7700 1 -- 172.21.15.40:0/1847827447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee701a56d0 con 0x7fee70102740 2024-06-06T03:48:46.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.086+0000 7fee751c7700 1 -- 172.21.15.40:0/1847827447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee70069180 con 0x7fee70102740 2024-06-06T03:48:46.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.087+0000 7fee57fff700 1 -- 172.21.15.40:0/1847827447 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fee60004750 con 0x7fee70102740 2024-06-06T03:48:46.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.088+0000 7fee57fff700 1 --2- 172.21.15.40:0/1847827447 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fee580a2d90 0x7fee580a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:46.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.088+0000 7fee57fff700 1 -- 172.21.15.40:0/1847827447 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8426+0+0 (secure 0 0 0) 0x7fee600cf830 con 0x7fee70102740 2024-06-06T03:48:46.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.088+0000 7fee6e59c700 1 --2- 172.21.15.40:0/1847827447 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fee580a2d90 0x7fee580a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:46.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.089+0000 7fee6e59c700 1 --2- 172.21.15.40:0/1847827447 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fee580a2d90 0x7fee580a5250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fee70108c00 tx=0x7fee64009190 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:46.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.091+0000 7fee57fff700 1 -- 172.21.15.40:0/1847827447 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fee6001f020 con 0x7fee70102740 2024-06-06T03:48:46.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.344+0000 7fee751c7700 1 -- 172.21.15.40:0/1847827447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee7002dca0 con 0x7fee70102740 2024-06-06T03:48:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.344+0000 7fee57fff700 1 -- 172.21.15.40:0/1847827447 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11961 (secure 0 0 0) 0x7fee6009c450 con 0x7fee70102740 2024-06-06T03:48:46.345 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:48:46.345 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:46:58.889752+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24403,24398,24425,24470,24443]},"id":1}]} 2024-06-06T03:48:46.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.347+0000 7fee751c7700 1 -- 172.21.15.40:0/1847827447 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fee580a2d90 msgr2=0x7fee580a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:46.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.347+0000 7fee751c7700 1 --2- 172.21.15.40:0/1847827447 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fee580a2d90 0x7fee580a5250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fee70108c00 tx=0x7fee64009190 comp rx=0 tx=0).stop 2024-06-06T03:48:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.347+0000 7fee751c7700 1 -- 172.21.15.40:0/1847827447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee70102740 msgr2=0x7fee7019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.347+0000 7fee751c7700 1 --2- 172.21.15.40:0/1847827447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee70102740 0x7fee7019e800 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fee6000d980 tx=0x7fee6000dc90 comp rx=0 tx=0).stop 2024-06-06T03:48:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.347+0000 7fee751c7700 1 -- 172.21.15.40:0/1847827447 shutdown_connections 2024-06-06T03:48:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.347+0000 7fee751c7700 1 --2- 172.21.15.40:0/1847827447 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fee580a2d90 0x7fee580a5250 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.347+0000 7fee751c7700 1 --2- 172.21.15.40:0/1847827447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee70107840 0x7fee70198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.347+0000 7fee751c7700 1 --2- 172.21.15.40:0/1847827447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee70102740 0x7fee7019e800 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.347+0000 7fee751c7700 1 --2- 172.21.15.40:0/1847827447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee701030f0 0x7fee7019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.347+0000 7fee751c7700 1 -- 172.21.15.40:0/1847827447 >> 172.21.15.40:0/1847827447 conn(0x7fee700795f0 msgr2=0x7fee70100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.347+0000 7fee751c7700 1 -- 172.21.15.40:0/1847827447 shutdown_connections 2024-06-06T03:48:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:46.347+0000 7fee751c7700 1 -- 172.21.15.40:0/1847827447 wait complete. 2024-06-06T03:48:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 42 2024-06-06T03:48:47.021 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:46 smithi040 ceph-mon[32220]: pgmap v315: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-06-06T03:48:47.021 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:46 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1847827447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:48:47.049 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --cancel --all' 2024-06-06T03:48:47.049 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --cancel --all 2024-06-06T03:48:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:46 smithi131 ceph-mon[36687]: pgmap v315: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-06-06T03:48:47.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:46 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1847827447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:48:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:48:47.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:46 smithi104 ceph-mon[36569]: pgmap v315: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-06-06T03:48:47.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:46 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1847827447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:48:49.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:48 smithi040 ceph-mon[32220]: pgmap v316: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-06-06T03:48:49.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.214+0000 7f4cf1990700 1 -- 172.21.15.40:0/1633155882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cec107400 msgr2=0x7f4cec1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:49.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.214+0000 7f4cf1990700 1 --2- 172.21.15.40:0/1633155882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cec107400 0x7f4cec1077e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009990 tx=0x7f4cdc009ca0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.214+0000 7f4cf1990700 1 -- 172.21.15.40:0/1633155882 shutdown_connections 2024-06-06T03:48:49.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.214+0000 7f4cf1990700 1 --2- 172.21.15.40:0/1633155882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cec101d50 0x7f4cec10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.214+0000 7f4cf1990700 1 --2- 172.21.15.40:0/1633155882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cec107400 0x7f4cec1077e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.214+0000 7f4cf1990700 1 --2- 172.21.15.40:0/1633155882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cec101390 0x7f4cec101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.214+0000 7f4cf1990700 1 -- 172.21.15.40:0/1633155882 >> 172.21.15.40:0/1633155882 conn(0x7f4cec078340 msgr2=0x7f4cec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.214+0000 7f4cf1990700 1 -- 172.21.15.40:0/1633155882 shutdown_connections 2024-06-06T03:48:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.214+0000 7f4cf1990700 1 -- 172.21.15.40:0/1633155882 wait complete. 2024-06-06T03:48:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.215+0000 7f4cf1990700 1 Processor -- start 2024-06-06T03:48:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.215+0000 7f4cf1990700 1 -- start start 2024-06-06T03:48:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cf1990700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cec101390 0x7f4cec19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cf1990700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cec101d50 0x7f4cec19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cf1990700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cec107400 0x7f4cec197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cf1990700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec1117c0 con 0x7f4cec101d50 2024-06-06T03:48:49.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cf1990700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec111640 con 0x7f4cec101390 2024-06-06T03:48:49.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cf1990700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec111940 con 0x7f4cec107400 2024-06-06T03:48:49.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cea7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cec101d50 0x7f4cec19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cea7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cec101d50 0x7f4cec19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54844/0 (socket says 172.21.15.40:54844) 2024-06-06T03:48:49.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cea7fc700 1 -- 172.21.15.40:0/1792824923 learned_addr learned my addr 172.21.15.40:0/1792824923 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:48:49.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4ceaffd700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cec101390 0x7f4cec19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4ceb7fe700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cec107400 0x7f4cec197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cea7fc700 1 -- 172.21.15.40:0/1792824923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cec107400 msgr2=0x7f4cec197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cea7fc700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cec107400 0x7f4cec197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cea7fc700 1 -- 172.21.15.40:0/1792824923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cec101390 msgr2=0x7f4cec19d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cea7fc700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cec101390 0x7f4cec19d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cea7fc700 1 -- 172.21.15.40:0/1792824923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cec1982a0 con 0x7f4cec101d50 2024-06-06T03:48:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4ceaffd700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cec101390 0x7f4cec19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:48:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.216+0000 7f4cea7fc700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cec101d50 0x7f4cec19de70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000d9e0 tx=0x7f4ce000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.217+0000 7f4cf098e700 1 -- 172.21.15.40:0/1792824923 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce001b070 con 0x7f4cec101d50 2024-06-06T03:48:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.217+0000 7f4cf098e700 1 -- 172.21.15.40:0/1792824923 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce000eb70 con 0x7f4cec101d50 2024-06-06T03:48:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.217+0000 7f4cf1990700 1 -- 172.21.15.40:0/1792824923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cec198590 con 0x7f4cec101d50 2024-06-06T03:48:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.217+0000 7f4cf1990700 1 -- 172.21.15.40:0/1792824923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cec06a170 con 0x7f4cec101d50 2024-06-06T03:48:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.217+0000 7f4cf098e700 1 -- 172.21.15.40:0/1792824923 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00145c0 con 0x7f4cec101d50 2024-06-06T03:48:49.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.219+0000 7f4cf098e700 1 -- 172.21.15.40:0/1792824923 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4ce001e070 con 0x7f4cec101d50 2024-06-06T03:48:49.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.220+0000 7f4cf098e700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4cd40a2d90 0x7f4cd40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.220+0000 7f4ceaffd700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4cd40a2d90 0x7f4cd40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.220+0000 7f4cf098e700 1 -- 172.21.15.40:0/1792824923 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8426+0+0 (secure 0 0 0) 0x7f4ce00d0870 con 0x7f4cec101d50 2024-06-06T03:48:49.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.220+0000 7f4ceaffd700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4cd40a2d90 0x7f4cd40a5250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009750 tx=0x7f4cdc005fd0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:49.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.220+0000 7f4cfc213b80 1 Processor -- start 2024-06-06T03:48:49.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.220+0000 7f4cfc213b80 1 -- start start 2024-06-06T03:48:49.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4cfc213b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db95850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4cfc213b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db95d90 0x560a0db8f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4cfc213b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db8fbc0 0x560a0db8ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4cfc213b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560a0da81fd0 con 0x560a0db95470 2024-06-06T03:48:49.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4cfc213b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560a0dae08c0 con 0x560a0db8fbc0 2024-06-06T03:48:49.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4cfc213b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560a0da484d0 con 0x560a0db95d90 2024-06-06T03:48:49.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4ceaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db95850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4ceaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db95850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54860/0 (socket says 172.21.15.40:54860) 2024-06-06T03:48:49.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4ceaffd700 1 -- 172.21.15.40:0/1794487497 learned_addr learned my addr 172.21.15.40:0/1794487497 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:48:49.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4ceb7fe700 1 --2- 172.21.15.40:0/1794487497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db8fbc0 0x560a0db8ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4cea7fc700 1 --2- 172.21.15.40:0/1794487497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db95d90 0x560a0db8f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4ceaffd700 1 -- 172.21.15.40:0/1794487497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db95d90 msgr2=0x560a0db8f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:48:49.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4ceaffd700 1 --2- 172.21.15.40:0/1794487497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db95d90 0x560a0db8f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4ceaffd700 1 -- 172.21.15.40:0/1794487497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db8fbc0 msgr2=0x560a0db8ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:48:49.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4ceaffd700 1 --2- 172.21.15.40:0/1794487497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db8fbc0 0x560a0db8ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4ceaffd700 1 -- 172.21.15.40:0/1794487497 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a0da2f940 con 0x560a0db95470 2024-06-06T03:48:49.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4ceaffd700 1 --2- 172.21.15.40:0/1794487497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db95850 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00be50 tx=0x7f4cdc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b350b100239fd0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:49.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.221+0000 7f4cd1ffb700 1 -- 172.21.15.40:0/1794487497 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc0050c0 con 0x560a0db95470 2024-06-06T03:48:49.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.222+0000 7f4cd1ffb700 1 -- 172.21.15.40:0/1794487497 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cdc00f460 con 0x560a0db95470 2024-06-06T03:48:49.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.222+0000 7f4cd1ffb700 1 -- 172.21.15.40:0/1794487497 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc024610 con 0x560a0db95470 2024-06-06T03:48:49.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.222+0000 7f4cfc213b80 1 -- 172.21.15.40:0/1794487497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 msgr2=0x560a0db95850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:49.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.222+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/1794487497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db95850 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00be50 tx=0x7f4cdc020000 comp rx=0 tx=0).stop 2024-06-06T03:48:49.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.222+0000 7f4cfc213b80 1 -- 172.21.15.40:0/1794487497 shutdown_connections 2024-06-06T03:48:49.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.222+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/1794487497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db8fbc0 0x560a0db8ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.222+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/1794487497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db95850 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.222+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/1794487497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db95d90 0x560a0db8f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.222+0000 7f4cfc213b80 1 -- 172.21.15.40:0/1794487497 >> 172.21.15.40:0/1794487497 conn(0x560a0dabc0f0 msgr2=0x560a0db8a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:49.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.222+0000 7f4cfc213b80 1 -- 172.21.15.40:0/1794487497 shutdown_connections 2024-06-06T03:48:49.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.222+0000 7f4cfc213b80 1 -- 172.21.15.40:0/1794487497 wait complete. 2024-06-06T03:48:49.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.223+0000 7f4cfc213b80 1 -- start start 2024-06-06T03:48:49.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.223+0000 7f4cfc213b80 1 Processor -- start 2024-06-06T03:48:49.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.223+0000 7f4cfc213b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db8fbc0 0x560a0db9baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.223+0000 7f4cfc213b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db98df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.223+0000 7f4cfc213b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db95d90 0x560a0db99330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.223+0000 7f4cfc213b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560a0da81030 con 0x560a0db95470 2024-06-06T03:48:49.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.223+0000 7f4cfc213b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560a0dadc880 con 0x560a0db95d90 2024-06-06T03:48:49.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.223+0000 7f4cfc213b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560a0db9c710 con 0x560a0db8fbc0 2024-06-06T03:48:49.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.223+0000 7f4cfc213b80 10 client.0 _ll_register_callbacks cb 0x560a0da76620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:48:49.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.223+0000 7f4cea7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db98df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.223+0000 7f4cea7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db98df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54872/0 (socket says 172.21.15.40:54872) 2024-06-06T03:48:49.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.223+0000 7f4cea7fc700 1 -- 172.21.15.40:0/3843116335 learned_addr learned my addr 172.21.15.40:0/3843116335 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:48:49.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.223+0000 7f4ceaffd700 1 --2- 172.21.15.40:0/3843116335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db8fbc0 0x560a0db9baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.223+0000 7f4ceb7fe700 1 --2- 172.21.15.40:0/3843116335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db95d90 0x560a0db99330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.224+0000 7f4cea7fc700 1 -- 172.21.15.40:0/3843116335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db8fbc0 msgr2=0x560a0db9baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:48:49.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.224+0000 7f4cea7fc700 1 --2- 172.21.15.40:0/3843116335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db8fbc0 0x560a0db9baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.224+0000 7f4cea7fc700 1 -- 172.21.15.40:0/3843116335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db95d90 msgr2=0x560a0db99330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:48:49.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.224+0000 7f4cea7fc700 1 --2- 172.21.15.40:0/3843116335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db95d90 0x560a0db99330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.224+0000 7f4cea7fc700 1 -- 172.21.15.40:0/3843116335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a0daa67c0 con 0x560a0db95470 2024-06-06T03:48:49.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.224+0000 7f4cea7fc700 1 --2- 172.21.15.40:0/3843116335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db98df0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000dd60 tx=0x7f4ce0004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab04897f41ae7852 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:49.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.224+0000 7f4ccaffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:48:49.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.224+0000 7f4ccaffd700 1 -- 172.21.15.40:0/3843116335 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce009d310 con 0x560a0db95470 2024-06-06T03:48:49.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.224+0000 7f4ccaffd700 1 -- 172.21.15.40:0/3843116335 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8426+0+0 (secure 0 0 0) 0x7f4ce00a5680 con 0x560a0db95470 2024-06-06T03:48:49.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.224+0000 7f4cfc213b80 1 -- 172.21.15.40:0/3843116335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cdc00f460 con 0x560a0db95470 2024-06-06T03:48:49.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.224+0000 7f4ccaffd700 1 -- 172.21.15.40:0/3843116335 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4ce00a5880 con 0x560a0db95470 2024-06-06T03:48:49.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.224+0000 7f4cfc213b80 10 client.15774 fetch_fsmap learned FSMap version 43 2024-06-06T03:48:49.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.224+0000 7f4cfc213b80 1 -- 172.21.15.40:0/3843116335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560a0dba6f10 con 0x560a0db95470 2024-06-06T03:48:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.224+0000 7f4ccaffd700 1 -- 172.21.15.40:0/3843116335 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce00b9030 con 0x560a0db95470 2024-06-06T03:48:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.225+0000 7f4ccaffd700 1 -- 172.21.15.40:0/3843116335 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 43) ==== 8414+0+0 (secure 0 0 0) 0x7f4ce000e040 con 0x560a0db95470 2024-06-06T03:48:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.225+0000 7f4cfc213b80 10 client.15774 fetch_fsmap finished waiting for FSMap version 43 2024-06-06T03:48:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.225+0000 7f4cfc213b80 10 client.15774 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:48:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.225+0000 7f4cfc213b80 20 client.15774 populate_metadata read hostname 'smithi040' 2024-06-06T03:48:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.225+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/3843116335 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560a0dba94f0 0x560a0dbab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.225+0000 7f4cfc213b80 4 client.15774 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T03:48:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.225+0000 7f4cfc213b80 1 -- 172.21.15.40:0/3843116335 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a0daeddc0 con 0x560a0dba94f0 2024-06-06T03:48:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.225+0000 7f4ceaffd700 1 --2- 172.21.15.40:0/3843116335 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560a0dba94f0 0x560a0dbab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.226+0000 7f4ccaffd700 10 client.15774 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:48:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.226+0000 7f4ceaffd700 1 --2- 172.21.15.40:0/3843116335 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560a0dba94f0 0x560a0dbab8e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d284c7634c6cf5b1 server_cookie=a212cdc578be3f07 in_seq=0 out_seq=0 2024-06-06T03:48:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.226+0000 7f4ccaffd700 1 -- 172.21.15.40:0/3843116335 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560a0daeddc0 con 0x560a0dba94f0 2024-06-06T03:48:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.226+0000 7f4ccaffd700 10 client.15774 handle_command_reply: tid=0 2024-06-06T03:48:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.226+0000 7f4cfc213b80 1 client.15774 shutdown 2024-06-06T03:48:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.226+0000 7f4cfc213b80 1 -- 172.21.15.40:0/3843116335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 msgr2=0x560a0db98df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.226+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/3843116335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db98df0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000dd60 tx=0x7f4ce0004860 comp rx=0 tx=0).stop 2024-06-06T03:48:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.227+0000 7f4cfc213b80 1 -- 172.21.15.40:0/3843116335 shutdown_connections 2024-06-06T03:48:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.227+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/3843116335 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560a0dba94f0 0x560a0dbab8e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.227+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/3843116335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db8fbc0 0x560a0db9baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.227+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/3843116335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db95d90 0x560a0db99330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.227+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/3843116335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db98df0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.227+0000 7f4cfc213b80 1 -- 172.21.15.40:0/3843116335 >> 172.21.15.40:0/3843116335 conn(0x560a0dabc0f0 msgr2=0x560a0db89290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.227+0000 7f4ccaffd700 0 client.15774 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:48:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.227+0000 7f4cfc213b80 1 -- 172.21.15.40:0/3843116335 shutdown_connections 2024-06-06T03:48:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.227+0000 7f4cfc213b80 1 -- 172.21.15.40:0/3843116335 wait complete. 2024-06-06T03:48:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.227+0000 7f4cfc213b80 20 client.15774 trim_cache size 0 max 16384 2024-06-06T03:48:49.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.247+0000 7f4cfc213b80 1 Processor -- start 2024-06-06T03:48:49.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.247+0000 7f4cfc213b80 1 -- start start 2024-06-06T03:48:49.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.247+0000 7f4cfc213b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0dabc0f0 0x560a0db8f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.247+0000 7f4cfc213b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db8fbc0 0x560a0db8ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.247+0000 7f4cfc213b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db96a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.247+0000 7f4cfc213b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec1aaae0 con 0x560a0db95470 2024-06-06T03:48:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.247+0000 7f4cfc213b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec1ac830 con 0x560a0db8fbc0 2024-06-06T03:48:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.247+0000 7f4cfc213b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec1ab270 con 0x560a0dabc0f0 2024-06-06T03:48:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.247+0000 7f4ceb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db96a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.247+0000 7f4ceb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db96a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54874/0 (socket says 172.21.15.40:54874) 2024-06-06T03:48:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.247+0000 7f4ceb7fe700 1 -- 172.21.15.40:0/2210747682 learned_addr learned my addr 172.21.15.40:0/2210747682 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:48:49.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.247+0000 7f4ceaffd700 1 --2- 172.21.15.40:0/2210747682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0dabc0f0 0x560a0db8f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.247+0000 7f4cea7fc700 1 --2- 172.21.15.40:0/2210747682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db8fbc0 0x560a0db8ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.248+0000 7f4ceb7fe700 1 -- 172.21.15.40:0/2210747682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0dabc0f0 msgr2=0x560a0db8f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:48:49.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.248+0000 7f4ceb7fe700 1 --2- 172.21.15.40:0/2210747682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0dabc0f0 0x560a0db8f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.248+0000 7f4ceb7fe700 1 -- 172.21.15.40:0/2210747682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db8fbc0 msgr2=0x560a0db8ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:48:49.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.248+0000 7f4ceb7fe700 1 --2- 172.21.15.40:0/2210747682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db8fbc0 0x560a0db8ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.248+0000 7f4ceb7fe700 1 -- 172.21.15.40:0/2210747682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a0dbabc20 con 0x560a0db95470 2024-06-06T03:48:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.248+0000 7f4ceb7fe700 1 --2- 172.21.15.40:0/2210747682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db96a00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400a530 tx=0x7f4ce4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec551a9fdd3d56ce server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.248+0000 7f4cd17fa700 1 -- 172.21.15.40:0/2210747682 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4008030 con 0x560a0db95470 2024-06-06T03:48:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.248+0000 7f4cd17fa700 1 -- 172.21.15.40:0/2210747682 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce4017b00 con 0x560a0db95470 2024-06-06T03:48:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.248+0000 7f4cd17fa700 1 -- 172.21.15.40:0/2210747682 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce401fc50 con 0x560a0db95470 2024-06-06T03:48:49.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.249+0000 7f4cfc213b80 1 -- 172.21.15.40:0/2210747682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 msgr2=0x560a0db96a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.249+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/2210747682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db96a00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400a530 tx=0x7f4ce4009650 comp rx=0 tx=0).stop 2024-06-06T03:48:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.249+0000 7f4cfc213b80 1 -- 172.21.15.40:0/2210747682 shutdown_connections 2024-06-06T03:48:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.249+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/2210747682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0dabc0f0 0x560a0db8f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.249+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/2210747682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db95470 0x560a0db96a00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.249+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/2210747682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0db8fbc0 0x560a0db8ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.249+0000 7f4cfc213b80 1 -- 172.21.15.40:0/2210747682 >> 172.21.15.40:0/2210747682 conn(0x560a0db9bc00 msgr2=0x560a0db9a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.249+0000 7f4cfc213b80 1 -- 172.21.15.40:0/2210747682 shutdown_connections 2024-06-06T03:48:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.249+0000 7f4cfc213b80 1 -- 172.21.15.40:0/2210747682 wait complete. 2024-06-06T03:48:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.249+0000 7f4cfc213b80 1 -- start start 2024-06-06T03:48:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.249+0000 7f4cfc213b80 1 Processor -- start 2024-06-06T03:48:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cfc213b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0dabc0f0 0x560a0dba1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cfc213b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db8fbc0 0x560a0dba1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cfc213b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db95470 0x560a0db9e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cfc213b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec1ac0a0 con 0x560a0db8fbc0 2024-06-06T03:48:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cfc213b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec1ab940 con 0x560a0dabc0f0 2024-06-06T03:48:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cfc213b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec1a8aa0 con 0x560a0db95470 2024-06-06T03:48:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cfc213b80 10 client.0 _ll_register_callbacks cb 0x560a0da76620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T03:48:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cea7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db8fbc0 0x560a0dba1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cea7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db8fbc0 0x560a0dba1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54888/0 (socket says 172.21.15.40:54888) 2024-06-06T03:48:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cea7fc700 1 -- 172.21.15.40:0/510831214 learned_addr learned my addr 172.21.15.40:0/510831214 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:48:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4ceb7fe700 1 --2- 172.21.15.40:0/510831214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db95470 0x560a0db9e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4ceaffd700 1 --2- 172.21.15.40:0/510831214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0dabc0f0 0x560a0dba1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cea7fc700 1 -- 172.21.15.40:0/510831214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db95470 msgr2=0x560a0db9e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:48:49.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cea7fc700 1 --2- 172.21.15.40:0/510831214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db95470 0x560a0db9e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cea7fc700 1 -- 172.21.15.40:0/510831214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0dabc0f0 msgr2=0x560a0dba1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T03:48:49.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cea7fc700 1 --2- 172.21.15.40:0/510831214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0dabc0f0 0x560a0dba1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cea7fc700 1 -- 172.21.15.40:0/510831214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a0db9ef10 con 0x560a0db8fbc0 2024-06-06T03:48:49.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4ceb7fe700 1 --2- 172.21.15.40:0/510831214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db95470 0x560a0db9e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:48:49.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4cea7fc700 1 --2- 172.21.15.40:0/510831214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db8fbc0 0x560a0dba1f40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0099870 tx=0x7f4ce0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2922ef06c61c40 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:49.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4ccaffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-06-06T03:48:49.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.250+0000 7f4ccaffd700 1 -- 172.21.15.40:0/510831214 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce009cea0 con 0x560a0db8fbc0 2024-06-06T03:48:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.251+0000 7f4ccaffd700 1 -- 172.21.15.40:0/510831214 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8426+0+0 (secure 0 0 0) 0x7f4ce00a5680 con 0x560a0db8fbc0 2024-06-06T03:48:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.251+0000 7f4cfc213b80 1 -- 172.21.15.40:0/510831214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce4017b00 con 0x560a0db8fbc0 2024-06-06T03:48:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.251+0000 7f4ccaffd700 1 -- 172.21.15.40:0/510831214 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4ce0098e80 con 0x560a0db8fbc0 2024-06-06T03:48:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.251+0000 7f4cfc213b80 10 client.15780 fetch_fsmap learned FSMap version 43 2024-06-06T03:48:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.251+0000 7f4cfc213b80 1 -- 172.21.15.40:0/510831214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560a0dc18ca0 con 0x560a0db8fbc0 2024-06-06T03:48:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.251+0000 7f4ccaffd700 1 -- 172.21.15.40:0/510831214 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce4017b00 con 0x560a0db8fbc0 2024-06-06T03:48:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.251+0000 7f4ccaffd700 1 -- 172.21.15.40:0/510831214 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 43) ==== 8414+0+0 (secure 0 0 0) 0x7f4ce0082020 con 0x560a0db8fbc0 2024-06-06T03:48:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.251+0000 7f4cfc213b80 10 client.15780 fetch_fsmap finished waiting for FSMap version 43 2024-06-06T03:48:49.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.251+0000 7f4cfc213b80 10 client.15780 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-06T03:48:49.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.251+0000 7f4cfc213b80 20 client.15780 populate_metadata read hostname 'smithi040' 2024-06-06T03:48:49.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.251+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/510831214 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560a0db95d90 0x560a0dc1b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:49.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.251+0000 7f4cfc213b80 4 client.15780 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-06-06T03:48:49.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.251+0000 7f4cfc213b80 1 -- 172.21.15.40:0/510831214 --> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x560a0dae8340 con 0x560a0db95d90 2024-06-06T03:48:49.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.251+0000 7f4ceaffd700 1 --2- 172.21.15.40:0/510831214 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560a0db95d90 0x560a0dc1b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:49.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4ceaffd700 1 --2- 172.21.15.40:0/510831214 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560a0db95d90 0x560a0dc1b240 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d216184908d3c816 server_cookie=29b4fde2ac3cf90d in_seq=0 out_seq=0 2024-06-06T03:48:49.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4ccaffd700 10 client.15780 ms_handle_connect on v2:172.21.15.104:6838/489480648 2024-06-06T03:48:49.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4ccaffd700 1 -- 172.21.15.40:0/510831214 <== mds.0 v2:172.21.15.104:6838/489480648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x560a0dae8340 con 0x560a0db95d90 2024-06-06T03:48:49.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4ccaffd700 10 client.15780 handle_command_reply: tid=0 2024-06-06T03:48:49.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4cfc213b80 1 client.15780 shutdown 2024-06-06T03:48:49.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4cfc213b80 1 -- 172.21.15.40:0/510831214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db8fbc0 msgr2=0x560a0dba1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:49.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/510831214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db8fbc0 0x560a0dba1f40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0099870 tx=0x7f4ce0004750 comp rx=0 tx=0).stop 2024-06-06T03:48:49.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4cfc213b80 1 -- 172.21.15.40:0/510831214 shutdown_connections 2024-06-06T03:48:49.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/510831214 >> [v2:172.21.15.104:6838/489480648,v1:172.21.15.104:6839/489480648] conn(0x560a0db95d90 0x560a0dc1b240 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/510831214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0db95470 0x560a0db9e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/510831214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560a0dabc0f0 0x560a0dba1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4ccaffd700 0 client.15780 ms_handle_reset on v2:172.21.15.104:6838/489480648 2024-06-06T03:48:49.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4cfc213b80 1 --2- 172.21.15.40:0/510831214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a0db8fbc0 0x560a0dba1f40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4cfc213b80 1 -- 172.21.15.40:0/510831214 >> 172.21.15.40:0/510831214 conn(0x560a0db9bc00 msgr2=0x560a0db9ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:49.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4cfc213b80 1 -- 172.21.15.40:0/510831214 shutdown_connections 2024-06-06T03:48:49.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.252+0000 7f4cfc213b80 1 -- 172.21.15.40:0/510831214 wait complete. 2024-06-06T03:48:49.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.253+0000 7f4cfc213b80 20 client.15780 trim_cache size 0 max 16384 2024-06-06T03:48:49.263 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:48:49.263 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 43, 2024-06-06T03:48:49.263 INFO:teuthology.orchestra.run.smithi040.stdout: "leader": 24470, 2024-06-06T03:48:49.263 INFO:teuthology.orchestra.run.smithi040.stdout: "set_version": 28, 2024-06-06T03:48:49.263 INFO:teuthology.orchestra.run.smithi040.stdout: "sets": {} 2024-06-06T03:48:49.263 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:48:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.253+0000 7f4cf1990700 1 -- 172.21.15.40:0/1792824923 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4cd40a2d90 msgr2=0x7f4cd40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.253+0000 7f4cf1990700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4cd40a2d90 0x7f4cd40a5250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009750 tx=0x7f4cdc005fd0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.253+0000 7f4cf1990700 1 -- 172.21.15.40:0/1792824923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cec101d50 msgr2=0x7f4cec19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.253+0000 7f4cf1990700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cec101d50 0x7f4cec19de70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000d9e0 tx=0x7f4ce000dcf0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.253+0000 7f4cf1990700 1 -- 172.21.15.40:0/1792824923 shutdown_connections 2024-06-06T03:48:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.253+0000 7f4cf1990700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f4cd40a2d90 0x7f4cd40a5250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.253+0000 7f4cf1990700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cec107400 0x7f4cec197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.253+0000 7f4cf1990700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cec101390 0x7f4cec19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.253+0000 7f4cf1990700 1 --2- 172.21.15.40:0/1792824923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cec101d50 0x7f4cec19de70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.253+0000 7f4cf1990700 1 -- 172.21.15.40:0/1792824923 >> 172.21.15.40:0/1792824923 conn(0x7f4cec078340 msgr2=0x7f4cec1000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.253+0000 7f4cf1990700 1 -- 172.21.15.40:0/1792824923 shutdown_connections 2024-06-06T03:48:49.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:49.253+0000 7f4cf1990700 1 -- 172.21.15.40:0/1792824923 wait complete. 2024-06-06T03:48:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:48 smithi131 ceph-mon[36687]: pgmap v316: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-06-06T03:48:49.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:48 smithi104 ceph-mon[36569]: pgmap v316: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-06-06T03:48:50.022 INFO:tasks.quiescer:done joining Quiescers 2024-06-06T03:48:50.022 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-06-06T03:48:50.032 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-06-06T03:48:50.033 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T03:48:50.033 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi040.front.sepia.ceph.com 2024-06-06T03:48:50.033 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-06-06T03:48:50.061 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:49 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:48:50.065 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-06-06T03:48:50.135 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-06-06T03:48:50.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.282+0000 7fabc79d5640 1 -- 172.21.15.40:0/3087583755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc01068b0 msgr2=0x7fabc0106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:50.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.282+0000 7fabc79d5640 1 --2- 172.21.15.40:0/3087583755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc01068b0 0x7fabc0106c90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009930 tx=0x7fabbc02a420 comp rx=0 tx=0).stop 2024-06-06T03:48:50.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.282+0000 7fabc79d5640 1 -- 172.21.15.40:0/3087583755 shutdown_connections 2024-06-06T03:48:50.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.282+0000 7fabc79d5640 1 --2- 172.21.15.40:0/3087583755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabc0101260 0x7fabc010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.282+0000 7fabc79d5640 1 --2- 172.21.15.40:0/3087583755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabc01008a0 0x7fabc0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.282+0000 7fabc79d5640 1 --2- 172.21.15.40:0/3087583755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc01068b0 0x7fabc0106c90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.282+0000 7fabc79d5640 1 -- 172.21.15.40:0/3087583755 >> 172.21.15.40:0/3087583755 conn(0x7fabc0077a20 msgr2=0x7fabc0077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:50.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.282+0000 7fabc79d5640 1 -- 172.21.15.40:0/3087583755 shutdown_connections 2024-06-06T03:48:50.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.282+0000 7fabc79d5640 1 -- 172.21.15.40:0/3087583755 wait complete. 2024-06-06T03:48:50.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.282+0000 7fabc79d5640 1 Processor -- start 2024-06-06T03:48:50.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc79d5640 1 -- start start 2024-06-06T03:48:50.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc79d5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabc01008a0 0x7fabc0199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:50.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc79d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc0101260 0x7fabc0199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:50.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc79d5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabc01068b0 0x7fabc01a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:50.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc79d5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fabc0110c80 con 0x7fabc0101260 2024-06-06T03:48:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc79d5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fabc0110b00 con 0x7fabc01008a0 2024-06-06T03:48:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc79d5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fabc0110e00 con 0x7fabc01068b0 2024-06-06T03:48:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc4f70640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc0101260 0x7fabc0199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc4f70640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc0101260 0x7fabc0199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54900/0 (socket says 172.21.15.40:54900) 2024-06-06T03:48:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc4f70640 1 -- 172.21.15.40:0/4104357463 learned_addr learned my addr 172.21.15.40:0/4104357463 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:48:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc5f72640 1 --2- 172.21.15.40:0/4104357463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabc01068b0 0x7fabc01a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc5771640 1 --2- 172.21.15.40:0/4104357463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabc01008a0 0x7fabc0199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc4f70640 1 -- 172.21.15.40:0/4104357463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabc01068b0 msgr2=0x7fabc01a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc4f70640 1 --2- 172.21.15.40:0/4104357463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabc01068b0 0x7fabc01a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc4f70640 1 -- 172.21.15.40:0/4104357463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabc01008a0 msgr2=0x7fabc0199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc4f70640 1 --2- 172.21.15.40:0/4104357463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabc01008a0 0x7fabc0199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc4f70640 1 -- 172.21.15.40:0/4104357463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc01a45c0 con 0x7fabc0101260 2024-06-06T03:48:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc4f70640 1 --2- 172.21.15.40:0/4104357463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc0101260 0x7fabc0199540 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fabb400e9f0 tx=0x7fabb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabb27fc640 1 -- 172.21.15.40:0/4104357463 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb400cd80 con 0x7fabc0101260 2024-06-06T03:48:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabb27fc640 1 -- 172.21.15.40:0/4104357463 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb4010910 con 0x7fabc0101260 2024-06-06T03:48:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.283+0000 7fabc79d5640 1 -- 172.21.15.40:0/4104357463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc01a4790 con 0x7fabc0101260 2024-06-06T03:48:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.284+0000 7fabb27fc640 1 -- 172.21.15.40:0/4104357463 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4018990 con 0x7fabc0101260 2024-06-06T03:48:50.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.284+0000 7fabc79d5640 1 -- 172.21.15.40:0/4104357463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc01a4d00 con 0x7fabc0101260 2024-06-06T03:48:50.289 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:49 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:48:50.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.285+0000 7fabc79d5640 1 -- 172.21.15.40:0/4104357463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc004f760 con 0x7fabc0101260 2024-06-06T03:48:50.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.286+0000 7fabb27fc640 1 -- 172.21.15.40:0/4104357463 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fabb4010ac0 con 0x7fabc0101260 2024-06-06T03:48:50.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.288+0000 7fabb27fc640 1 --2- 172.21.15.40:0/4104357463 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fabac0a2e20 0x7fabac0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:50.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.288+0000 7fabc5771640 1 --2- 172.21.15.40:0/4104357463 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fabac0a2e20 0x7fabac0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:50.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.288+0000 7fabb27fc640 1 -- 172.21.15.40:0/4104357463 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8426+0+0 (secure 0 0 0) 0x7fabb40cf7b0 con 0x7fabc0101260 2024-06-06T03:48:50.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.288+0000 7fabc5771640 1 --2- 172.21.15.40:0/4104357463 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fabac0a2e20 0x7fabac0a52e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fabbc000c00 tx=0x7fabbc005a20 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:50.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.292+0000 7fabb27fc640 1 -- 172.21.15.40:0/4104357463 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fabb409c430 con 0x7fabc0101260 2024-06-06T03:48:50.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:49 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:48:50.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.498+0000 7fabc79d5640 1 -- 172.21.15.40:0/4104357463 --> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fabc0061de0 con 0x7fabac0a2e20 2024-06-06T03:48:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.506+0000 7fabb27fc640 1 -- 172.21.15.40:0/4104357463 <== mgr.24790 v2:172.21.15.40:6800/1568651190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fabc0061de0 con 0x7fabac0a2e20 2024-06-06T03:48:50.506 INFO:teuthology.orchestra.run.smithi040.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-06-06T00:00:00", "created": "2024-06-06T03:38:38", "first": "2024-06-06T03:39:00", "last": "2024-06-06T03:48:00", "last_pruned": "2024-06-06T03:48:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-06-06T03:48:50.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.508+0000 7fabc79d5640 1 -- 172.21.15.40:0/4104357463 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fabac0a2e20 msgr2=0x7fabac0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:50.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.508+0000 7fabc79d5640 1 --2- 172.21.15.40:0/4104357463 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fabac0a2e20 0x7fabac0a52e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fabbc000c00 tx=0x7fabbc005a20 comp rx=0 tx=0).stop 2024-06-06T03:48:50.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.508+0000 7fabc79d5640 1 -- 172.21.15.40:0/4104357463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc0101260 msgr2=0x7fabc0199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:50.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.508+0000 7fabc79d5640 1 --2- 172.21.15.40:0/4104357463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc0101260 0x7fabc0199540 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fabb400e9f0 tx=0x7fabb400eeb0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.508+0000 7fabc79d5640 1 -- 172.21.15.40:0/4104357463 shutdown_connections 2024-06-06T03:48:50.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.508+0000 7fabc79d5640 1 --2- 172.21.15.40:0/4104357463 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fabac0a2e20 0x7fabac0a52e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.508+0000 7fabc79d5640 1 --2- 172.21.15.40:0/4104357463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabc01068b0 0x7fabc01a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.508+0000 7fabc79d5640 1 --2- 172.21.15.40:0/4104357463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc0101260 0x7fabc0199540 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.508+0000 7fabc79d5640 1 --2- 172.21.15.40:0/4104357463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabc01008a0 0x7fabc0199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.508+0000 7fabc79d5640 1 -- 172.21.15.40:0/4104357463 >> 172.21.15.40:0/4104357463 conn(0x7fabc0077a20 msgr2=0x7fabc01033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:50.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.508+0000 7fabc79d5640 1 -- 172.21.15.40:0/4104357463 shutdown_connections 2024-06-06T03:48:50.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.508+0000 7fabc79d5640 1 -- 172.21.15.40:0/4104357463 wait complete. 2024-06-06T03:48:50.519 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-06-06T03:48:50.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.626+0000 7f83c6c09640 1 -- 172.21.15.40:0/1010508370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102340 msgr2=0x7f83c010e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:50.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.626+0000 7f83c6c09640 1 --2- 172.21.15.40:0/1010508370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102340 0x7f83c010e850 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009cd0 tx=0x7f83b802a8e0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.626+0000 7f83c6c09640 1 -- 172.21.15.40:0/1010508370 shutdown_connections 2024-06-06T03:48:50.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.626+0000 7f83c6c09640 1 --2- 172.21.15.40:0/1010508370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102340 0x7f83c010e850 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.626+0000 7f83c6c09640 1 --2- 172.21.15.40:0/1010508370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83c0101980 0x7f83c0101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.626+0000 7f83c6c09640 1 --2- 172.21.15.40:0/1010508370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83c0107990 0x7f83c0107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.626+0000 7f83c6c09640 1 -- 172.21.15.40:0/1010508370 >> 172.21.15.40:0/1010508370 conn(0x7f83c0078e30 msgr2=0x7f83c0079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:50.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.626+0000 7f83c6c09640 1 -- 172.21.15.40:0/1010508370 shutdown_connections 2024-06-06T03:48:50.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.626+0000 7f83c6c09640 1 -- 172.21.15.40:0/1010508370 wait complete. 2024-06-06T03:48:50.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83c6c09640 1 Processor -- start 2024-06-06T03:48:50.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83c6c09640 1 -- start start 2024-06-06T03:48:50.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83c6c09640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83c0101980 0x7f83c01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:50.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83c6c09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102340 0x7f83c0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:50.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83c6c09640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83c0107990 0x7f83c019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:50.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83c6c09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f83c0111da0 con 0x7f83c0102340 2024-06-06T03:48:50.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83c6c09640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f83c0111c20 con 0x7f83c0101980 2024-06-06T03:48:50.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83c6c09640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f83c0111f20 con 0x7f83c0107990 2024-06-06T03:48:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102340 0x7f83c0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102340 0x7f83c0198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54920/0 (socket says 172.21.15.40:54920) 2024-06-06T03:48:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83b7fff640 1 -- 172.21.15.40:0/4163057316 learned_addr learned my addr 172.21.15.40:0/4163057316 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:48:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83c49a5640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83c0101980 0x7f83c01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83b7fff640 1 -- 172.21.15.40:0/4163057316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83c0107990 msgr2=0x7f83c019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83c51a6640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83c0107990 0x7f83c019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.627+0000 7f83b7fff640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83c0107990 0x7f83c019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.628+0000 7f83b7fff640 1 -- 172.21.15.40:0/4163057316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83c0101980 msgr2=0x7f83c01981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.628+0000 7f83b7fff640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83c0101980 0x7f83c01981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.628+0000 7f83b7fff640 1 -- 172.21.15.40:0/4163057316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c019d1b0 con 0x7f83c0102340 2024-06-06T03:48:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.628+0000 7f83c49a5640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83c0101980 0x7f83c01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:48:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.628+0000 7f83b7fff640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102340 0x7f83c0198700 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f83b000c890 tx=0x7f83b000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.628+0000 7f83b5ffb640 1 -- 172.21.15.40:0/4163057316 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0004220 con 0x7f83c0102340 2024-06-06T03:48:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.628+0000 7f83b5ffb640 1 -- 172.21.15.40:0/4163057316 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b00043c0 con 0x7f83c0102340 2024-06-06T03:48:50.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.628+0000 7f83c6c09640 1 -- 172.21.15.40:0/4163057316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c019d4a0 con 0x7f83c0102340 2024-06-06T03:48:50.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.628+0000 7f83b5ffb640 1 -- 172.21.15.40:0/4163057316 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b00049e0 con 0x7f83c0102340 2024-06-06T03:48:50.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.628+0000 7f83c6c09640 1 -- 172.21.15.40:0/4163057316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c01a4d30 con 0x7f83c0102340 2024-06-06T03:48:50.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.629+0000 7f83c6c09640 1 -- 172.21.15.40:0/4163057316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c004f760 con 0x7f83c0102340 2024-06-06T03:48:50.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.630+0000 7f83b5ffb640 1 -- 172.21.15.40:0/4163057316 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f83b00079b0 con 0x7f83c0102340 2024-06-06T03:48:50.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.631+0000 7f83b5ffb640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f83a80a2e20 0x7f83a80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:50.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.631+0000 7f83b5ffb640 1 -- 172.21.15.40:0/4163057316 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8426+0+0 (secure 0 0 0) 0x7f83b00cf480 con 0x7f83c0102340 2024-06-06T03:48:50.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.631+0000 7f83c49a5640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f83a80a2e20 0x7f83a80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:50.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.631+0000 7f83c49a5640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f83a80a2e20 0x7f83a80a52e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f83bc005fd0 tx=0x7f83bc0094d0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:50.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.635+0000 7f83b5ffb640 1 -- 172.21.15.40:0/4163057316 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f83b0014030 con 0x7f83c0102340 2024-06-06T03:48:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.843+0000 7f83c6c09640 1 -- 172.21.15.40:0/4163057316 --> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f83c00ff8c0 con 0x7f83a80a2e20 2024-06-06T03:48:50.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.851+0000 7f83b5ffb640 1 -- 172.21.15.40:0/4163057316 <== mgr.24790 v2:172.21.15.40:6800/1568651190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f83c00ff8c0 con 0x7f83a80a2e20 2024-06-06T03:48:50.851 INFO:teuthology.orchestra.run.smithi040.stdout:/ 1m 6m3h 2024-06-06T03:48:50.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.854+0000 7f83c6c09640 1 -- 172.21.15.40:0/4163057316 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f83a80a2e20 msgr2=0x7f83a80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:50.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.854+0000 7f83c6c09640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f83a80a2e20 0x7f83a80a52e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f83bc005fd0 tx=0x7f83bc0094d0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.854+0000 7f83c6c09640 1 -- 172.21.15.40:0/4163057316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102340 msgr2=0x7f83c0198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.854+0000 7f83c6c09640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102340 0x7f83c0198700 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f83b000c890 tx=0x7f83b000cd50 comp rx=0 tx=0).stop 2024-06-06T03:48:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.854+0000 7f83c6c09640 1 -- 172.21.15.40:0/4163057316 shutdown_connections 2024-06-06T03:48:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.854+0000 7f83c6c09640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f83a80a2e20 0x7f83a80a52e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.854+0000 7f83c6c09640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83c0107990 0x7f83c019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.854+0000 7f83c6c09640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c0102340 0x7f83c0198700 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.854+0000 7f83c6c09640 1 --2- 172.21.15.40:0/4163057316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83c0101980 0x7f83c01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.854+0000 7f83c6c09640 1 -- 172.21.15.40:0/4163057316 >> 172.21.15.40:0/4163057316 conn(0x7f83c0078e30 msgr2=0x7f83c00ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:50.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.854+0000 7f83c6c09640 1 -- 172.21.15.40:0/4163057316 shutdown_connections 2024-06-06T03:48:50.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.854+0000 7f83c6c09640 1 -- 172.21.15.40:0/4163057316 wait complete. 2024-06-06T03:48:50.864 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-06T03:48:50.874 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-06T03:48:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.980+0000 7f2149b9e640 1 -- 172.21.15.40:0/2268794693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2144102c80 msgr2=0x7f2144103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.980+0000 7f2149b9e640 1 --2- 172.21.15.40:0/2268794693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2144102c80 0x7f2144103060 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2134009ca0 tx=0x7f213402a950 comp rx=0 tx=0).stop 2024-06-06T03:48:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.980+0000 7f2149b9e640 1 -- 172.21.15.40:0/2268794693 shutdown_connections 2024-06-06T03:48:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.980+0000 7f2149b9e640 1 --2- 172.21.15.40:0/2268794693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f214410bfc0 0x7f214410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.980+0000 7f2149b9e640 1 --2- 172.21.15.40:0/2268794693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21441035a0 0x7f214410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.980+0000 7f2149b9e640 1 --2- 172.21.15.40:0/2268794693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2144102c80 0x7f2144103060 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.980+0000 7f2149b9e640 1 -- 172.21.15.40:0/2268794693 >> 172.21.15.40:0/2268794693 conn(0x7f2144077a00 msgr2=0x7f2144077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:50.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.980+0000 7f2149b9e640 1 -- 172.21.15.40:0/2268794693 shutdown_connections 2024-06-06T03:48:50.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.980+0000 7f2149b9e640 1 -- 172.21.15.40:0/2268794693 wait complete. 2024-06-06T03:48:50.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.980+0000 7f2149b9e640 1 Processor -- start 2024-06-06T03:48:50.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2149b9e640 1 -- start start 2024-06-06T03:48:50.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2149b9e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2144102c80 0x7f2144198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:50.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2149b9e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21441035a0 0x7f2144198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:50.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2149b9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f214410bfc0 0x7f214419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:50.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2149b9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2144111dd0 con 0x7f214410bfc0 2024-06-06T03:48:50.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2149b9e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2144111c50 con 0x7f21441035a0 2024-06-06T03:48:50.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2149b9e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2144111f50 con 0x7f2144102c80 2024-06-06T03:48:50.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2143fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f214410bfc0 0x7f214419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:50.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2143fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f214410bfc0 0x7f214419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54938/0 (socket says 172.21.15.40:54938) 2024-06-06T03:48:50.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2143fff640 1 -- 172.21.15.40:0/1571594110 learned_addr learned my addr 172.21.15.40:0/1571594110 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:48:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f21437fe640 1 --2- 172.21.15.40:0/1571594110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2144102c80 0x7f2144198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2142ffd640 1 --2- 172.21.15.40:0/1571594110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21441035a0 0x7f2144198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2143fff640 1 -- 172.21.15.40:0/1571594110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2144102c80 msgr2=0x7f2144198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2143fff640 1 --2- 172.21.15.40:0/1571594110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2144102c80 0x7f2144198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2143fff640 1 -- 172.21.15.40:0/1571594110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21441035a0 msgr2=0x7f2144198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2143fff640 1 --2- 172.21.15.40:0/1571594110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21441035a0 0x7f2144198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.981+0000 7f2143fff640 1 -- 172.21.15.40:0/1571594110 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214419d300 con 0x7f214410bfc0 2024-06-06T03:48:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.982+0000 7f2143fff640 1 --2- 172.21.15.40:0/1571594110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f214410bfc0 0x7f214419cbd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f213c00c720 tx=0x7f213c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.982+0000 7f2140ff9640 1 -- 172.21.15.40:0/1571594110 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c007bb0 con 0x7f214410bfc0 2024-06-06T03:48:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.982+0000 7f2149b9e640 1 -- 172.21.15.40:0/1571594110 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214419d5f0 con 0x7f214410bfc0 2024-06-06T03:48:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.982+0000 7f2140ff9640 1 -- 172.21.15.40:0/1571594110 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f213c00ce30 con 0x7f214410bfc0 2024-06-06T03:48:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.982+0000 7f2140ff9640 1 -- 172.21.15.40:0/1571594110 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c004e30 con 0x7f214410bfc0 2024-06-06T03:48:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.982+0000 7f2149b9e640 1 -- 172.21.15.40:0/1571594110 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21441a4e80 con 0x7f214410bfc0 2024-06-06T03:48:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.983+0000 7f2149b9e640 1 -- 172.21.15.40:0/1571594110 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2144068c10 con 0x7f214410bfc0 2024-06-06T03:48:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.984+0000 7f2140ff9640 1 -- 172.21.15.40:0/1571594110 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f213c005000 con 0x7f214410bfc0 2024-06-06T03:48:50.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.985+0000 7f2140ff9640 1 --2- 172.21.15.40:0/1571594110 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f212c0a2d00 0x7f212c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:50.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.985+0000 7f21437fe640 1 --2- 172.21.15.40:0/1571594110 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f212c0a2d00 0x7f212c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:50.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.985+0000 7f2140ff9640 1 -- 172.21.15.40:0/1571594110 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8426+0+0 (secure 0 0 0) 0x7f213c0cf3c0 con 0x7f214410bfc0 2024-06-06T03:48:50.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.985+0000 7f21437fe640 1 --2- 172.21.15.40:0/1571594110 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f212c0a2d00 0x7f212c0a51c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f21340093d0 tx=0x7f2134009360 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:50.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:50.988+0000 7f2140ff9640 1 -- 172.21.15.40:0/1571594110 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f213c014030 con 0x7f214410bfc0 2024-06-06T03:48:51.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.173+0000 7f2149b9e640 1 -- 172.21.15.40:0/1571594110 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21441071a0 con 0x7f214410bfc0 2024-06-06T03:48:51.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.174+0000 7f2140ff9640 1 -- 172.21.15.40:0/1571594110 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+22999 (secure 0 0 0) 0x7f213c09c040 con 0x7f214410bfc0 2024-06-06T03:48:51.174 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:48:51.174 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":99,"fsid":"77c25d68-23b4-11ef-bc9b-c7b262605968","created":"2024-06-06T03:26:53.165489+0000","modified":"2024-06-06T03:48:13.804352+0000","last_up_change":"2024-06-06T03:34:07.114448+0000","last_in_change":"2024-06-06T03:33:46.152948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T03:30:14.102785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T03:35:08.521025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T03:35:12.369035+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63c0a320-a2f2-4b84-87e7-68501d335285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6803","nonce":3453526793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6805","nonce":3453526793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6809","nonce":3453526793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6807","nonce":3453526793}]},"public_addr":"172.21.15.40:6803/3453526793","cluster_addr":"172.21.15.40:6805/3453526793","heartbeat_back_addr":"172.21.15.40:6809/3453526793","heartbeat_front_addr":"172.21.15.40:6807/3453526793","state":["exists","up"]},{"osd":1,"uuid":"70a31534-06a1-4ffa-a54d-577d45bccf98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6801","nonce":405854252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6803","nonce":405854252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6807","nonce":405854252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6805","nonce":405854252}]},"public_addr":"172.21.15.104:6801/405854252","cluster_addr":"172.21.15.104:6803/405854252","heartbeat_back_addr":"172.21.15.104:6807/405854252","heartbeat_front_addr":"172.21.15.104:6805/405854252","state":["exists","up"]},{"osd":2,"uuid":"f5a89f43-e533-45f0-a777-bd3e9bfa8072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6801","nonce":3919221141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6803","nonce":3919221141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6807","nonce":3919221141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6805","nonce":3919221141}]},"public_addr":"172.21.15.131:6801/3919221141","cluster_addr":"172.21.15.131:6803/3919221141","heartbeat_back_addr":"172.21.15.131:6807/3919221141","heartbeat_front_addr":"172.21.15.131:6805/3919221141","state":["exists","up"]},{"osd":3,"uuid":"4df19390-6378-4346-a2be-29bce1dd2380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6811","nonce":2008541522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6813","nonce":2008541522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6817","nonce":2008541522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6815","nonce":2008541522}]},"public_addr":"172.21.15.40:6811/2008541522","cluster_addr":"172.21.15.40:6813/2008541522","heartbeat_back_addr":"172.21.15.40:6817/2008541522","heartbeat_front_addr":"172.21.15.40:6815/2008541522","state":["exists","up"]},{"osd":4,"uuid":"508de408-1cb1-4423-8199-db6bde9034f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6809","nonce":1153394847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6811","nonce":1153394847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6815","nonce":1153394847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6813","nonce":1153394847}]},"public_addr":"172.21.15.104:6809/1153394847","cluster_addr":"172.21.15.104:6811/1153394847","heartbeat_back_addr":"172.21.15.104:6815/1153394847","heartbeat_front_addr":"172.21.15.104:6813/1153394847","state":["exists","up"]},{"osd":5,"uuid":"a8c0cec9-a8ad-4a5d-800b-8b891ca6d836","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6809","nonce":1051933303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6811","nonce":1051933303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6815","nonce":1051933303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6813","nonce":1051933303}]},"public_addr":"172.21.15.131:6809/1051933303","cluster_addr":"172.21.15.131:6811/1051933303","heartbeat_back_addr":"172.21.15.131:6815/1051933303","heartbeat_front_addr":"172.21.15.131:6813/1051933303","state":["exists","up"]},{"osd":6,"uuid":"5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6819","nonce":2670922011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6821","nonce":2670922011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6825","nonce":2670922011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6823","nonce":2670922011}]},"public_addr":"172.21.15.40:6819/2670922011","cluster_addr":"172.21.15.40:6821/2670922011","heartbeat_back_addr":"172.21.15.40:6825/2670922011","heartbeat_front_addr":"172.21.15.40:6823/2670922011","state":["exists","up"]},{"osd":7,"uuid":"bfc536bb-6ae4-459e-afd9-8197c7420bd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6817","nonce":2353571051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6819","nonce":2353571051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6823","nonce":2353571051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6821","nonce":2353571051}]},"public_addr":"172.21.15.104:6817/2353571051","cluster_addr":"172.21.15.104:6819/2353571051","heartbeat_back_addr":"172.21.15.104:6823/2353571051","heartbeat_front_addr":"172.21.15.104:6821/2353571051","state":["exists","up"]},{"osd":8,"uuid":"1fe214f2-5ea3-4fc5-83bf-d9bccafe1529","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6817","nonce":1364533794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6819","nonce":1364533794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6823","nonce":1364533794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6821","nonce":1364533794}]},"public_addr":"172.21.15.131:6817/1364533794","cluster_addr":"172.21.15.131:6819/1364533794","heartbeat_back_addr":"172.21.15.131:6823/1364533794","heartbeat_front_addr":"172.21.15.131:6821/1364533794","state":["exists","up"]},{"osd":9,"uuid":"4b2b3555-6870-4ead-be50-b635799265a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6827","nonce":2604133190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6829","nonce":2604133190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6833","nonce":2604133190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6831","nonce":2604133190}]},"public_addr":"172.21.15.40:6827/2604133190","cluster_addr":"172.21.15.40:6829/2604133190","heartbeat_back_addr":"172.21.15.40:6833/2604133190","heartbeat_front_addr":"172.21.15.40:6831/2604133190","state":["exists","up"]},{"osd":10,"uuid":"70049e24-94ce-458e-97e3-dbcf9de05444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6825","nonce":2231164832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6827","nonce":2231164832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6831","nonce":2231164832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6829","nonce":2231164832}]},"public_addr":"172.21.15.104:6825/2231164832","cluster_addr":"172.21.15.104:6827/2231164832","heartbeat_back_addr":"172.21.15.104:6831/2231164832","heartbeat_front_addr":"172.21.15.104:6829/2231164832","state":["exists","up"]},{"osd":11,"uuid":"80447548-1fbe-4892-ac13-eb4bb6acf10d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6825","nonce":4128185670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6827","nonce":4128185670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6831","nonce":4128185670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6829","nonce":4128185670}]},"public_addr":"172.21.15.131:6825/4128185670","cluster_addr":"172.21.15.131:6827/4128185670","heartbeat_back_addr":"172.21.15.131:6831/4128185670","heartbeat_front_addr":"172.21.15.131:6829/4128185670","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:23.289784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:47.271677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:10.866792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:36.781924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:01.995848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:26.904606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:52.997438+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:19.221567+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:44.543977+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:11.948545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:38.402943+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:34:04.712200+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/889893607":"2024-06-07T03:38:37.622492+0000","172.21.15.40:6801/869712339":"2024-06-07T03:38:37.622492+0000","172.21.15.40:0/1294024108":"2024-06-07T03:38:37.622492+0000","172.21.15.40:0/23942578":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/1915158516":"2024-06-07T03:38:37.622492+0000","172.21.15.40:6800/869712339":"2024-06-07T03:38:37.622492+0000","172.21.15.40:0/2234980855":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6801/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6800/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2301695357":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/1676880727":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/2237379034":"2024-06-07T03:38:37.622492+0000","172.21.15.40:6801/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6800/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/1597665640":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/513375601":"2024-06-07T03:27:19.799499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T03:48:51.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.176+0000 7f2149b9e640 1 -- 172.21.15.40:0/1571594110 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f212c0a2d00 msgr2=0x7f212c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.176+0000 7f2149b9e640 1 --2- 172.21.15.40:0/1571594110 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f212c0a2d00 0x7f212c0a51c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f21340093d0 tx=0x7f2134009360 comp rx=0 tx=0).stop 2024-06-06T03:48:51.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.176+0000 7f2149b9e640 1 -- 172.21.15.40:0/1571594110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f214410bfc0 msgr2=0x7f214419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.176+0000 7f2149b9e640 1 --2- 172.21.15.40:0/1571594110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f214410bfc0 0x7f214419cbd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f213c00c720 tx=0x7f213c00cbe0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.176+0000 7f2149b9e640 1 -- 172.21.15.40:0/1571594110 shutdown_connections 2024-06-06T03:48:51.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.176+0000 7f2149b9e640 1 --2- 172.21.15.40:0/1571594110 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f212c0a2d00 0x7f212c0a51c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.176+0000 7f2149b9e640 1 --2- 172.21.15.40:0/1571594110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f214410bfc0 0x7f214419cbd0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.177+0000 7f2149b9e640 1 --2- 172.21.15.40:0/1571594110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21441035a0 0x7f2144198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.177+0000 7f2149b9e640 1 --2- 172.21.15.40:0/1571594110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2144102c80 0x7f2144198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.177+0000 7f2149b9e640 1 -- 172.21.15.40:0/1571594110 >> 172.21.15.40:0/1571594110 conn(0x7f2144077a00 msgr2=0x7f21440fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:51.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.177+0000 7f2149b9e640 1 -- 172.21.15.40:0/1571594110 shutdown_connections 2024-06-06T03:48:51.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.177+0000 7f2149b9e640 1 -- 172.21.15.40:0/1571594110 wait complete. 2024-06-06T03:48:51.186 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:50 smithi040 ceph-mon[32220]: pgmap v317: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-06-06T03:48:51.188 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-06T03:30:14.102785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-06T03:35:08.521025+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-06T03:35:12.369035+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-06T03:48:51.188 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-06T03:48:51.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:50 smithi131 ceph-mon[36687]: pgmap v317: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-06-06T03:48:51.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.297+0000 7f50cabb7640 1 -- 172.21.15.40:0/715555909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50c4107990 msgr2=0x7f50c4107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.297+0000 7f50cabb7640 1 --2- 172.21.15.40:0/715555909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50c4107990 0x7f50c4107d70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009930 tx=0x7f50c002a420 comp rx=0 tx=0).stop 2024-06-06T03:48:51.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.297+0000 7f50cabb7640 1 -- 172.21.15.40:0/715555909 shutdown_connections 2024-06-06T03:48:51.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.297+0000 7f50cabb7640 1 --2- 172.21.15.40:0/715555909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c406a9e0 0x7f50c410e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.297+0000 7f50cabb7640 1 --2- 172.21.15.40:0/715555909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50c406a020 0x7f50c406a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.297+0000 7f50cabb7640 1 --2- 172.21.15.40:0/715555909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50c4107990 0x7f50c4107d70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.297+0000 7f50cabb7640 1 -- 172.21.15.40:0/715555909 >> 172.21.15.40:0/715555909 conn(0x7f50c4077900 msgr2=0x7f50c4077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:51.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.297+0000 7f50cabb7640 1 -- 172.21.15.40:0/715555909 shutdown_connections 2024-06-06T03:48:51.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.297+0000 7f50cabb7640 1 -- 172.21.15.40:0/715555909 wait complete. 2024-06-06T03:48:51.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.298+0000 7f50cabb7640 1 Processor -- start 2024-06-06T03:48:51.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.298+0000 7f50cabb7640 1 -- start start 2024-06-06T03:48:51.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.298+0000 7f50cabb7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c406a020 0x7f50c4198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:51.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.298+0000 7f50cabb7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50c406a9e0 0x7f50c4198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:51.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.298+0000 7f50cabb7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50c4107990 0x7f50c419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:51.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.298+0000 7f50cabb7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f50c4111d70 con 0x7f50c4107990 2024-06-06T03:48:51.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.298+0000 7f50cabb7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50c4111bf0 con 0x7f50c406a020 2024-06-06T03:48:51.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.298+0000 7f50cabb7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50c4111ef0 con 0x7f50c406a9e0 2024-06-06T03:48:51.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.298+0000 7f50c9154640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50c4107990 0x7f50c419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:51.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.298+0000 7f50c9154640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50c4107990 0x7f50c419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54964/0 (socket says 172.21.15.40:54964) 2024-06-06T03:48:51.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.298+0000 7f50c9154640 1 -- 172.21.15.40:0/3987547144 learned_addr learned my addr 172.21.15.40:0/3987547144 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:48:51.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.298+0000 7f50c8953640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c406a020 0x7f50c4198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:51.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.298+0000 7f50bbfff640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50c406a9e0 0x7f50c4198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:51.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.299+0000 7f50c9154640 1 -- 172.21.15.40:0/3987547144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50c406a9e0 msgr2=0x7f50c4198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.299+0000 7f50c9154640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50c406a9e0 0x7f50c4198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.299+0000 7f50c9154640 1 -- 172.21.15.40:0/3987547144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c406a020 msgr2=0x7f50c4198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.299+0000 7f50c9154640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c406a020 0x7f50c4198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.299+0000 7f50c9154640 1 -- 172.21.15.40:0/3987547144 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c419d300 con 0x7f50c4107990 2024-06-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.299+0000 7f50bbfff640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50c406a9e0 0x7f50c4198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.299+0000 7f50c9154640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50c4107990 0x7f50c419cbd0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00eaa0 tx=0x7f50bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.299+0000 7f50b9ffb640 1 -- 172.21.15.40:0/3987547144 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc00cdb0 con 0x7f50c4107990 2024-06-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.299+0000 7f50b9ffb640 1 -- 172.21.15.40:0/3987547144 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50bc005020 con 0x7f50c4107990 2024-06-06T03:48:51.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.299+0000 7f50cabb7640 1 -- 172.21.15.40:0/3987547144 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c419d5f0 con 0x7f50c4107990 2024-06-06T03:48:51.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.299+0000 7f50b9ffb640 1 -- 172.21.15.40:0/3987547144 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc0116b0 con 0x7f50c4107990 2024-06-06T03:48:51.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.299+0000 7f50cabb7640 1 -- 172.21.15.40:0/3987547144 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c41a4e80 con 0x7f50c4107990 2024-06-06T03:48:51.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.300+0000 7f50cabb7640 1 -- 172.21.15.40:0/3987547144 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c4068c10 con 0x7f50c4107990 2024-06-06T03:48:51.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.301+0000 7f50b9ffb640 1 -- 172.21.15.40:0/3987547144 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f50bc005560 con 0x7f50c4107990 2024-06-06T03:48:51.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.302+0000 7f50b9ffb640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f50ac0a2d80 0x7f50ac0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:51.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.302+0000 7f50c8953640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f50ac0a2d80 0x7f50ac0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:51.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.302+0000 7f50b9ffb640 1 -- 172.21.15.40:0/3987547144 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8426+0+0 (secure 0 0 0) 0x7f50bc0cf5e0 con 0x7f50c4107990 2024-06-06T03:48:51.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.303+0000 7f50c8953640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f50ac0a2d80 0x7f50ac0a5240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009900 tx=0x7f50c00058d0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:51.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.307+0000 7f50b9ffb640 1 -- 172.21.15.40:0/3987547144 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f50bc016030 con 0x7f50c4107990 2024-06-06T03:48:51.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:50 smithi104 ceph-mon[36569]: pgmap v317: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-06-06T03:48:51.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.492+0000 7f50cabb7640 1 -- 172.21.15.40:0/3987547144 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f50c4109f50 con 0x7f50c4107990 2024-06-06T03:48:51.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.492+0000 7f50b9ffb640 1 -- 172.21.15.40:0/3987547144 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f50bc09c200 con 0x7f50c4107990 2024-06-06T03:48:51.492 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-06-06T03:48:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.494+0000 7f50cabb7640 1 -- 172.21.15.40:0/3987547144 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f50ac0a2d80 msgr2=0x7f50ac0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.494+0000 7f50cabb7640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f50ac0a2d80 0x7f50ac0a5240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009900 tx=0x7f50c00058d0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.494+0000 7f50cabb7640 1 -- 172.21.15.40:0/3987547144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50c4107990 msgr2=0x7f50c419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.494+0000 7f50cabb7640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50c4107990 0x7f50c419cbd0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00eaa0 tx=0x7f50bc00ef60 comp rx=0 tx=0).stop 2024-06-06T03:48:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.494+0000 7f50cabb7640 1 -- 172.21.15.40:0/3987547144 shutdown_connections 2024-06-06T03:48:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.494+0000 7f50cabb7640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f50ac0a2d80 0x7f50ac0a5240 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.494+0000 7f50cabb7640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50c4107990 0x7f50c419cbd0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.494+0000 7f50cabb7640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50c406a9e0 0x7f50c4198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.494+0000 7f50cabb7640 1 --2- 172.21.15.40:0/3987547144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c406a020 0x7f50c4198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.494+0000 7f50cabb7640 1 -- 172.21.15.40:0/3987547144 >> 172.21.15.40:0/3987547144 conn(0x7f50c4077900 msgr2=0x7f50c40fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:51.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.494+0000 7f50cabb7640 1 -- 172.21.15.40:0/3987547144 shutdown_connections 2024-06-06T03:48:51.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.494+0000 7f50cabb7640 1 -- 172.21.15.40:0/3987547144 wait complete. 2024-06-06T03:48:51.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-06T03:48:51.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.613+0000 7f574a9dc640 1 -- 172.21.15.40:0/311123413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f574406a020 msgr2=0x7f574406a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.613+0000 7f574a9dc640 1 --2- 172.21.15.40:0/311123413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f574406a020 0x7f574406a4a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5738009930 tx=0x7f573802a420 comp rx=0 tx=0).stop 2024-06-06T03:48:51.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.614+0000 7f574a9dc640 1 -- 172.21.15.40:0/311123413 shutdown_connections 2024-06-06T03:48:51.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.614+0000 7f574a9dc640 1 --2- 172.21.15.40:0/311123413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574406a9e0 0x7f574410e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.614+0000 7f574a9dc640 1 --2- 172.21.15.40:0/311123413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f574406a020 0x7f574406a4a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.614+0000 7f574a9dc640 1 --2- 172.21.15.40:0/311123413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5744107990 0x7f5744107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.614+0000 7f574a9dc640 1 -- 172.21.15.40:0/311123413 >> 172.21.15.40:0/311123413 conn(0x7f5744077900 msgr2=0x7f5744077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.614+0000 7f574a9dc640 1 -- 172.21.15.40:0/311123413 shutdown_connections 2024-06-06T03:48:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.614+0000 7f574a9dc640 1 -- 172.21.15.40:0/311123413 wait complete. 2024-06-06T03:48:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.614+0000 7f574a9dc640 1 Processor -- start 2024-06-06T03:48:51.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.614+0000 7f574a9dc640 1 -- start start 2024-06-06T03:48:51.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f574a9dc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574406a020 0x7f57441981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:51.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f574a9dc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f574406a9e0 0x7f57441986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:51.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f574a9dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5744107990 0x7f574419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:51.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f574a9dc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5744111d70 con 0x7f5744107990 2024-06-06T03:48:51.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f574a9dc640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5744111bf0 con 0x7f574406a020 2024-06-06T03:48:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f574a9dc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5744111ef0 con 0x7f574406a9e0 2024-06-06T03:48:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f5748f79640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5744107990 0x7f574419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f5748f79640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5744107990 0x7f574419ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54980/0 (socket says 172.21.15.40:54980) 2024-06-06T03:48:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f5748f79640 1 -- 172.21.15.40:0/3077443142 learned_addr learned my addr 172.21.15.40:0/3077443142 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:48:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f5743fff640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574406a020 0x7f57441981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f57437fe640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f574406a9e0 0x7f57441986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f5748f79640 1 -- 172.21.15.40:0/3077443142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f574406a9e0 msgr2=0x7f57441986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f5748f79640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f574406a9e0 0x7f57441986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f5748f79640 1 -- 172.21.15.40:0/3077443142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574406a020 msgr2=0x7f57441981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f5748f79640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574406a020 0x7f57441981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f5748f79640 1 -- 172.21.15.40:0/3077443142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574419d150 con 0x7f5744107990 2024-06-06T03:48:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f5743fff640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574406a020 0x7f57441981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:48:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f5748f79640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5744107990 0x7f574419ca20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f573c00cce0 tx=0x7f573c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f57417fa640 1 -- 172.21.15.40:0/3077443142 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c00f450 con 0x7f5744107990 2024-06-06T03:48:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f57417fa640 1 -- 172.21.15.40:0/3077443142 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f573c00fad0 con 0x7f5744107990 2024-06-06T03:48:51.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f574a9dc640 1 -- 172.21.15.40:0/3077443142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574419d440 con 0x7f5744107990 2024-06-06T03:48:51.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f57417fa640 1 -- 172.21.15.40:0/3077443142 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c018c10 con 0x7f5744107990 2024-06-06T03:48:51.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.615+0000 7f574a9dc640 1 -- 172.21.15.40:0/3077443142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57441a4cd0 con 0x7f5744107990 2024-06-06T03:48:51.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.616+0000 7f574a9dc640 1 -- 172.21.15.40:0/3077443142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5744068c10 con 0x7f5744107990 2024-06-06T03:48:51.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.618+0000 7f57417fa640 1 -- 172.21.15.40:0/3077443142 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f573c00f5f0 con 0x7f5744107990 2024-06-06T03:48:51.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.618+0000 7f57417fa640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f572c0a2d00 0x7f572c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:51.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.618+0000 7f5743fff640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f572c0a2d00 0x7f572c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:51.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.618+0000 7f57417fa640 1 -- 172.21.15.40:0/3077443142 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8426+0+0 (secure 0 0 0) 0x7f573c0cfbc0 con 0x7f5744107990 2024-06-06T03:48:51.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.619+0000 7f5743fff640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f572c0a2d00 0x7f572c0a51c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5734005e10 tx=0x7f5734005da0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:51.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.623+0000 7f57417fa640 1 -- 172.21.15.40:0/3077443142 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f573c014030 con 0x7f5744107990 2024-06-06T03:48:51.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.807+0000 7f574a9dc640 1 -- 172.21.15.40:0/3077443142 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f574402d2e0 con 0x7f5744107990 2024-06-06T03:48:51.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.808+0000 7f57417fa640 1 -- 172.21.15.40:0/3077443142 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f573c09c7e0 con 0x7f5744107990 2024-06-06T03:48:51.807 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-06-06T03:48:51.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.810+0000 7f574a9dc640 1 -- 172.21.15.40:0/3077443142 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f572c0a2d00 msgr2=0x7f572c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.810+0000 7f574a9dc640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f572c0a2d00 0x7f572c0a51c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5734005e10 tx=0x7f5734005da0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.810+0000 7f574a9dc640 1 -- 172.21.15.40:0/3077443142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5744107990 msgr2=0x7f574419ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.810+0000 7f574a9dc640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5744107990 0x7f574419ca20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f573c00cce0 tx=0x7f573c00a620 comp rx=0 tx=0).stop 2024-06-06T03:48:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.810+0000 7f574a9dc640 1 -- 172.21.15.40:0/3077443142 shutdown_connections 2024-06-06T03:48:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.810+0000 7f574a9dc640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f572c0a2d00 0x7f572c0a51c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.810+0000 7f574a9dc640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5744107990 0x7f574419ca20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.810+0000 7f574a9dc640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f574406a9e0 0x7f57441986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.810+0000 7f574a9dc640 1 --2- 172.21.15.40:0/3077443142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574406a020 0x7f57441981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.810+0000 7f574a9dc640 1 -- 172.21.15.40:0/3077443142 >> 172.21.15.40:0/3077443142 conn(0x7f5744077900 msgr2=0x7f57440fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:51.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.810+0000 7f574a9dc640 1 -- 172.21.15.40:0/3077443142 shutdown_connections 2024-06-06T03:48:51.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.810+0000 7f574a9dc640 1 -- 172.21.15.40:0/3077443142 wait complete. 2024-06-06T03:48:51.821 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-06T03:48:51.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.930+0000 7f678f529640 1 -- 172.21.15.40:0/1330626788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f678806a020 msgr2=0x7f678806a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.930+0000 7f678f529640 1 --2- 172.21.15.40:0/1330626788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f678806a020 0x7f678806a4a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6778009930 tx=0x7f677802a420 comp rx=0 tx=0).stop 2024-06-06T03:48:51.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.930+0000 7f678f529640 1 -- 172.21.15.40:0/1330626788 shutdown_connections 2024-06-06T03:48:51.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.930+0000 7f678f529640 1 --2- 172.21.15.40:0/1330626788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f678806a9e0 0x7f678810e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.930+0000 7f678f529640 1 --2- 172.21.15.40:0/1330626788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f678806a020 0x7f678806a4a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.930+0000 7f678f529640 1 --2- 172.21.15.40:0/1330626788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6788107990 0x7f6788107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.930+0000 7f678f529640 1 -- 172.21.15.40:0/1330626788 >> 172.21.15.40:0/1330626788 conn(0x7f6788077900 msgr2=0x7f6788077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:51.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.930+0000 7f678f529640 1 -- 172.21.15.40:0/1330626788 shutdown_connections 2024-06-06T03:48:51.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.930+0000 7f678f529640 1 -- 172.21.15.40:0/1330626788 wait complete. 2024-06-06T03:48:51.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.930+0000 7f678f529640 1 Processor -- start 2024-06-06T03:48:51.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.930+0000 7f678f529640 1 -- start start 2024-06-06T03:48:51.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678f529640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f678806a020 0x7f6788198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:51.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678f529640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f678806a9e0 0x7f6788198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:51.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678f529640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6788107990 0x7f678819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:51.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678f529640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6788111da0 con 0x7f678806a020 2024-06-06T03:48:51.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678f529640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6788111c20 con 0x7f6788107990 2024-06-06T03:48:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678f529640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6788111f20 con 0x7f678806a9e0 2024-06-06T03:48:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678d2c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f678806a020 0x7f6788198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678d2c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f678806a020 0x7f6788198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55004/0 (socket says 172.21.15.40:55004) 2024-06-06T03:48:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678d2c5640 1 -- 172.21.15.40:0/2817560726 learned_addr learned my addr 172.21.15.40:0/2817560726 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:48:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678dac6640 1 --2- 172.21.15.40:0/2817560726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6788107990 0x7f678819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678d2c5640 1 -- 172.21.15.40:0/2817560726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f678806a9e0 msgr2=0x7f6788198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678d2c5640 1 --2- 172.21.15.40:0/2817560726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f678806a9e0 0x7f6788198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678d2c5640 1 -- 172.21.15.40:0/2817560726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6788107990 msgr2=0x7f678819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678d2c5640 1 --2- 172.21.15.40:0/2817560726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6788107990 0x7f678819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678d2c5640 1 -- 172.21.15.40:0/2817560726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678819d310 con 0x7f678806a020 2024-06-06T03:48:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678dac6640 1 --2- 172.21.15.40:0/2817560726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6788107990 0x7f678819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:48:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678d2c5640 1 --2- 172.21.15.40:0/2817560726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f678806a020 0x7f6788198320 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f678400d950 tx=0x7f678400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f677e7fc640 1 -- 172.21.15.40:0/2817560726 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784017070 con 0x7f678806a020 2024-06-06T03:48:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f677e7fc640 1 -- 172.21.15.40:0/2817560726 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6784009c30 con 0x7f678806a020 2024-06-06T03:48:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f678f529640 1 -- 172.21.15.40:0/2817560726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678819d600 con 0x7f678806a020 2024-06-06T03:48:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.931+0000 7f677e7fc640 1 -- 172.21.15.40:0/2817560726 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678400f6b0 con 0x7f678806a020 2024-06-06T03:48:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.932+0000 7f678f529640 1 -- 172.21.15.40:0/2817560726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67881a4e90 con 0x7f678806a020 2024-06-06T03:48:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.933+0000 7f678f529640 1 -- 172.21.15.40:0/2817560726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6788068c10 con 0x7f678806a020 2024-06-06T03:48:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.934+0000 7f677e7fc640 1 -- 172.21.15.40:0/2817560726 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f67840109d0 con 0x7f678806a020 2024-06-06T03:48:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.934+0000 7f677e7fc640 1 --2- 172.21.15.40:0/2817560726 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f67740a2d00 0x7f67740a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.934+0000 7f677e7fc640 1 -- 172.21.15.40:0/2817560726 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8426+0+0 (secure 0 0 0) 0x7f67840cf040 con 0x7f678806a020 2024-06-06T03:48:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.934+0000 7f678cac4640 1 --2- 172.21.15.40:0/2817560726 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f67740a2d00 0x7f67740a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.935+0000 7f678cac4640 1 --2- 172.21.15.40:0/2817560726 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f67740a2d00 0x7f67740a51c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f677802a940 tx=0x7f6778000bc0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:51.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:51.939+0000 7f677e7fc640 1 -- 172.21.15.40:0/2817560726 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6784009de0 con 0x7f678806a020 2024-06-06T03:48:52.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:51 smithi040 ceph-mon[32220]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:48:52.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:51 smithi040 ceph-mon[32220]: from='client.15792 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:48:52.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:51 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1571594110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:48:52.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:51 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3987547144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T03:48:52.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:51 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3077443142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T03:48:52.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.123+0000 7f678f529640 1 -- 172.21.15.40:0/2817560726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6788109f70 con 0x7f678806a020 2024-06-06T03:48:52.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.123+0000 7f677e7fc640 1 -- 172.21.15.40:0/2817560726 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f678409bcc0 con 0x7f678806a020 2024-06-06T03:48:52.123 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-06-06T03:48:52.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.125+0000 7f678f529640 1 -- 172.21.15.40:0/2817560726 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f67740a2d00 msgr2=0x7f67740a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:52.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.125+0000 7f678f529640 1 --2- 172.21.15.40:0/2817560726 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f67740a2d00 0x7f67740a51c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f677802a940 tx=0x7f6778000bc0 comp rx=0 tx=0).stop 2024-06-06T03:48:52.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.125+0000 7f678f529640 1 -- 172.21.15.40:0/2817560726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f678806a020 msgr2=0x7f6788198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:52.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.125+0000 7f678f529640 1 --2- 172.21.15.40:0/2817560726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f678806a020 0x7f6788198320 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f678400d950 tx=0x7f678400de10 comp rx=0 tx=0).stop 2024-06-06T03:48:52.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.125+0000 7f678f529640 1 -- 172.21.15.40:0/2817560726 shutdown_connections 2024-06-06T03:48:52.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.125+0000 7f678f529640 1 --2- 172.21.15.40:0/2817560726 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f67740a2d00 0x7f67740a51c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:52.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.125+0000 7f678f529640 1 --2- 172.21.15.40:0/2817560726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6788107990 0x7f678819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:52.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.125+0000 7f678f529640 1 --2- 172.21.15.40:0/2817560726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f678806a9e0 0x7f6788198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:52.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.125+0000 7f678f529640 1 --2- 172.21.15.40:0/2817560726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f678806a020 0x7f6788198320 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:52.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.125+0000 7f678f529640 1 -- 172.21.15.40:0/2817560726 >> 172.21.15.40:0/2817560726 conn(0x7f6788077900 msgr2=0x7f67880fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:52.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.125+0000 7f678f529640 1 -- 172.21.15.40:0/2817560726 shutdown_connections 2024-06-06T03:48:52.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.125+0000 7f678f529640 1 -- 172.21.15.40:0/2817560726 wait complete. 2024-06-06T03:48:52.136 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-06T03:48:52.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.241+0000 7f81679d5640 1 -- 172.21.15.40:0/574585579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8160102c80 msgr2=0x7f8160103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:52.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.241+0000 7f81679d5640 1 --2- 172.21.15.40:0/574585579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8160102c80 0x7f8160103060 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f815c009900 tx=0x7f815c02a480 comp rx=0 tx=0).stop 2024-06-06T03:48:52.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.241+0000 7f81679d5640 1 -- 172.21.15.40:0/574585579 shutdown_connections 2024-06-06T03:48:52.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.241+0000 7f81679d5640 1 --2- 172.21.15.40:0/574585579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f816010bfc0 0x7f816010e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:52.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.241+0000 7f81679d5640 1 --2- 172.21.15.40:0/574585579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81601035a0 0x7f816010ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:52.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.241+0000 7f81679d5640 1 --2- 172.21.15.40:0/574585579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8160102c80 0x7f8160103060 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:52.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.241+0000 7f81679d5640 1 -- 172.21.15.40:0/574585579 >> 172.21.15.40:0/574585579 conn(0x7f8160077a00 msgr2=0x7f8160077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:52.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.241+0000 7f81679d5640 1 -- 172.21.15.40:0/574585579 shutdown_connections 2024-06-06T03:48:52.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.241+0000 7f81679d5640 1 -- 172.21.15.40:0/574585579 wait complete. 2024-06-06T03:48:52.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.242+0000 7f81679d5640 1 Processor -- start 2024-06-06T03:48:52.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.242+0000 7f81679d5640 1 -- start start 2024-06-06T03:48:52.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.242+0000 7f81679d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8160102c80 0x7f8160198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:52.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.242+0000 7f81679d5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81601035a0 0x7f8160198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:52.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.242+0000 7f81679d5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f816010bfc0 0x7f816019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:52.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.242+0000 7f81679d5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8160111d90 con 0x7f8160102c80 2024-06-06T03:48:52.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.242+0000 7f81679d5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8160111c10 con 0x7f81601035a0 2024-06-06T03:48:52.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.242+0000 7f81679d5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8160111f10 con 0x7f816010bfc0 2024-06-06T03:48:52.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.242+0000 7f8165771640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8160102c80 0x7f8160198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:52.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.242+0000 7f8165771640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8160102c80 0x7f8160198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55024/0 (socket says 172.21.15.40:55024) 2024-06-06T03:48:52.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.242+0000 7f8165771640 1 -- 172.21.15.40:0/32819536 learned_addr learned my addr 172.21.15.40:0/32819536 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:48:52.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.242+0000 7f8165f72640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f816010bfc0 0x7f816019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:52.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.242+0000 7f8164f70640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81601035a0 0x7f8160198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:52.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.243+0000 7f8165771640 1 -- 172.21.15.40:0/32819536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f816010bfc0 msgr2=0x7f816019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:52.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.243+0000 7f8165771640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f816010bfc0 0x7f816019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:52.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.243+0000 7f8165771640 1 -- 172.21.15.40:0/32819536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81601035a0 msgr2=0x7f8160198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:52.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.243+0000 7f8165771640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81601035a0 0x7f8160198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:52.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.243+0000 7f8165771640 1 -- 172.21.15.40:0/32819536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816019d1f0 con 0x7f8160102c80 2024-06-06T03:48:52.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.243+0000 7f8165f72640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f816010bfc0 0x7f816019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:48:52.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.243+0000 7f8164f70640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81601035a0 0x7f8160198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:48:52.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.243+0000 7f8165771640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8160102c80 0x7f8160198250 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f815c006010 tx=0x7f815c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:52.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.243+0000 7f81567fc640 1 -- 172.21.15.40:0/32819536 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c03f030 con 0x7f8160102c80 2024-06-06T03:48:52.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.243+0000 7f81567fc640 1 -- 172.21.15.40:0/32819536 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f815c003ce0 con 0x7f8160102c80 2024-06-06T03:48:52.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.243+0000 7f81679d5640 1 -- 172.21.15.40:0/32819536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816019d480 con 0x7f8160102c80 2024-06-06T03:48:52.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.243+0000 7f81567fc640 1 -- 172.21.15.40:0/32819536 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c035070 con 0x7f8160102c80 2024-06-06T03:48:52.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.243+0000 7f81679d5640 1 -- 172.21.15.40:0/32819536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81601a4d70 con 0x7f8160102c80 2024-06-06T03:48:52.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.244+0000 7f81679d5640 1 -- 172.21.15.40:0/32819536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8160068c10 con 0x7f8160102c80 2024-06-06T03:48:52.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.245+0000 7f81567fc640 1 -- 172.21.15.40:0/32819536 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f815c042030 con 0x7f8160102c80 2024-06-06T03:48:52.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.246+0000 7f81567fc640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f814c0a2d80 0x7f814c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:48:52.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.246+0000 7f8164f70640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f814c0a2d80 0x7f814c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:48:52.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.246+0000 7f81567fc640 1 -- 172.21.15.40:0/32819536 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8426+0+0 (secure 0 0 0) 0x7f815c0efdb0 con 0x7f8160102c80 2024-06-06T03:48:52.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.246+0000 7f8164f70640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f814c0a2d80 0x7f814c0a5240 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8150005d00 tx=0x7f8150009550 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:48:52.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.249+0000 7f81567fc640 1 -- 172.21.15.40:0/32819536 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f815c0bcaf0 con 0x7f8160102c80 2024-06-06T03:48:52.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:51 smithi131 ceph-mon[36687]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:48:52.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:51 smithi131 ceph-mon[36687]: from='client.15792 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:48:52.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:51 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1571594110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:48:52.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:51 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3987547144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T03:48:52.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:51 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3077443142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T03:48:52.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:51 smithi104 ceph-mon[36569]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:48:52.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:51 smithi104 ceph-mon[36569]: from='client.15792 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T03:48:52.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:51 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1571594110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:48:52.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:51 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3987547144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T03:48:52.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:51 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3077443142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T03:48:52.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.484+0000 7f81679d5640 1 -- 172.21.15.40:0/32819536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f8160107170 con 0x7f8160102c80 2024-06-06T03:48:52.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.493+0000 7f81567fc640 1 -- 172.21.15.40:0/32819536 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740902 (secure 0 0 0) 0x7f815c0bc3f0 con 0x7f8160102c80 2024-06-06T03:48:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:48:52.501 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:48:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 26, 2024-06-06T03:48:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "active_gid": 24790, 2024-06-06T03:48:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "active_name": "x", 2024-06-06T03:48:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "active_addrs": { 2024-06-06T03:48:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-06-06T03:48:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-06-06T03:48:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2024-06-06T03:48:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1568651190 2024-06-06T03:48:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-06-06T03:48:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2024-06-06T03:48:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1568651190 2024-06-06T03:48:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-06-06T03:48:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "active_addr": "172.21.15.40:6801/1568651190", 2024-06-06T03:48:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "active_change": "2024-06-06T03:38:37.622692+0000", 2024-06-06T03:48:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "active_mgr_features": 4540701547738038271, 2024-06-06T03:48:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "available": true, 2024-06-06T03:48:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "standbys": [ 2024-06-06T03:48:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "gid": 24787, 2024-06-06T03:48:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "z", 2024-06-06T03:48:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_features": 4540701547738038271, 2024-06-06T03:48:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "available_modules": [ 2024-06-06T03:48:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "alerts", 2024-06-06T03:48:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T03:48:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.511 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.511 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.511 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.511 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.511 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.511 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.511 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.512 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.512 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.512 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.512 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.512 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.512 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.512 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.512 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.512 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.513 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.513 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.513 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.513 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.513 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.513 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.513 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.513 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.514 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.514 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.514 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.514 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.514 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.514 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.514 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.514 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.515 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.515 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.515 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.515 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.515 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.515 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.515 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.515 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.516 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.516 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.516 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.516 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_destination": { 2024-06-06T03:48:52.516 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_destination", 2024-06-06T03:48:52.516 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.516 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.516 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.516 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.517 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.517 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.517 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.517 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Email address to send alerts to", 2024-06-06T03:48:52.517 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.517 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.517 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.517 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.518 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_from_name": { 2024-06-06T03:48:52.518 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_from_name", 2024-06-06T03:48:52.518 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.518 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.518 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.518 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "Ceph", 2024-06-06T03:48:52.518 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.518 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.519 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.519 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Email From: name", 2024-06-06T03:48:52.519 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.519 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.519 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.519 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.519 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_host": { 2024-06-06T03:48:52.519 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_host", 2024-06-06T03:48:52.519 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.520 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.520 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.520 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.520 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.520 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.520 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.520 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP server", 2024-06-06T03:48:52.520 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.520 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.521 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.521 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.521 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_password": { 2024-06-06T03:48:52.521 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_password", 2024-06-06T03:48:52.521 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.521 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.521 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.521 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.522 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.522 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.522 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.522 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Password to authenticate with", 2024-06-06T03:48:52.522 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.522 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.522 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.522 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.523 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_port": { 2024-06-06T03:48:52.523 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_port", 2024-06-06T03:48:52.523 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.523 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.523 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.523 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "465", 2024-06-06T03:48:52.523 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.523 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.523 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.524 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP port", 2024-06-06T03:48:52.524 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.524 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.524 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.524 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.524 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_sender": { 2024-06-06T03:48:52.524 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_sender", 2024-06-06T03:48:52.524 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.524 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.525 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.525 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.525 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.525 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.525 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.525 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP envelope sender", 2024-06-06T03:48:52.525 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.525 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.526 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.526 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.526 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_ssl": { 2024-06-06T03:48:52.526 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_ssl", 2024-06-06T03:48:52.526 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.526 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.526 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.527 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.527 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.527 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.527 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.527 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T03:48:52.527 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.527 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.527 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.527 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.528 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_user": { 2024-06-06T03:48:52.528 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_user", 2024-06-06T03:48:52.528 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.528 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.528 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.528 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.528 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.528 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.528 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.529 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "User to authenticate as", 2024-06-06T03:48:52.529 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.529 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.529 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.529 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.529 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.529 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.529 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.529 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "balancer", 2024-06-06T03:48:52.530 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.530 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.530 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.530 INFO:teuthology.orchestra.run.smithi040.stdout: "active": { 2024-06-06T03:48:52.530 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "active", 2024-06-06T03:48:52.530 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.530 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.530 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.531 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.531 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.531 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.531 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.531 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T03:48:52.531 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.531 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.531 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.532 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.532 INFO:teuthology.orchestra.run.smithi040.stdout: "begin_time": { 2024-06-06T03:48:52.532 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "begin_time", 2024-06-06T03:48:52.532 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.532 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.532 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.532 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0000", 2024-06-06T03:48:52.532 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.532 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.533 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.533 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T03:48:52.533 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T03:48:52.533 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.533 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.533 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.533 INFO:teuthology.orchestra.run.smithi040.stdout: "begin_weekday": { 2024-06-06T03:48:52.533 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "begin_weekday", 2024-06-06T03:48:52.534 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:52.534 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.534 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.534 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:52.534 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0", 2024-06-06T03:48:52.534 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "6", 2024-06-06T03:48:52.534 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.534 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T03:48:52.535 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T03:48:52.535 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.535 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.535 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.535 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_max_iterations": { 2024-06-06T03:48:52.535 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_max_iterations", 2024-06-06T03:48:52.535 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:52.535 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.536 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.536 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "25", 2024-06-06T03:48:52.536 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2024-06-06T03:48:52.536 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "250", 2024-06-06T03:48:52.536 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.536 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T03:48:52.536 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.536 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.536 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.537 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_metrics": { 2024-06-06T03:48:52.537 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_metrics", 2024-06-06T03:48:52.537 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.537 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.537 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.537 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T03:48:52.537 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.538 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.538 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.538 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T03:48:52.538 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T03:48:52.538 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.538 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.539 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.539 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_step": { 2024-06-06T03:48:52.539 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_step", 2024-06-06T03:48:52.539 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:52.539 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.539 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.539 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0.5", 2024-06-06T03:48:52.539 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0.001", 2024-06-06T03:48:52.540 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "0.999", 2024-06-06T03:48:52.540 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.540 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "aggressiveness of optimization", 2024-06-06T03:48:52.540 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T03:48:52.540 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.540 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.540 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.540 INFO:teuthology.orchestra.run.smithi040.stdout: "end_time": { 2024-06-06T03:48:52.541 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "end_time", 2024-06-06T03:48:52.541 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.541 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.541 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.541 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "2359", 2024-06-06T03:48:52.541 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.541 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.541 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.541 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T03:48:52.542 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T03:48:52.542 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.542 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.542 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.542 INFO:teuthology.orchestra.run.smithi040.stdout: "end_weekday": { 2024-06-06T03:48:52.542 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "end_weekday", 2024-06-06T03:48:52.542 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:52.543 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.543 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.543 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:52.543 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0", 2024-06-06T03:48:52.543 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "6", 2024-06-06T03:48:52.543 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.543 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T03:48:52.543 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T03:48:52.543 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.544 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.544 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.544 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.544 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.544 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.544 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.544 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.544 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.545 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.545 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.545 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.545 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.545 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.545 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.545 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.545 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.545 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.546 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.546 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.546 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.546 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.546 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.546 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.546 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.546 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.547 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.547 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.547 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.547 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.547 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.547 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.547 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.547 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.548 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.548 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.548 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.548 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.548 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.548 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.548 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.548 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.549 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.549 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.549 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.549 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.549 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.549 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.549 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.549 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.550 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.550 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.550 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.550 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.550 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.550 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.550 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.550 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.551 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.551 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.551 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.551 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.551 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.551 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.551 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.551 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.552 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.552 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.552 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.552 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.552 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.552 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.552 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.552 INFO:teuthology.orchestra.run.smithi040.stdout: "min_score": { 2024-06-06T03:48:52.552 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "min_score", 2024-06-06T03:48:52.553 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:52.553 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.553 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.553 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:52.553 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.553 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.553 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.553 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T03:48:52.553 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.554 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.554 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.554 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.554 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": { 2024-06-06T03:48:52.554 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mode", 2024-06-06T03:48:52.554 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.554 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.554 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.555 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "upmap", 2024-06-06T03:48:52.555 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.555 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.555 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.555 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-compat", 2024-06-06T03:48:52.555 INFO:teuthology.orchestra.run.smithi040.stdout: "none", 2024-06-06T03:48:52.555 INFO:teuthology.orchestra.run.smithi040.stdout: "read", 2024-06-06T03:48:52.555 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap", 2024-06-06T03:48:52.556 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap-read" 2024-06-06T03:48:52.556 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.556 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Balancer mode", 2024-06-06T03:48:52.556 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.556 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.556 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.556 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_ids": { 2024-06-06T03:48:52.556 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pool_ids", 2024-06-06T03:48:52.556 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.557 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.557 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.557 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.557 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.557 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.557 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.557 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T03:48:52.557 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.557 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.558 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.558 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:52.558 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:52.558 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.558 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.558 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.558 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:52.559 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.559 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.559 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.559 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T03:48:52.559 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.559 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.559 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.559 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap_max_deviation": { 2024-06-06T03:48:52.560 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "upmap_max_deviation", 2024-06-06T03:48:52.560 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.560 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.560 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.560 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2024-06-06T03:48:52.560 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2024-06-06T03:48:52.560 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.560 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.560 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T03:48:52.561 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T03:48:52.561 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.561 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.561 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap_max_optimizations": { 2024-06-06T03:48:52.561 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "upmap_max_optimizations", 2024-06-06T03:48:52.561 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:52.561 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.561 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.562 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:52.562 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.562 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.562 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.562 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T03:48:52.562 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.562 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.562 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.563 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.563 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.563 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.563 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cephadm", 2024-06-06T03:48:52.563 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.563 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.563 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.563 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_down_multiplier": { 2024-06-06T03:48:52.564 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_down_multiplier", 2024-06-06T03:48:52.564 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:52.564 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.564 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.564 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3.0", 2024-06-06T03:48:52.564 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.564 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.564 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.564 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T03:48:52.565 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.565 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.565 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.565 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_refresh_rate": { 2024-06-06T03:48:52.565 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_refresh_rate", 2024-06-06T03:48:52.565 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.565 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.566 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.566 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "20", 2024-06-06T03:48:52.566 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.566 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.566 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.566 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T03:48:52.566 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.566 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.566 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.567 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.567 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_starting_port": { 2024-06-06T03:48:52.567 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_starting_port", 2024-06-06T03:48:52.567 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.567 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.567 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.567 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "4721", 2024-06-06T03:48:52.567 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.567 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.568 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.568 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T03:48:52.568 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.568 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.568 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.568 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.568 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_ptrace": { 2024-06-06T03:48:52.569 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_ptrace", 2024-06-06T03:48:52.569 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.569 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.569 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.569 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.569 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.569 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.569 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.570 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T03:48:52.570 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T03:48:52.570 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.570 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.570 INFO:teuthology.orchestra.run.smithi040.stdout: "autotune_interval": { 2024-06-06T03:48:52.570 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "autotune_interval", 2024-06-06T03:48:52.570 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.571 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.571 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.571 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:52.571 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.571 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.571 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.571 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T03:48:52.571 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.572 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.572 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.572 INFO:teuthology.orchestra.run.smithi040.stdout: "autotune_memory_target_ratio": { 2024-06-06T03:48:52.572 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T03:48:52.572 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:52.572 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.572 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.573 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0.7", 2024-06-06T03:48:52.573 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.573 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.573 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.573 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T03:48:52.573 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.573 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.573 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.574 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm_log_destination": { 2024-06-06T03:48:52.574 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cephadm_log_destination", 2024-06-06T03:48:52.574 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.574 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.574 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.574 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.574 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.575 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.575 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.575 INFO:teuthology.orchestra.run.smithi040.stdout: "file", 2024-06-06T03:48:52.575 INFO:teuthology.orchestra.run.smithi040.stdout: "file,syslog", 2024-06-06T03:48:52.575 INFO:teuthology.orchestra.run.smithi040.stdout: "syslog" 2024-06-06T03:48:52.575 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.575 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T03:48:52.575 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.575 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.576 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.576 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.576 INFO:teuthology.orchestra.run.smithi040.stdout: "cgroups_split": { 2024-06-06T03:48:52.576 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cgroups_split", 2024-06-06T03:48:52.576 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.576 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.576 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.576 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.577 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.577 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.577 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.577 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T03:48:52.577 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.577 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.577 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.578 INFO:teuthology.orchestra.run.smithi040.stdout: "config_checks_enabled": { 2024-06-06T03:48:52.578 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_checks_enabled", 2024-06-06T03:48:52.578 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.578 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.578 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.578 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.578 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.578 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.579 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.579 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T03:48:52.579 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.579 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.579 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.579 INFO:teuthology.orchestra.run.smithi040.stdout: "config_dashboard": { 2024-06-06T03:48:52.579 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_dashboard", 2024-06-06T03:48:52.580 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.580 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.580 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.580 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.580 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.580 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.580 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.580 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T03:48:52.581 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.581 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.581 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.581 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.581 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_alertmanager": { 2024-06-06T03:48:52.581 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_alertmanager", 2024-06-06T03:48:52.581 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.581 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.582 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.582 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T03:48:52.582 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.582 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.582 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.582 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2024-06-06T03:48:52.582 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.582 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.582 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.583 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_base": { 2024-06-06T03:48:52.583 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_base", 2024-06-06T03:48:52.583 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.583 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.583 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.583 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T03:48:52.583 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.584 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.584 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.584 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Container image name, without the tag", 2024-06-06T03:48:52.584 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.584 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.584 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.584 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_elasticsearch": { 2024-06-06T03:48:52.585 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_elasticsearch", 2024-06-06T03:48:52.585 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.585 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.585 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.585 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T03:48:52.585 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.585 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.586 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.586 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "elasticsearch container image", 2024-06-06T03:48:52.586 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.586 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.586 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.586 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.587 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_grafana": { 2024-06-06T03:48:52.587 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_grafana", 2024-06-06T03:48:52.587 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.587 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.587 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.587 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T03:48:52.587 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.587 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.587 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.588 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2024-06-06T03:48:52.588 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.588 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.588 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.588 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_haproxy": { 2024-06-06T03:48:52.588 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_haproxy", 2024-06-06T03:48:52.588 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.588 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.589 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.589 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T03:48:52.589 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.589 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.589 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.589 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "HAproxy container image", 2024-06-06T03:48:52.589 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.589 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.590 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.590 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_agent": { 2024-06-06T03:48:52.590 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_agent", 2024-06-06T03:48:52.590 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.590 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.590 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.591 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T03:48:52.591 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.591 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.591 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.591 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger agent container image", 2024-06-06T03:48:52.591 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.591 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.591 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.592 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_collector": { 2024-06-06T03:48:52.592 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_collector", 2024-06-06T03:48:52.592 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.592 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.592 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.592 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T03:48:52.592 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.592 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.593 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.593 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger collector container image", 2024-06-06T03:48:52.593 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.593 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.593 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.593 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_query": { 2024-06-06T03:48:52.593 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_query", 2024-06-06T03:48:52.593 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.594 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.594 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.594 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T03:48:52.594 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.594 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.594 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.594 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger query container image", 2024-06-06T03:48:52.594 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.595 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.595 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.595 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_keepalived": { 2024-06-06T03:48:52.595 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_keepalived", 2024-06-06T03:48:52.595 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.595 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.595 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.595 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T03:48:52.596 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.596 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.596 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.596 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Keepalived container image", 2024-06-06T03:48:52.596 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.596 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.596 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.596 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_loki": { 2024-06-06T03:48:52.597 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_loki", 2024-06-06T03:48:52.597 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.597 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.597 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.597 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T03:48:52.597 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.597 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.597 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.598 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Loki container image", 2024-06-06T03:48:52.598 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.598 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.598 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.598 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_node_exporter": { 2024-06-06T03:48:52.598 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_node_exporter", 2024-06-06T03:48:52.598 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.599 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.599 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.599 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T03:48:52.599 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.599 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.599 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.599 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2024-06-06T03:48:52.599 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.599 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.600 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.600 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.600 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_nvmeof": { 2024-06-06T03:48:52.600 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_nvmeof", 2024-06-06T03:48:52.600 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.600 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.600 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.600 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T03:48:52.600 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.600 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.601 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.601 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Nvme-of container image", 2024-06-06T03:48:52.601 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.601 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.601 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.601 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_prometheus": { 2024-06-06T03:48:52.601 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_prometheus", 2024-06-06T03:48:52.601 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.602 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.602 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.602 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T03:48:52.602 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.602 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.602 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.602 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2024-06-06T03:48:52.602 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.603 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.603 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.603 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_promtail": { 2024-06-06T03:48:52.603 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_promtail", 2024-06-06T03:48:52.603 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.603 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.603 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.604 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T03:48:52.604 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.604 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.604 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.604 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Promtail container image", 2024-06-06T03:48:52.604 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.604 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.604 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.604 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_samba": { 2024-06-06T03:48:52.605 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_samba", 2024-06-06T03:48:52.605 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.605 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.605 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.605 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T03:48:52.605 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.605 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.605 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.606 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Samba/SMB container image", 2024-06-06T03:48:52.606 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.606 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.606 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.606 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_snmp_gateway": { 2024-06-06T03:48:52.606 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_snmp_gateway", 2024-06-06T03:48:52.606 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.607 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.607 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.607 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T03:48:52.607 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.607 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.607 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.607 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SNMP Gateway container image", 2024-06-06T03:48:52.607 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.608 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.608 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.608 INFO:teuthology.orchestra.run.smithi040.stdout: "container_init": { 2024-06-06T03:48:52.608 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_init", 2024-06-06T03:48:52.608 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.608 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.608 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.609 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.609 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.609 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.609 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.609 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T03:48:52.609 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.609 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.610 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.610 INFO:teuthology.orchestra.run.smithi040.stdout: "daemon_cache_timeout": { 2024-06-06T03:48:52.610 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "daemon_cache_timeout", 2024-06-06T03:48:52.610 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.610 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.610 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.610 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:52.611 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.611 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.611 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.611 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T03:48:52.611 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.611 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.611 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.612 INFO:teuthology.orchestra.run.smithi040.stdout: "default_cephadm_command_timeout": { 2024-06-06T03:48:52.612 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T03:48:52.612 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.612 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.612 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.612 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "900", 2024-06-06T03:48:52.612 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.612 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.613 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.613 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T03:48:52.613 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.613 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.613 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.613 INFO:teuthology.orchestra.run.smithi040.stdout: "default_registry": { 2024-06-06T03:48:52.613 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "default_registry", 2024-06-06T03:48:52.614 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.614 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.614 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.614 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io", 2024-06-06T03:48:52.614 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.614 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.614 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.614 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T03:48:52.615 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.615 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.615 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.615 INFO:teuthology.orchestra.run.smithi040.stdout: "device_cache_timeout": { 2024-06-06T03:48:52.615 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_cache_timeout", 2024-06-06T03:48:52.615 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.615 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.616 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.616 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1800", 2024-06-06T03:48:52.616 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.616 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.616 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.616 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache device inventory", 2024-06-06T03:48:52.616 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.616 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.616 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.617 INFO:teuthology.orchestra.run.smithi040.stdout: "device_enhanced_scan": { 2024-06-06T03:48:52.617 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_enhanced_scan", 2024-06-06T03:48:52.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.617 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.617 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.617 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.617 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.618 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.618 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.618 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T03:48:52.618 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.618 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.618 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.618 INFO:teuthology.orchestra.run.smithi040.stdout: "facts_cache_timeout": { 2024-06-06T03:48:52.618 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "facts_cache_timeout", 2024-06-06T03:48:52.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.619 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.619 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.619 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:52.619 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.619 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.619 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.619 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache host facts data", 2024-06-06T03:48:52.619 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.620 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.620 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.620 INFO:teuthology.orchestra.run.smithi040.stdout: "grafana_dashboards_path": { 2024-06-06T03:48:52.620 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "grafana_dashboards_path", 2024-06-06T03:48:52.620 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.620 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.620 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.621 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T03:48:52.621 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.621 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.621 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.621 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T03:48:52.621 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.621 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.621 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.622 INFO:teuthology.orchestra.run.smithi040.stdout: "host_check_interval": { 2024-06-06T03:48:52.622 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "host_check_interval", 2024-06-06T03:48:52.622 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.622 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.622 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.622 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:52.622 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.622 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.623 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.623 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to perform a host check", 2024-06-06T03:48:52.623 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.623 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.623 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.623 INFO:teuthology.orchestra.run.smithi040.stdout: "hw_monitoring": { 2024-06-06T03:48:52.623 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "hw_monitoring", 2024-06-06T03:48:52.624 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.624 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.624 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.624 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.624 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.624 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.624 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.624 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T03:48:52.625 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.625 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.625 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.625 INFO:teuthology.orchestra.run.smithi040.stdout: "inventory_list_all": { 2024-06-06T03:48:52.625 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "inventory_list_all", 2024-06-06T03:48:52.625 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.625 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.625 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.626 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.626 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.626 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.626 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.626 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T03:48:52.626 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.626 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.626 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.627 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.627 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.627 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.627 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.627 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.627 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.627 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.627 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.628 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.628 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.628 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.628 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.628 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.628 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.628 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.628 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.628 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.629 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.629 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.629 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.629 INFO:teuthology.orchestra.run.smithi040.stdout: "log_refresh_metadata": { 2024-06-06T03:48:52.629 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_refresh_metadata", 2024-06-06T03:48:52.629 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.629 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.629 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.630 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.630 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.630 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.630 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.630 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T03:48:52.630 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.630 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.631 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.631 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.631 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.631 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.631 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.631 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.631 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.631 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.632 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.632 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.632 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T03:48:52.632 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.632 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.632 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.632 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.633 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.633 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.633 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.633 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.633 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.633 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.633 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.633 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.633 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.634 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.634 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.634 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.634 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.634 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.634 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.634 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.634 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.635 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.635 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.635 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.635 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.635 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.635 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.635 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.636 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.636 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.636 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.636 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.636 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.636 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.636 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.636 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.637 INFO:teuthology.orchestra.run.smithi040.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T03:48:52.637 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T03:48:52.637 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.637 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.637 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.637 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.637 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.637 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.638 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.638 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T03:48:52.638 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.638 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.638 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.639 INFO:teuthology.orchestra.run.smithi040.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T03:48:52.639 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T03:48:52.639 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.639 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.639 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.639 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "*", 2024-06-06T03:48:52.639 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.639 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.639 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.640 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T03:48:52.640 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.640 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.640 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.640 INFO:teuthology.orchestra.run.smithi040.stdout: "max_count_per_host": { 2024-06-06T03:48:52.640 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_count_per_host", 2024-06-06T03:48:52.640 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.641 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.641 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.641 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:52.641 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.641 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.641 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.641 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "max number of daemons per service per host", 2024-06-06T03:48:52.641 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.642 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.642 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.642 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd_draining_count": { 2024-06-06T03:48:52.642 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_osd_draining_count", 2024-06-06T03:48:52.642 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.642 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.642 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.643 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:52.643 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.643 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.643 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.643 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T03:48:52.643 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.643 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.643 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.644 INFO:teuthology.orchestra.run.smithi040.stdout: "migration_current": { 2024-06-06T03:48:52.644 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "migration_current", 2024-06-06T03:48:52.644 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.644 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.644 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.644 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.644 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.645 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.645 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.645 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "internal - do not modify", 2024-06-06T03:48:52.645 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.645 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.645 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.645 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": { 2024-06-06T03:48:52.645 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mode", 2024-06-06T03:48:52.646 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.646 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.646 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.646 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "root", 2024-06-06T03:48:52.646 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.646 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.646 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.646 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm-package", 2024-06-06T03:48:52.647 INFO:teuthology.orchestra.run.smithi040.stdout: "root" 2024-06-06T03:48:52.647 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.647 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T03:48:52.647 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.647 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.647 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.647 INFO:teuthology.orchestra.run.smithi040.stdout: "oob_default_addr": { 2024-06-06T03:48:52.648 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "oob_default_addr", 2024-06-06T03:48:52.648 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.648 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.648 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.648 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "169.254.1.1", 2024-06-06T03:48:52.648 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.648 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.648 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.649 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T03:48:52.649 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.649 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.649 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.649 INFO:teuthology.orchestra.run.smithi040.stdout: "prometheus_alerts_path": { 2024-06-06T03:48:52.649 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus_alerts_path", 2024-06-06T03:48:52.649 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.650 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.650 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.650 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T03:48:52.650 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.650 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.650 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.650 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T03:48:52.650 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.651 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.651 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.651 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_insecure": { 2024-06-06T03:48:52.651 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_insecure", 2024-06-06T03:48:52.651 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.651 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.651 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.652 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.652 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.652 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.652 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.652 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T03:48:52.652 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.652 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.652 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.653 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_password": { 2024-06-06T03:48:52.653 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_password", 2024-06-06T03:48:52.653 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.653 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.653 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.653 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.653 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.653 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.654 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.654 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T03:48:52.654 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.654 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.654 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.654 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.654 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_url": { 2024-06-06T03:48:52.655 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_url", 2024-06-06T03:48:52.655 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.655 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.655 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.655 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.655 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.655 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.655 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.655 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T03:48:52.656 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.656 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.656 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.656 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.656 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_username": { 2024-06-06T03:48:52.656 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_username", 2024-06-06T03:48:52.656 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.656 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.658 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.658 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.658 INFO:teuthology.orchestra.run.smithi040.stdout: "secure_monitoring_stack": { 2024-06-06T03:48:52.658 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "secure_monitoring_stack", 2024-06-06T03:48:52.658 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.658 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.658 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.658 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.658 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.659 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.659 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.659 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T03:48:52.659 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.659 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.661 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.661 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "service_discovery_port": { 2024-06-06T03:48:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "service_discovery_port", 2024-06-06T03:48:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8765", 2024-06-06T03:48:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "cephadm service discovery port", 2024-06-06T03:48:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "ssh_config_file": { 2024-06-06T03:48:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssh_config_file", 2024-06-06T03:48:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T03:48:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "use_agent": { 2024-06-06T03:48:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "use_agent", 2024-06-06T03:48:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T03:48:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: "use_repo_digest": { 2024-06-06T03:48:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "use_repo_digest", 2024-06-06T03:48:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T03:48:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.669 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_failed_host_check": { 2024-06-06T03:48:52.669 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_failed_host_check", 2024-06-06T03:48:52.669 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.669 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.669 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.669 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.669 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.669 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.670 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.670 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T03:48:52.670 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.670 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.670 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.670 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.670 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_stray_daemons": { 2024-06-06T03:48:52.670 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_stray_daemons", 2024-06-06T03:48:52.671 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.671 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.671 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.671 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.671 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.671 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.671 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.671 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T03:48:52.672 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.672 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.672 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.672 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.672 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_stray_hosts": { 2024-06-06T03:48:52.672 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_stray_hosts", 2024-06-06T03:48:52.672 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.672 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.672 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T03:48:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.674 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.674 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.674 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.674 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crash", 2024-06-06T03:48:52.674 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.675 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.675 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.675 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.675 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.675 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.675 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.675 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.682 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.682 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.682 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.682 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.682 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.682 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.682 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.682 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.683 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.683 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.683 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.683 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.683 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.683 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.683 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.683 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.684 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.684 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.684 INFO:teuthology.orchestra.run.smithi040.stdout: "retain_interval": { 2024-06-06T03:48:52.684 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "retain_interval", 2024-06-06T03:48:52.684 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.684 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.684 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.685 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "31536000", 2024-06-06T03:48:52.685 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.685 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.685 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.685 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T03:48:52.685 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.686 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.686 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.686 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.686 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_recent_interval": { 2024-06-06T03:48:52.686 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_recent_interval", 2024-06-06T03:48:52.686 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.687 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.687 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.687 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1209600", 2024-06-06T03:48:52.687 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.687 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.687 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.688 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T03:48:52.688 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.688 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.688 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.688 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.688 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.689 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.689 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "dashboard", 2024-06-06T03:48:52.689 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.689 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.689 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.690 INFO:teuthology.orchestra.run.smithi040.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T03:48:52.690 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T03:48:52.690 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.690 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.690 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.690 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:52.690 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.691 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.691 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.691 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.691 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.691 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.691 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.691 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.691 INFO:teuthology.orchestra.run.smithi040.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T03:48:52.692 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T03:48:52.692 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.692 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.692 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.692 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.692 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.692 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.692 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.693 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.693 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.693 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.693 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.693 INFO:teuthology.orchestra.run.smithi040.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T03:48:52.693 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T03:48:52.693 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.693 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.694 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.694 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.694 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.694 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.694 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.694 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.694 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.695 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.695 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.695 INFO:teuthology.orchestra.run.smithi040.stdout: "AUDIT_API_ENABLED": { 2024-06-06T03:48:52.695 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T03:48:52.695 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.695 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.695 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.696 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.696 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.696 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.696 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.696 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.696 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.696 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.696 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.697 INFO:teuthology.orchestra.run.smithi040.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T03:48:52.697 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T03:48:52.697 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.697 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.697 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.697 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.697 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.697 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.698 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.698 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.698 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.698 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.698 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.698 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.698 INFO:teuthology.orchestra.run.smithi040.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T03:48:52.698 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T03:48:52.699 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.699 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.699 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.699 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.699 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.699 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.699 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.699 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.700 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.700 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.700 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.700 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.700 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T03:48:52.700 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T03:48:52.700 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.700 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.701 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.701 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.701 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.701 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.701 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.701 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.701 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.701 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.702 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.702 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.702 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T03:48:52.702 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T03:48:52.702 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.702 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.702 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.702 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.703 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.703 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.703 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.703 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.703 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.703 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.703 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.704 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T03:48:52.704 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T03:48:52.704 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.704 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.704 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.704 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.704 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.704 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.705 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.705 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.705 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.705 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.705 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.705 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T03:48:52.705 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T03:48:52.706 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.706 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.706 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.706 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.706 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.706 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.706 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.706 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.707 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.707 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.707 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.707 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T03:48:52.707 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T03:48:52.707 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.707 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.708 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.708 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.708 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.708 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.708 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.708 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.708 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.708 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.709 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.709 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.709 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T03:48:52.709 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T03:48:52.709 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.709 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.709 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.709 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.710 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.710 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.710 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.710 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.710 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.710 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.710 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.710 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.711 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T03:48:52.711 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T03:48:52.711 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.711 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.711 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.711 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.711 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.712 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.712 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.712 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.712 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.712 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.712 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.712 INFO:teuthology.orchestra.run.smithi040.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T03:48:52.713 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T03:48:52.713 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.713 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.713 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.713 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.713 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.713 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.714 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.714 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.714 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.714 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.714 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.714 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.714 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T03:48:52.714 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T03:48:52.715 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.715 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.715 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.715 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2024-06-06T03:48:52.715 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.715 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.715 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.715 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.715 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.716 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.716 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.716 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.716 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T03:48:52.716 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T03:48:52.716 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.716 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.716 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.717 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.717 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.717 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.717 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.717 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.717 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.717 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.717 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.717 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.718 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_URL": { 2024-06-06T03:48:52.718 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_URL", 2024-06-06T03:48:52.718 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.718 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.718 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.718 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.718 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.718 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.719 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.719 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.719 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.719 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.719 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.719 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.719 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T03:48:52.719 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T03:48:52.719 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.720 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.720 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.720 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2024-06-06T03:48:52.720 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.720 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.720 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.720 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.721 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.721 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.721 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.721 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T03:48:52.721 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T03:48:52.721 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.721 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.722 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.722 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.722 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.722 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.722 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.722 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.722 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.723 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.723 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.723 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.723 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T03:48:52.723 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T03:48:52.723 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.723 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.724 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.724 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.724 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.724 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.724 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.724 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.724 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.724 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.725 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.725 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.725 INFO:teuthology.orchestra.run.smithi040.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T03:48:52.725 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T03:48:52.725 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.725 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.725 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.726 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.726 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.726 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.726 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.726 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.726 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.726 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.726 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.727 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.727 INFO:teuthology.orchestra.run.smithi040.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T03:48:52.727 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T03:48:52.727 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.727 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.727 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.727 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.727 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.728 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.728 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.728 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.728 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.728 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.728 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.728 INFO:teuthology.orchestra.run.smithi040.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T03:48:52.729 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T03:48:52.729 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.729 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.729 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.729 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.729 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.729 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.729 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.729 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.730 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.730 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.730 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.730 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.730 INFO:teuthology.orchestra.run.smithi040.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T03:48:52.730 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T03:48:52.730 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.730 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.731 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.731 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.731 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.731 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.731 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.731 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.731 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.731 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.732 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.732 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.732 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T03:48:52.732 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T03:48:52.732 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.732 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.732 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.732 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.732 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.733 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.733 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.733 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.733 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.733 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.733 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.733 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.733 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T03:48:52.734 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T03:48:52.734 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.734 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.734 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.734 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.734 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.734 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.734 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.734 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.735 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.735 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.735 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.735 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.735 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T03:48:52.735 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T03:48:52.735 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.735 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.736 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.736 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.736 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.736 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.736 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.736 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.736 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.736 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.737 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.737 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.737 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T03:48:52.737 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T03:48:52.737 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.737 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.737 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.738 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.738 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.738 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.738 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.738 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.738 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.738 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.738 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.739 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.739 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T03:48:52.739 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T03:48:52.739 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.739 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.739 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.739 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.740 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.740 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.740 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.740 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.740 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.740 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.740 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.740 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.740 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T03:48:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T03:48:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.742 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.742 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.742 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.742 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.742 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.742 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T03:48:52.742 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T03:48:52.742 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.743 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.743 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.743 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.743 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.743 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.743 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.743 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.743 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.744 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.744 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.744 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.744 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T03:48:52.744 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T03:48:52.744 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.744 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.744 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.745 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.745 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.745 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.745 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.745 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.745 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.745 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.745 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.746 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.746 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T03:48:52.746 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T03:48:52.746 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.746 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.746 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.746 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T03:48:52.747 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.747 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.747 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.747 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.747 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.747 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.747 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.747 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.747 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T03:48:52.748 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T03:48:52.748 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.748 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.748 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.748 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:52.748 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.748 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.748 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.749 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.749 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.749 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.749 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.749 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T03:48:52.749 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T03:48:52.749 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.750 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.750 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.750 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8", 2024-06-06T03:48:52.750 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.750 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.750 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.750 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.750 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.751 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.751 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.751 INFO:teuthology.orchestra.run.smithi040.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T03:48:52.751 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T03:48:52.751 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.751 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.751 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.751 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "45", 2024-06-06T03:48:52.752 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.752 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.752 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.752 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.752 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.752 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.752 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.753 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T03:48:52.753 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T03:48:52.753 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.753 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.753 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.753 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.753 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.753 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.753 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.754 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.754 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.754 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.754 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.754 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T03:48:52.754 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T03:48:52.754 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.755 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.755 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.755 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2024-06-06T03:48:52.755 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.755 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.755 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.755 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.755 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.756 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.756 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.756 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T03:48:52.756 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T03:48:52.756 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.756 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.756 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.756 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.757 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.757 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.757 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.757 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.757 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.757 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.757 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.757 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.758 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T03:48:52.758 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T03:48:52.758 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.758 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.758 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.758 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.758 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.758 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.759 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.759 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.759 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.759 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.759 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.759 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.759 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T03:48:52.759 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T03:48:52.760 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.760 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.760 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.760 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:52.760 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.760 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.760 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.760 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.760 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.761 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.761 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.761 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.761 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T03:48:52.761 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T03:48:52.761 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.761 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.761 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.762 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:52.762 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.762 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.762 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.762 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.762 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.762 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.762 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.763 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.763 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T03:48:52.763 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T03:48:52.763 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.763 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.763 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.763 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2024-06-06T03:48:52.763 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.763 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.764 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.764 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.764 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.764 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.764 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.764 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.764 INFO:teuthology.orchestra.run.smithi040.stdout: "cross_origin_url": { 2024-06-06T03:48:52.764 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cross_origin_url", 2024-06-06T03:48:52.764 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.765 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.765 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.765 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.765 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.765 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.765 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.765 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.765 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.766 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.766 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.766 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.766 INFO:teuthology.orchestra.run.smithi040.stdout: "crt_file": { 2024-06-06T03:48:52.766 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crt_file", 2024-06-06T03:48:52.766 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.766 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.767 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.767 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.767 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.767 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.767 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.767 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.767 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.767 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.767 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.768 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.768 INFO:teuthology.orchestra.run.smithi040.stdout: "debug": { 2024-06-06T03:48:52.768 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "debug", 2024-06-06T03:48:52.768 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.768 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.768 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.768 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.769 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.769 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.769 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.769 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable/disable debug options", 2024-06-06T03:48:52.769 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.769 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.769 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.769 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.770 INFO:teuthology.orchestra.run.smithi040.stdout: "jwt_token_ttl": { 2024-06-06T03:48:52.770 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "jwt_token_ttl", 2024-06-06T03:48:52.770 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.770 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.770 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.770 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "28800", 2024-06-06T03:48:52.770 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.770 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.771 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.771 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.771 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.771 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.771 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.771 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.771 INFO:teuthology.orchestra.run.smithi040.stdout: "key_file": { 2024-06-06T03:48:52.771 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "key_file", 2024-06-06T03:48:52.772 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.772 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.772 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.772 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.772 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.772 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.772 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.772 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.773 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.773 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.773 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.773 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.773 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.773 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.773 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.774 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.774 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.774 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.774 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.774 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.774 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.774 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.774 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.775 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.775 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.775 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.775 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.775 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.775 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.775 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.775 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.776 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.776 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.776 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.776 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.776 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.776 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.776 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.777 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.777 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.777 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.777 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.777 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.777 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.778 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.778 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.778 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.778 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.778 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.778 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.778 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.779 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.779 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.779 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.779 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.779 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.779 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.779 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.779 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.779 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.780 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.780 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.780 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.780 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.780 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.780 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.780 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.780 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.780 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.781 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.781 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.781 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.781 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.781 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.781 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.781 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.781 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.782 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.782 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.782 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.782 INFO:teuthology.orchestra.run.smithi040.stdout: "motd": { 2024-06-06T03:48:52.782 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "motd", 2024-06-06T03:48:52.782 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.782 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.783 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.783 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.783 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.783 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.783 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.783 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "The message of the day", 2024-06-06T03:48:52.783 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.784 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.784 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.784 INFO:teuthology.orchestra.run.smithi040.stdout: "redirect_resolve_ip_addr": { 2024-06-06T03:48:52.784 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T03:48:52.784 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.784 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.784 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.784 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.785 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.785 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.785 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.785 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.785 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.785 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.785 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.785 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.786 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2024-06-06T03:48:52.786 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2024-06-06T03:48:52.786 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.786 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.786 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.786 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "::", 2024-06-06T03:48:52.786 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.787 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.787 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.787 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.787 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.787 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.787 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.787 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2024-06-06T03:48:52.788 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2024-06-06T03:48:52.788 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.788 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.788 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.788 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8080", 2024-06-06T03:48:52.788 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.788 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.788 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.788 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.789 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.789 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.789 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.789 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl": { 2024-06-06T03:48:52.789 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl", 2024-06-06T03:48:52.789 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.789 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.789 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.790 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.790 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.790 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.790 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.790 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.790 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.790 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.790 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.791 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl_server_port": { 2024-06-06T03:48:52.791 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl_server_port", 2024-06-06T03:48:52.791 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.791 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.791 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.791 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8443", 2024-06-06T03:48:52.791 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.792 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.792 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.792 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.792 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.792 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.792 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.793 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_behaviour": { 2024-06-06T03:48:52.793 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_behaviour", 2024-06-06T03:48:52.793 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.793 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.793 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.793 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "redirect", 2024-06-06T03:48:52.793 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.793 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.794 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.794 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.794 INFO:teuthology.orchestra.run.smithi040.stdout: "redirect" 2024-06-06T03:48:52.794 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.794 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.794 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.794 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.794 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.795 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_error_status_code": { 2024-06-06T03:48:52.795 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_error_status_code", 2024-06-06T03:48:52.795 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.795 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.795 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.795 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "500", 2024-06-06T03:48:52.795 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "400", 2024-06-06T03:48:52.796 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "599", 2024-06-06T03:48:52.796 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.796 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.796 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.796 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.796 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.796 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.796 INFO:teuthology.orchestra.run.smithi040.stdout: "url_prefix": { 2024-06-06T03:48:52.796 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "url_prefix", 2024-06-06T03:48:52.797 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.797 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.797 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.797 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.797 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.797 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.797 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.797 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.798 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.798 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.798 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.798 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.798 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.798 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.798 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "devicehealth", 2024-06-06T03:48:52.799 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.799 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.799 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.799 INFO:teuthology.orchestra.run.smithi040.stdout: "enable_monitoring": { 2024-06-06T03:48:52.799 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enable_monitoring", 2024-06-06T03:48:52.799 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.799 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.799 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.800 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.800 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.800 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.800 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.800 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "monitor device health metrics", 2024-06-06T03:48:52.800 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.800 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.800 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.801 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.801 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.801 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.801 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.801 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.801 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.801 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.801 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.802 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.802 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.802 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.802 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.802 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.802 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.802 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.802 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.803 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.803 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.803 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.803 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.803 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.803 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.803 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.803 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.804 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.804 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.804 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.804 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.804 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.804 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.804 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.805 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.805 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.805 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.805 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.805 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.805 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.805 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.806 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.806 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.806 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.806 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.806 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.806 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.806 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.806 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.807 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.807 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.807 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.807 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.807 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.807 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.807 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.807 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.807 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.808 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.808 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.808 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.808 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.808 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.808 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.808 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.809 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.809 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.809 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.809 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.809 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.809 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.809 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.809 INFO:teuthology.orchestra.run.smithi040.stdout: "mark_out_threshold": { 2024-06-06T03:48:52.810 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mark_out_threshold", 2024-06-06T03:48:52.810 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.810 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.810 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.810 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "2419200", 2024-06-06T03:48:52.810 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.810 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.810 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.810 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T03:48:52.811 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.811 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.811 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.811 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.811 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": { 2024-06-06T03:48:52.811 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pool_name", 2024-06-06T03:48:52.811 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.812 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.812 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.812 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "device_health_metrics", 2024-06-06T03:48:52.812 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.812 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.812 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.812 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T03:48:52.812 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.813 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.813 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.813 INFO:teuthology.orchestra.run.smithi040.stdout: "retention_period": { 2024-06-06T03:48:52.813 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "retention_period", 2024-06-06T03:48:52.813 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.813 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.813 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.814 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15552000", 2024-06-06T03:48:52.814 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.814 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.814 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.814 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long to retain device health metrics", 2024-06-06T03:48:52.814 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.814 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.814 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.815 INFO:teuthology.orchestra.run.smithi040.stdout: "scrape_frequency": { 2024-06-06T03:48:52.815 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "scrape_frequency", 2024-06-06T03:48:52.815 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.815 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.815 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.815 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "86400", 2024-06-06T03:48:52.815 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.816 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.816 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.816 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T03:48:52.816 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.816 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.816 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.816 INFO:teuthology.orchestra.run.smithi040.stdout: "self_heal": { 2024-06-06T03:48:52.817 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "self_heal", 2024-06-06T03:48:52.817 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.817 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.817 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.817 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.817 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.817 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.817 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.818 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T03:48:52.818 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.818 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.818 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.818 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:52.818 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:52.818 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.819 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.819 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.819 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:52.819 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.819 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.819 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.819 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T03:48:52.819 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.820 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.820 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.820 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_threshold": { 2024-06-06T03:48:52.820 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_threshold", 2024-06-06T03:48:52.820 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.820 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.820 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.821 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "7257600", 2024-06-06T03:48:52.821 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.821 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.821 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.821 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T03:48:52.821 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.821 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.821 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.822 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.822 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.822 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.822 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.822 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "diskprediction_local", 2024-06-06T03:48:52.822 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.822 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.822 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.823 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.823 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.823 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.823 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.823 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.823 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.823 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.823 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.824 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.824 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.824 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.824 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.824 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.824 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.824 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.824 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.824 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.825 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.825 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.825 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.825 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.825 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.825 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.825 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.825 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.826 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.826 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.826 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.826 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.826 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.826 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.826 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.826 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.827 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.827 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.827 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.827 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.827 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.827 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.827 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.827 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.827 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.828 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.828 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.828 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.828 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.828 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.828 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.828 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.829 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.829 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.829 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.829 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.829 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.829 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.829 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.829 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.829 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "predict_interval": { 2024-06-06T03:48:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "predict_interval", 2024-06-06T03:48:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "86400", 2024-06-06T03:48:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "predictor_model": { 2024-06-06T03:48:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "predictor_model", 2024-06-06T03:48:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "prophetstor", 2024-06-06T03:48:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "influx", 2024-06-06T03:48:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": false, 2024-06-06T03:48:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "influxdb python module not found", 2024-06-06T03:48:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "batch_size": { 2024-06-06T03:48:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "batch_size", 2024-06-06T03:48:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5000", 2024-06-06T03:48:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T03:48:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "database": { 2024-06-06T03:48:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "database", 2024-06-06T03:48:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "ceph", 2024-06-06T03:48:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T03:48:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": { 2024-06-06T03:48:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "hostname", 2024-06-06T03:48:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB server hostname", 2024-06-06T03:48:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "30", 2024-06-06T03:48:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "5", 2024-06-06T03:48:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.844 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T03:48:52.844 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.844 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.844 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.844 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.844 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.844 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.845 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.845 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.845 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.845 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.845 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.845 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.845 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.845 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.845 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.846 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.846 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.846 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.846 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.846 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.846 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.846 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.847 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.847 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.847 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.847 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.847 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.847 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.847 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.847 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.848 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.848 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.848 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.848 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.848 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.848 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.848 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.849 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.849 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.849 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.849 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.849 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.849 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.849 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.849 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.850 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.850 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.850 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.850 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.850 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.850 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.850 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.850 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.851 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.851 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.851 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.851 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.851 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.851 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.851 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.851 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.852 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.852 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.852 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.852 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.852 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.852 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.852 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.852 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.853 INFO:teuthology.orchestra.run.smithi040.stdout: "password": { 2024-06-06T03:48:52.853 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "password", 2024-06-06T03:48:52.853 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.853 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.853 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.853 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.853 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.853 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.854 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.854 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "password of InfluxDB server user", 2024-06-06T03:48:52.854 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.854 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.854 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.854 INFO:teuthology.orchestra.run.smithi040.stdout: "port": { 2024-06-06T03:48:52.854 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "port", 2024-06-06T03:48:52.855 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.855 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.855 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.855 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8086", 2024-06-06T03:48:52.855 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.855 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.855 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.855 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB server port", 2024-06-06T03:48:52.856 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.856 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.856 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.856 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl": { 2024-06-06T03:48:52.856 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl", 2024-06-06T03:48:52.856 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.856 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.856 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.857 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "false", 2024-06-06T03:48:52.857 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.857 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.857 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.857 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T03:48:52.857 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.857 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.857 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.858 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.858 INFO:teuthology.orchestra.run.smithi040.stdout: "threads": { 2024-06-06T03:48:52.858 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "threads", 2024-06-06T03:48:52.858 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.858 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.858 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.858 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2024-06-06T03:48:52.858 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2024-06-06T03:48:52.858 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "32", 2024-06-06T03:48:52.859 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.859 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T03:48:52.859 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.859 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.859 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.859 INFO:teuthology.orchestra.run.smithi040.stdout: "username": { 2024-06-06T03:48:52.859 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "username", 2024-06-06T03:48:52.860 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.860 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.860 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.860 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.860 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.860 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.860 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.860 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "username of InfluxDB server user", 2024-06-06T03:48:52.860 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.861 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.861 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.861 INFO:teuthology.orchestra.run.smithi040.stdout: "verify_ssl": { 2024-06-06T03:48:52.861 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "verify_ssl", 2024-06-06T03:48:52.861 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.861 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.862 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.862 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "true", 2024-06-06T03:48:52.862 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.862 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.862 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.862 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T03:48:52.862 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.862 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.863 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.863 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.863 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.863 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "insights", 2024-06-06T03:48:52.863 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.863 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.864 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.864 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.864 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.864 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.864 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.864 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.864 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.864 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.864 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.865 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.865 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.865 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.865 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.865 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.865 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.865 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.865 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.866 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.866 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.866 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.866 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.866 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.866 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.866 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.866 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.867 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.867 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.867 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.867 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.867 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.869 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.869 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.869 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.870 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.870 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.870 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.870 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.870 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.870 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.870 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.870 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.871 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.871 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.871 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.871 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.871 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.871 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.871 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.871 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.871 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.872 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.872 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.872 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.872 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.872 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.872 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.872 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.873 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.873 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.873 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.873 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.873 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.873 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.873 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.873 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.873 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.874 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.874 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.874 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.874 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.874 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.874 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.875 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "iostat", 2024-06-06T03:48:52.875 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.875 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.875 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.875 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.875 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.875 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.875 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.876 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.876 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.876 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.876 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.876 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.876 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.876 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.876 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.876 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.877 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.877 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.877 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.877 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.877 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.877 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.877 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.878 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.878 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.878 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.878 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.878 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.878 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.878 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.878 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.878 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.879 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.879 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.879 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.879 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.879 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.879 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.879 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.880 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.880 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.880 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.880 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.880 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.880 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.880 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.880 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.881 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.881 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.881 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.881 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.881 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.881 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.881 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.881 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.881 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.882 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.882 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.882 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.882 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.882 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.882 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.883 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.883 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.883 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.883 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.883 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.883 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.883 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.883 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.884 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.884 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "k8sevents", 2024-06-06T03:48:52.884 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.884 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.884 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.884 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_event_retention_days": { 2024-06-06T03:48:52.884 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ceph_event_retention_days", 2024-06-06T03:48:52.884 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.885 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.885 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.885 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "7", 2024-06-06T03:48:52.885 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.885 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.885 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.885 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T03:48:52.885 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.886 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.886 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.886 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.886 INFO:teuthology.orchestra.run.smithi040.stdout: "config_check_secs": { 2024-06-06T03:48:52.886 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_check_secs", 2024-06-06T03:48:52.886 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.886 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.886 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.887 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:52.887 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "10", 2024-06-06T03:48:52.887 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.887 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.887 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T03:48:52.887 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.888 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.888 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.888 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.888 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.888 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.888 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.888 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.888 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.889 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.889 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.889 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.889 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.889 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.889 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.889 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.890 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.890 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.890 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.890 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.890 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.890 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.890 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.890 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.891 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.891 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.891 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.891 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.891 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.891 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.891 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.891 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.891 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.892 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.892 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.892 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.892 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.892 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.892 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.892 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.892 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.892 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.893 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.893 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.893 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.893 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.893 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.893 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.893 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.893 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.894 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.894 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.894 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.894 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.894 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.894 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.894 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.894 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "localpool", 2024-06-06T03:48:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "failure_domain": { 2024-06-06T03:48:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "failure_domain", 2024-06-06T03:48:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "host", 2024-06-06T03:48:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "failure domain for any created local pool", 2024-06-06T03:48:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T03:48:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.907 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.907 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.907 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.907 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.907 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": { 2024-06-06T03:48:52.907 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "min_size", 2024-06-06T03:48:52.907 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.907 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.908 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.908 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.908 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.908 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.908 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.908 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default min_size for any created local pool", 2024-06-06T03:48:52.908 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T03:48:52.908 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.909 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.909 INFO:teuthology.orchestra.run.smithi040.stdout: "num_rep": { 2024-06-06T03:48:52.909 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "num_rep", 2024-06-06T03:48:52.909 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.909 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.909 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.909 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3", 2024-06-06T03:48:52.909 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default replica count for any created local pool", 2024-06-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": { 2024-06-06T03:48:52.911 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pg_num", 2024-06-06T03:48:52.911 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:52.911 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.911 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.911 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "128", 2024-06-06T03:48:52.911 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.911 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.911 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.912 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T03:48:52.912 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.912 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.912 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.912 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.912 INFO:teuthology.orchestra.run.smithi040.stdout: "prefix": { 2024-06-06T03:48:52.912 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prefix", 2024-06-06T03:48:52.912 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.912 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.913 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.913 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.913 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.913 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.913 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.913 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "name prefix for any created local pool", 2024-06-06T03:48:52.913 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.913 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.914 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.914 INFO:teuthology.orchestra.run.smithi040.stdout: "subtree": { 2024-06-06T03:48:52.914 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "subtree", 2024-06-06T03:48:52.914 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.914 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.914 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.914 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "rack", 2024-06-06T03:48:52.914 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.915 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.915 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.915 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T03:48:52.915 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T03:48:52.915 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.915 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.915 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.915 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.916 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.916 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mds_autoscaler", 2024-06-06T03:48:52.916 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.916 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.916 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.916 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.916 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.916 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.917 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.917 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.917 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.917 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.917 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.917 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.917 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.917 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.918 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.918 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.918 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.918 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.918 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.918 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.918 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.918 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.918 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.919 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.919 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.919 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.919 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.919 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.919 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.919 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.919 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.920 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.920 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.920 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.920 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.920 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.920 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.920 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.920 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.920 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.921 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.921 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.921 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.921 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.921 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.921 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.921 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.921 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.922 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.922 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.922 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.922 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.922 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.922 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.922 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.922 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.922 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.923 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.923 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.923 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.923 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.923 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.923 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.923 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.923 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.924 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.924 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.924 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.924 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.924 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.924 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.924 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.924 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.924 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.925 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.925 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mirroring", 2024-06-06T03:48:52.925 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.925 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.925 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.925 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.925 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.926 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.926 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.926 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.926 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.926 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.926 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.926 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.926 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.926 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.927 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.927 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.927 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.927 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.927 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.927 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.927 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.927 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.928 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.928 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.928 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.928 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.928 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.928 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.928 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.928 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.929 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.929 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.929 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.929 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.929 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.929 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.929 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.929 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.929 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.930 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.930 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.930 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.930 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.930 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.930 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.930 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.930 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.931 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.931 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.931 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.931 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.931 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.931 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.931 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.931 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.931 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.932 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.932 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.932 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.932 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.932 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.932 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.932 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.932 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.933 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.933 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.933 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.933 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.933 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.933 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.933 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.933 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.934 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.934 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.934 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.934 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "nfs", 2024-06-06T03:48:52.934 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.934 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.934 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.934 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.934 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.935 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.935 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.935 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.935 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.935 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.935 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.935 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.935 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.936 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.936 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.936 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.936 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.936 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.936 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.936 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.936 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.937 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.937 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.937 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.937 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.937 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.937 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.937 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.937 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.937 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.938 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.938 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.938 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.938 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.938 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.938 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.938 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.938 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.939 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.939 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.939 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.939 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.939 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.939 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.939 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.939 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.940 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.940 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.940 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.940 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.940 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.940 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.940 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.940 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.941 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.941 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.941 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.941 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.941 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.941 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.941 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.941 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.942 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.942 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.942 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.942 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.942 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.942 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.942 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.942 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.943 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.943 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.943 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.943 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.943 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.943 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "orchestrator", 2024-06-06T03:48:52.943 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.944 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.944 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.944 INFO:teuthology.orchestra.run.smithi040.stdout: "fail_fs": { 2024-06-06T03:48:52.944 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "fail_fs", 2024-06-06T03:48:52.944 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.944 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.944 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.944 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.944 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.945 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.945 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.945 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T03:48:52.945 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.945 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.945 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.945 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.945 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.945 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.946 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.946 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.946 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.946 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.946 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.946 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.946 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.946 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.947 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.947 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.947 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.947 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.947 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.947 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.947 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.947 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.947 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.948 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.948 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.948 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.948 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.948 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.948 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.948 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.948 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.948 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.949 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.949 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.949 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.949 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.949 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.949 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.949 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.949 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.949 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.950 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.950 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.950 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.950 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.950 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.950 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.950 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.950 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.951 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.951 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.951 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.951 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.951 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.951 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.951 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.951 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.952 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.952 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.952 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.952 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.952 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.952 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.952 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.952 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.954 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator": { 2024-06-06T03:48:52.954 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "orchestrator", 2024-06-06T03:48:52.954 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.954 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.954 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.954 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.954 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.954 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.955 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.955 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm", 2024-06-06T03:48:52.955 INFO:teuthology.orchestra.run.smithi040.stdout: "rook", 2024-06-06T03:48:52.955 INFO:teuthology.orchestra.run.smithi040.stdout: "test_orchestrator" 2024-06-06T03:48:52.955 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.955 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Orchestrator backend", 2024-06-06T03:48:52.955 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.955 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.956 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.956 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.956 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.956 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.956 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.956 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "osd_perf_query", 2024-06-06T03:48:52.956 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.956 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.957 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.957 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.957 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.957 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.957 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.957 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.957 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.958 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.958 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.958 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.958 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.958 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.958 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.958 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.958 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.959 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.959 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.959 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.959 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.959 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.959 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.959 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.959 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.959 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.960 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.960 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.960 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.960 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.960 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.960 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.960 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.960 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.961 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.961 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.961 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.961 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.961 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.961 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.961 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.961 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.961 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.962 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.962 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.962 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.962 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.962 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.962 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.962 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.963 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.963 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.963 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.963 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.963 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.963 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.963 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.963 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.963 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.964 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.964 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.964 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.964 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.964 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.964 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.964 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.965 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.965 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.965 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.965 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.965 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.965 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.965 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.965 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.966 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.966 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "osd_support", 2024-06-06T03:48:52.966 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.966 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.966 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.966 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.966 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.967 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.967 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.967 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.967 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.967 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.967 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.967 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.967 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.968 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.968 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.968 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.968 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.968 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.968 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.968 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.968 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.969 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.969 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.969 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.969 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.969 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.969 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.969 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.970 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.970 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.970 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.970 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.970 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.970 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.970 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.970 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.971 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.971 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.971 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.971 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.971 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.971 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.971 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.972 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.972 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.972 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.972 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.972 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.972 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.972 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.972 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.973 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.973 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.973 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.973 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.973 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.973 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.973 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.973 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.973 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.974 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.974 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.974 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.974 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.974 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.974 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.974 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.974 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.974 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.975 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.975 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.975 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.975 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.975 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pg_autoscaler", 2024-06-06T03:48:52.975 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.975 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.975 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.975 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.976 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.976 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.976 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.976 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.976 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.976 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.976 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.976 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.976 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.976 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.981 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.981 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.981 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.981 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.981 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.981 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.981 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.981 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.982 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.982 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.982 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.982 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.982 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.982 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.982 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.982 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.983 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.983 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.983 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.983 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.983 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.983 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.983 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.983 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:52.983 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:52.984 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:52.984 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.984 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.984 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:52.984 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.984 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.984 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.984 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.985 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.985 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.985 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.985 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.985 INFO:teuthology.orchestra.run.smithi040.stdout: "threshold": { 2024-06-06T03:48:52.985 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "threshold", 2024-06-06T03:48:52.985 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:52.985 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.985 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.986 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3.0", 2024-06-06T03:48:52.986 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1.0", 2024-06-06T03:48:52.986 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.986 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.986 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "scaling threshold", 2024-06-06T03:48:52.986 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T03:48:52.986 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.986 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.987 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.987 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:52.987 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:52.987 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "progress", 2024-06-06T03:48:52.987 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:52.987 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:52.987 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:52.988 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_pg_recovery_event": { 2024-06-06T03:48:52.988 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_pg_recovery_event", 2024-06-06T03:48:52.988 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.988 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.988 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.988 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.988 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.988 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.989 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.989 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T03:48:52.989 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.989 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.989 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.989 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.989 INFO:teuthology.orchestra.run.smithi040.stdout: "enabled": { 2024-06-06T03:48:52.989 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enabled", 2024-06-06T03:48:52.989 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.990 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.990 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:52.990 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:52.990 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.990 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.990 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.990 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.990 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.991 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.991 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.991 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:52.991 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:52.991 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.991 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.991 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.992 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:52.992 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.992 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.992 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.992 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.992 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.992 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.992 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.993 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.993 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.993 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.993 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.993 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.993 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.993 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.993 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.994 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:52.994 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:52.994 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.994 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.994 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.994 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.994 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.994 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.995 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.995 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.995 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.995 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.995 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.995 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.995 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:52.995 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:52.995 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:52.996 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.996 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.996 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:52.996 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.996 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.996 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:52.996 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:52.996 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:52.997 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:52.997 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:52.997 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:52.997 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:52.997 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:52.997 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.997 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.997 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.998 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:52.998 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:52.998 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:52.998 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:52.998 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:52.998 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:52.998 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:52.999 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:52.999 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:52.999 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:52.999 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:52.999 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:52.999 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:52.999 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:52.999 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.000 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.000 INFO:teuthology.orchestra.run.smithi040.stdout: "max_completed_events": { 2024-06-06T03:48:53.000 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_completed_events", 2024-06-06T03:48:53.000 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.000 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.000 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.000 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "50", 2024-06-06T03:48:53.000 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.000 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "number of past completed events to remember", 2024-06-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:53.002 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.002 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.002 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.002 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2024-06-06T03:48:53.002 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.002 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.002 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.003 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long the module is going to sleep", 2024-06-06T03:48:53.003 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.003 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.003 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.003 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.003 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.003 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.003 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.003 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus", 2024-06-06T03:48:53.004 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.004 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.004 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.004 INFO:teuthology.orchestra.run.smithi040.stdout: "cache": { 2024-06-06T03:48:53.004 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cache", 2024-06-06T03:48:53.004 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.004 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.004 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.005 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.005 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.005 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.005 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.005 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.005 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.005 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.005 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.005 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.006 INFO:teuthology.orchestra.run.smithi040.stdout: "exclude_perf_counters": { 2024-06-06T03:48:53.006 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "exclude_perf_counters", 2024-06-06T03:48:53.006 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.006 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.006 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.006 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.006 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.006 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.007 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.007 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T03:48:53.007 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T03:48:53.007 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.007 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.007 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.007 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.007 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.007 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.008 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.008 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.008 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.008 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.008 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.008 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.008 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.008 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.008 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.009 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.009 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.009 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.009 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.009 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.009 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.009 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.009 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.010 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.010 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.010 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.010 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.010 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.010 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.010 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.010 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.010 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.011 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.011 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.011 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.011 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.011 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.011 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.011 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.012 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.012 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.012 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.012 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.012 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.012 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.012 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.012 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.012 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.013 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.013 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.013 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.013 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.013 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.013 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.013 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.013 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.013 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.013 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.014 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.014 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.014 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.014 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.014 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.014 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.014 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.014 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.014 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.015 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.015 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.015 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.015 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.015 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_stats_pools": { 2024-06-06T03:48:53.015 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_stats_pools", 2024-06-06T03:48:53.015 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.015 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.016 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.016 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.016 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.016 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.016 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.016 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.016 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.016 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.016 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.016 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.017 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T03:48:53.017 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T03:48:53.017 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.017 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.017 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.017 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "300", 2024-06-06T03:48:53.017 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.017 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.018 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.018 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.018 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.018 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.018 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.018 INFO:teuthology.orchestra.run.smithi040.stdout: "scrape_interval": { 2024-06-06T03:48:53.018 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "scrape_interval", 2024-06-06T03:48:53.018 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:53.019 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.019 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.019 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15.0", 2024-06-06T03:48:53.019 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.019 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.019 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.019 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.019 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.020 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.020 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.020 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2024-06-06T03:48:53.020 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2024-06-06T03:48:53.020 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.020 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.020 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.021 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "::", 2024-06-06T03:48:53.021 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.021 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.021 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.021 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T03:48:53.021 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.021 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.021 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.021 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.022 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2024-06-06T03:48:53.022 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2024-06-06T03:48:53.022 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.022 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.022 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.022 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "9283", 2024-06-06T03:48:53.022 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.022 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.022 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.023 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T03:48:53.023 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.023 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.023 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.023 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.023 INFO:teuthology.orchestra.run.smithi040.stdout: "stale_cache_strategy": { 2024-06-06T03:48:53.023 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "stale_cache_strategy", 2024-06-06T03:48:53.023 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.024 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.024 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.024 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "log", 2024-06-06T03:48:53.024 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.024 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.024 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.024 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.024 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.024 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.025 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.025 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.025 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_behaviour": { 2024-06-06T03:48:53.025 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_behaviour", 2024-06-06T03:48:53.025 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.025 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.025 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.025 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "default", 2024-06-06T03:48:53.025 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.026 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.026 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.026 INFO:teuthology.orchestra.run.smithi040.stdout: "default", 2024-06-06T03:48:53.026 INFO:teuthology.orchestra.run.smithi040.stdout: "error" 2024-06-06T03:48:53.026 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.026 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.026 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.026 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.027 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.027 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_error_status_code": { 2024-06-06T03:48:53.027 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_error_status_code", 2024-06-06T03:48:53.027 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.027 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.027 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.027 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "500", 2024-06-06T03:48:53.027 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "400", 2024-06-06T03:48:53.028 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "599", 2024-06-06T03:48:53.028 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.028 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.028 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.028 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.028 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.029 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.029 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_support", 2024-06-06T03:48:53.029 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.029 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.029 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.029 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.029 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.029 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.030 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.030 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.030 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.030 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.030 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.030 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.030 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.030 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.030 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.031 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.031 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.031 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.031 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.031 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.031 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.031 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.031 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.031 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.032 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.032 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.032 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.032 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.032 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.032 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.032 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.032 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.033 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.033 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.033 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.033 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.033 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.033 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.033 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.033 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.033 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.034 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.034 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.034 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.034 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.034 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.034 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.034 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.034 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.035 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.035 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.035 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.035 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.035 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.035 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.035 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.035 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.036 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.036 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.036 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.036 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.036 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.036 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.036 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.036 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.037 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.037 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.037 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.037 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.037 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.037 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.037 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.037 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.037 INFO:teuthology.orchestra.run.smithi040.stdout: "max_concurrent_snap_create": { 2024-06-06T03:48:53.038 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_concurrent_snap_create", 2024-06-06T03:48:53.038 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.038 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.038 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.038 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:53.038 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.038 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.038 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.039 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.039 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.039 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.039 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.039 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.039 INFO:teuthology.orchestra.run.smithi040.stdout: "mirror_snapshot_schedule": { 2024-06-06T03:48:53.039 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T03:48:53.039 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.039 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.040 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.040 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.040 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.040 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.040 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.040 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.040 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.040 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.040 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.041 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.041 INFO:teuthology.orchestra.run.smithi040.stdout: "trash_purge_schedule": { 2024-06-06T03:48:53.041 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "trash_purge_schedule", 2024-06-06T03:48:53.041 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.041 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.041 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.041 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.041 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.042 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.042 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.042 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.042 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.042 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.042 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.042 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.042 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.043 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.043 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.043 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "restful", 2024-06-06T03:48:53.043 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.043 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.043 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.043 INFO:teuthology.orchestra.run.smithi040.stdout: "enable_auth": { 2024-06-06T03:48:53.044 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enable_auth", 2024-06-06T03:48:53.044 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.044 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.044 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.044 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.044 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.044 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.044 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.044 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.045 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.045 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.045 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.045 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.045 INFO:teuthology.orchestra.run.smithi040.stdout: "key_file": { 2024-06-06T03:48:53.045 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "key_file", 2024-06-06T03:48:53.045 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.045 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.045 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.046 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.046 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.046 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.046 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.046 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.046 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.046 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.047 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.047 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.047 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.047 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.047 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.047 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.047 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.048 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.048 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.048 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.048 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.048 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.048 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.048 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.048 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.048 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.049 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.049 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.049 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.049 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.049 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.049 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.049 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.049 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.049 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.050 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.050 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.050 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.050 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.050 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.050 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.050 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.050 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.051 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.051 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.051 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.051 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.051 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.051 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.052 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.052 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.052 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.052 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.052 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.052 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.052 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.052 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.053 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.053 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.053 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.053 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.053 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.053 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.053 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.053 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.053 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.054 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.054 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.054 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.054 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.054 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.054 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.054 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.054 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.055 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.055 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.055 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.055 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.055 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.055 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.055 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2024-06-06T03:48:53.055 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2024-06-06T03:48:53.055 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.056 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.056 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.056 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.056 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.056 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.056 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.056 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.056 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.057 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.057 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.057 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.057 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2024-06-06T03:48:53.057 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2024-06-06T03:48:53.057 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.057 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.057 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.058 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.058 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.058 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.058 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.058 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.058 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.058 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.058 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.059 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.059 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.059 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.059 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.059 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rgw", 2024-06-06T03:48:53.059 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.059 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.059 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.059 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.060 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.060 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.060 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.060 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.060 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.060 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.060 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.060 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.061 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.061 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.061 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.061 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.061 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.061 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.061 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.061 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.062 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.062 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.062 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.062 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.062 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.062 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.062 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.062 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.063 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.063 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.063 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.063 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.063 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.063 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.063 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.063 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.064 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.064 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.064 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.064 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.064 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.064 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.064 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.064 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.065 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.065 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.065 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.065 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.065 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.065 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.065 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.065 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.065 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.066 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.066 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.066 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.066 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.066 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.066 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.066 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.066 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.067 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.067 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.067 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.067 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.067 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.067 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.067 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.067 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.067 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.067 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.068 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.068 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rook", 2024-06-06T03:48:53.068 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.068 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.068 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.068 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.068 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.068 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.069 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.069 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.069 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.069 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.069 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.069 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.069 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.069 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.069 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.070 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.070 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.070 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.070 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.070 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.070 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.070 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.070 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.071 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.071 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.071 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.071 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.071 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.071 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.071 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.072 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.073 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.073 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.074 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.074 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.074 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.074 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.074 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.074 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.074 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.074 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.075 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.075 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.075 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.075 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.075 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.075 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.075 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.075 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.076 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.076 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.076 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.076 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.076 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.076 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.076 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.076 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.076 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.077 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.077 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.077 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.077 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.077 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.077 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.077 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.077 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.078 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.078 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.078 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.078 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.078 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.078 INFO:teuthology.orchestra.run.smithi040.stdout: "storage_class": { 2024-06-06T03:48:53.078 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "storage_class", 2024-06-06T03:48:53.078 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.079 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.079 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.079 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "local", 2024-06-06T03:48:53.079 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.079 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.079 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.079 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T03:48:53.079 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.080 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.080 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.080 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.080 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.080 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.080 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "selftest", 2024-06-06T03:48:53.080 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.080 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.081 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.081 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.081 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.081 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.081 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.081 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.081 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.081 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.081 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.081 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.082 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.082 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.082 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.082 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.082 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.082 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.082 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.083 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.083 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.083 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.083 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.083 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.083 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.083 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.083 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.084 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.084 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.084 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.084 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.084 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.084 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.084 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.084 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.085 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.085 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.085 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.085 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.085 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.085 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.085 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.085 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.085 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.086 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.086 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.086 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.086 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.086 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.086 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.086 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.087 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.087 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.087 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.087 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.087 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.087 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.087 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.087 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.087 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.088 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.088 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.088 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.088 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.088 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.088 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.088 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.088 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.089 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.089 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.089 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.089 INFO:teuthology.orchestra.run.smithi040.stdout: "roption1": { 2024-06-06T03:48:53.089 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "roption1", 2024-06-06T03:48:53.089 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.089 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.090 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.090 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.090 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.090 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.090 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.090 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.090 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.090 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.091 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.091 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.091 INFO:teuthology.orchestra.run.smithi040.stdout: "roption2": { 2024-06-06T03:48:53.091 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "roption2", 2024-06-06T03:48:53.091 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.091 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.091 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.091 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "xyz", 2024-06-06T03:48:53.092 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.092 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.092 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.092 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.092 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.092 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.092 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.092 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.093 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption1": { 2024-06-06T03:48:53.093 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption1", 2024-06-06T03:48:53.093 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.093 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.093 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.093 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.093 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.093 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.094 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.094 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.094 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.094 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.094 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.094 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption2": { 2024-06-06T03:48:53.094 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption2", 2024-06-06T03:48:53.094 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.095 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.095 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.095 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.095 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.095 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.095 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.095 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.096 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.096 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.096 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.096 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption3": { 2024-06-06T03:48:53.096 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption3", 2024-06-06T03:48:53.096 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:53.096 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.096 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.097 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.097 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.097 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.097 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.097 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.097 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.097 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.097 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.098 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption4": { 2024-06-06T03:48:53.098 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption4", 2024-06-06T03:48:53.098 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.098 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.098 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.098 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.098 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.099 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.099 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.099 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.099 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.099 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.099 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.099 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption5": { 2024-06-06T03:48:53.100 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption5", 2024-06-06T03:48:53.100 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.100 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.100 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.100 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.100 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.100 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.100 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.100 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.101 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.101 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.101 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.101 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption6": { 2024-06-06T03:48:53.101 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption6", 2024-06-06T03:48:53.101 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.102 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.102 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.102 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.102 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.102 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.102 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.102 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.102 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.102 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption7": { 2024-06-06T03:48:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption7", 2024-06-06T03:48:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.104 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2024-06-06T03:48:53.104 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "42", 2024-06-06T03:48:53.104 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.104 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.104 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.104 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.104 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.104 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.105 INFO:teuthology.orchestra.run.smithi040.stdout: "testkey": { 2024-06-06T03:48:53.105 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testkey", 2024-06-06T03:48:53.105 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.105 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.105 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.105 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.105 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.105 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.105 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.106 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.106 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.106 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.106 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.106 INFO:teuthology.orchestra.run.smithi040.stdout: "testlkey": { 2024-06-06T03:48:53.106 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testlkey", 2024-06-06T03:48:53.107 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.107 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.107 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.107 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.107 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.107 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.107 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.107 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.107 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.108 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.108 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.108 INFO:teuthology.orchestra.run.smithi040.stdout: "testnewline": { 2024-06-06T03:48:53.108 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testnewline", 2024-06-06T03:48:53.108 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.108 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.108 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.108 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.109 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.109 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.109 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.109 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.109 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.109 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.109 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.109 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.110 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.110 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snap_schedule", 2024-06-06T03:48:53.110 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.110 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.110 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.110 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_m_granularity": { 2024-06-06T03:48:53.111 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_m_granularity", 2024-06-06T03:48:53.111 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.111 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.111 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.111 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.111 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.111 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.111 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.111 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T03:48:53.111 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.112 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.112 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.112 INFO:teuthology.orchestra.run.smithi040.stdout: "dump_on_update": { 2024-06-06T03:48:53.112 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "dump_on_update", 2024-06-06T03:48:53.112 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.112 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.112 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.113 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.113 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.113 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.113 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.113 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "dump database to debug log on update", 2024-06-06T03:48:53.113 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.113 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.113 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.114 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.114 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.114 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.114 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.114 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.114 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.114 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.114 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.115 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.115 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.115 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.115 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.115 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.115 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.115 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.115 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.115 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.115 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.116 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.116 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.116 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.116 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.116 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.116 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.116 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.116 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.117 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.117 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.117 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.117 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.117 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.117 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.117 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.118 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.118 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.118 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.118 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.118 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.118 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.118 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.119 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.119 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.119 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.119 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.119 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.119 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.119 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.119 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.119 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.120 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.120 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.120 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.120 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.120 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.120 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.120 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.121 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.121 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.121 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.121 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.121 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.121 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.121 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.121 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.122 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.122 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.122 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.122 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.122 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "stats", 2024-06-06T03:48:53.122 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.123 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.123 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.123 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.123 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.123 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.123 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.123 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.123 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.123 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.124 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.124 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.124 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.124 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.124 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.124 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.124 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.124 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.125 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.125 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.125 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.125 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.125 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.125 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.125 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.125 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.126 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.126 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.126 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.126 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.126 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.126 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.126 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.127 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.127 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.127 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.127 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.127 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.127 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.127 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.127 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.128 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.128 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.128 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.128 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.128 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.128 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.128 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.128 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.128 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.130 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.130 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.130 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.130 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.130 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.130 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.130 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.130 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.131 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.131 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.131 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.131 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.131 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.131 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.132 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "status", 2024-06-06T03:48:53.132 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.132 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.132 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.132 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.132 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.132 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.132 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.132 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.133 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.133 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.133 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.133 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.133 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.133 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.133 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.133 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.133 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.134 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.134 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.134 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.134 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.134 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.134 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.134 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.135 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.135 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.135 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.135 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.135 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.135 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.135 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.135 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.136 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.136 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.136 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.136 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.136 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.136 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.136 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.137 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.137 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.137 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.137 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.137 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.137 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.137 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.137 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.137 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.138 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.138 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.138 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.138 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.138 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.138 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.138 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.138 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.139 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.139 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.139 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.139 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.139 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.139 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.139 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.140 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.140 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.140 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.140 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.140 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.140 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.140 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.140 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.141 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "telegraf", 2024-06-06T03:48:53.141 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.141 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.141 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.141 INFO:teuthology.orchestra.run.smithi040.stdout: "address": { 2024-06-06T03:48:53.141 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "address", 2024-06-06T03:48:53.141 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.142 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.142 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.142 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T03:48:53.142 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.142 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.142 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.142 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.143 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.143 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.143 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.143 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:53.143 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:53.143 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.143 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.144 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.144 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15", 2024-06-06T03:48:53.144 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.144 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.144 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.144 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.144 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.144 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.144 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.145 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.145 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.145 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.145 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.145 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.145 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.145 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.145 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.146 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.146 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.146 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.146 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.146 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.146 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.146 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.146 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.147 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.147 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.147 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.147 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.147 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.147 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.147 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.148 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.148 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.148 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.148 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.148 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.148 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.148 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.148 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.148 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.149 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.149 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.149 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.149 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.149 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.149 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.149 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.149 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.150 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.150 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.150 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.150 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.150 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.150 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.150 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.151 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.151 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.151 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.151 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.151 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.151 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.151 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.151 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.152 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.152 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.152 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.152 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.152 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.152 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.152 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.152 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.153 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.153 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.153 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.153 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.153 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.153 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "telemetry", 2024-06-06T03:48:53.154 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.154 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.154 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.154 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_basic": { 2024-06-06T03:48:53.154 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_basic", 2024-06-06T03:48:53.154 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.154 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.154 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.155 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.155 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.155 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.155 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.155 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T03:48:53.155 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.155 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.155 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.156 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_crash": { 2024-06-06T03:48:53.156 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_crash", 2024-06-06T03:48:53.156 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.156 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.156 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.156 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.156 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.157 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.157 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.157 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T03:48:53.157 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.157 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.157 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.157 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_device": { 2024-06-06T03:48:53.157 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_device", 2024-06-06T03:48:53.158 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.158 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.158 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.158 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.158 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.158 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.158 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.158 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T03:48:53.159 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.159 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.159 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.159 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_ident": { 2024-06-06T03:48:53.159 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_ident", 2024-06-06T03:48:53.159 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.159 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.160 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.160 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.160 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.160 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.160 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.160 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T03:48:53.160 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.161 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.161 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.161 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_perf": { 2024-06-06T03:48:53.161 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_perf", 2024-06-06T03:48:53.161 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.161 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.161 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.162 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.162 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.162 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.162 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.162 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T03:48:53.162 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.162 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.163 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.163 INFO:teuthology.orchestra.run.smithi040.stdout: "contact": { 2024-06-06T03:48:53.163 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "contact", 2024-06-06T03:48:53.163 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.163 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.163 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.163 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.164 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.164 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.164 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.164 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.164 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.164 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.164 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.165 INFO:teuthology.orchestra.run.smithi040.stdout: "description": { 2024-06-06T03:48:53.165 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "description", 2024-06-06T03:48:53.165 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.165 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.165 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.165 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.165 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.166 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.166 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.166 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.166 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.166 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.166 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.167 INFO:teuthology.orchestra.run.smithi040.stdout: "device_url": { 2024-06-06T03:48:53.167 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_url", 2024-06-06T03:48:53.167 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.167 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.167 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.167 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T03:48:53.167 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.168 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.168 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.168 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.168 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.168 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.168 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.168 INFO:teuthology.orchestra.run.smithi040.stdout: "enabled": { 2024-06-06T03:48:53.168 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enabled", 2024-06-06T03:48:53.169 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.169 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.169 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.169 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.169 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.169 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.169 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.169 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.169 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.170 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.170 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.170 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:53.170 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:53.170 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.170 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.170 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.171 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "24", 2024-06-06T03:48:53.171 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "8", 2024-06-06T03:48:53.171 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.171 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.171 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.171 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.171 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.171 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.172 INFO:teuthology.orchestra.run.smithi040.stdout: "last_opt_revision": { 2024-06-06T03:48:53.172 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "last_opt_revision", 2024-06-06T03:48:53.172 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.172 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.172 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.172 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1", 2024-06-06T03:48:53.172 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.173 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.173 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.173 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.173 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.173 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.173 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.173 INFO:teuthology.orchestra.run.smithi040.stdout: "leaderboard": { 2024-06-06T03:48:53.173 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "leaderboard", 2024-06-06T03:48:53.174 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.174 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.174 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.174 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.174 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.174 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.174 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.174 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.175 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.175 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.175 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.175 INFO:teuthology.orchestra.run.smithi040.stdout: "leaderboard_description": { 2024-06-06T03:48:53.175 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "leaderboard_description", 2024-06-06T03:48:53.175 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.175 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.175 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.176 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.176 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.176 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.176 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.176 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.176 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.176 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.176 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.177 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.177 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.177 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.177 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.177 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.177 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.177 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.177 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.177 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.178 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.178 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.178 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.178 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.178 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.178 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.178 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.178 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.179 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.179 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.179 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.179 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.179 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.179 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.179 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.179 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.180 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.180 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.180 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.180 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.180 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.180 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.180 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.180 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.181 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.181 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.181 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.181 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.181 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.181 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.181 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.181 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.181 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.182 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.182 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.182 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.182 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.182 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.182 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.182 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.182 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.183 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.183 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.183 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.183 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.183 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.183 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.183 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.183 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.184 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.184 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.184 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.184 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.184 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.184 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.184 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.184 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.185 INFO:teuthology.orchestra.run.smithi040.stdout: "organization": { 2024-06-06T03:48:53.185 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "organization", 2024-06-06T03:48:53.185 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.185 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.185 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.185 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.185 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.185 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.186 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.186 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.186 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.186 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.186 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.186 INFO:teuthology.orchestra.run.smithi040.stdout: "proxy": { 2024-06-06T03:48:53.186 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "proxy", 2024-06-06T03:48:53.187 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.187 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.187 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.187 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.187 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.187 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.187 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.187 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.187 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.188 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.188 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.188 INFO:teuthology.orchestra.run.smithi040.stdout: "url": { 2024-06-06T03:48:53.188 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "url", 2024-06-06T03:48:53.188 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.188 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.188 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.189 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T03:48:53.189 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.189 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.189 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.189 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.189 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.189 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.189 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.190 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.190 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.190 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "test_orchestrator", 2024-06-06T03:48:53.190 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.190 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.190 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.191 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.191 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.191 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.191 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.191 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.191 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.191 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.191 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.192 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.192 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.192 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.192 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.192 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.192 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.192 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.192 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.192 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.193 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.193 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.193 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.193 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.193 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.193 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.193 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.194 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.194 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.194 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.194 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.194 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.194 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.194 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.194 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.195 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.195 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.195 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.195 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.195 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.195 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.195 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.196 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.196 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.196 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.196 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.196 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.196 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.196 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.196 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.196 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.197 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.197 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.197 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.197 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.197 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.197 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.197 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.197 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.198 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.198 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.198 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.198 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.198 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.198 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.198 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.199 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.199 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.199 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.199 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.199 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.199 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.199 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.199 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.200 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "volumes", 2024-06-06T03:48:53.200 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.200 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.200 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.200 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.200 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.200 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.200 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.200 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.201 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.201 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.201 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.201 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.201 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.201 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.201 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.201 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.202 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.202 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.202 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.202 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.202 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.202 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.202 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.202 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.203 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.203 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.203 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.203 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.203 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.203 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.203 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.203 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.203 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.204 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.204 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.204 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.204 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.204 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.204 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.204 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.204 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.205 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.205 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.205 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.205 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.205 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.205 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.205 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.205 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.205 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.206 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.206 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.206 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.206 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.206 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.206 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.206 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.206 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.207 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.207 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.207 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.207 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.207 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.207 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.207 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.207 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.208 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.208 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.208 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.208 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.208 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.208 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.208 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.208 INFO:teuthology.orchestra.run.smithi040.stdout: "max_concurrent_clones": { 2024-06-06T03:48:53.209 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_concurrent_clones", 2024-06-06T03:48:53.209 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.209 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.209 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.209 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "4", 2024-06-06T03:48:53.209 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.209 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.209 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.210 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T03:48:53.210 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.210 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.210 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.210 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.210 INFO:teuthology.orchestra.run.smithi040.stdout: "periodic_async_work": { 2024-06-06T03:48:53.210 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "periodic_async_work", 2024-06-06T03:48:53.210 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.211 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.211 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.211 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.211 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.211 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.211 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.211 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Periodically check for async work", 2024-06-06T03:48:53.211 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.212 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.212 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.212 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.212 INFO:teuthology.orchestra.run.smithi040.stdout: "snapshot_clone_delay": { 2024-06-06T03:48:53.212 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snapshot_clone_delay", 2024-06-06T03:48:53.212 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.212 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.212 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.213 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:53.213 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.213 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.213 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.213 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T03:48:53.213 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.213 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.213 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.214 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.214 INFO:teuthology.orchestra.run.smithi040.stdout: "snapshot_clone_no_wait": { 2024-06-06T03:48:53.214 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T03:48:53.214 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.214 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.214 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.214 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.214 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.215 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.215 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.215 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T03:48:53.215 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.215 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.215 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.215 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.215 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.215 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.216 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.216 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix", 2024-06-06T03:48:53.216 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.216 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.216 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.216 INFO:teuthology.orchestra.run.smithi040.stdout: "discovery_interval": { 2024-06-06T03:48:53.216 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "discovery_interval", 2024-06-06T03:48:53.216 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:53.217 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.217 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.217 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "100", 2024-06-06T03:48:53.217 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.217 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.217 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.217 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.217 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.217 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.218 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.218 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.218 INFO:teuthology.orchestra.run.smithi040.stdout: "identifier": { 2024-06-06T03:48:53.218 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "identifier", 2024-06-06T03:48:53.218 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.218 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.218 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.218 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.219 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.219 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.219 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.219 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.219 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.219 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.219 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.219 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.219 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:53.219 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.224 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.224 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.224 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.224 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.224 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.224 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.224 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.224 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.224 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.225 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.225 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.225 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.225 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.225 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.225 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.225 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.225 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.228 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.228 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.228 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.228 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.228 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.228 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.228 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.228 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.228 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.229 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.229 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.229 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.229 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.229 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.229 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.229 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_host": { 2024-06-06T03:48:53.229 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_host", 2024-06-06T03:48:53.230 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.230 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.230 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.230 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.230 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.230 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.230 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.230 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.231 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.231 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.231 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.231 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.231 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_port": { 2024-06-06T03:48:53.231 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_port", 2024-06-06T03:48:53.231 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.231 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.232 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.232 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10051", 2024-06-06T03:48:53.232 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.232 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.232 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.232 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.232 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.232 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.232 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.233 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.233 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_sender": { 2024-06-06T03:48:53.233 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_sender", 2024-06-06T03:48:53.233 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.233 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.233 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.233 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T03:48:53.233 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.234 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.234 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.234 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.234 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.234 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.234 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.234 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.234 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.235 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.235 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-06-06T03:48:53.235 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.235 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.235 INFO:teuthology.orchestra.run.smithi040.stdout: "gid": 24896, 2024-06-06T03:48:53.235 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "y", 2024-06-06T03:48:53.235 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_features": 4540701547738038271, 2024-06-06T03:48:53.235 INFO:teuthology.orchestra.run.smithi040.stdout: "available_modules": [ 2024-06-06T03:48:53.235 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.236 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "alerts", 2024-06-06T03:48:53.236 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.236 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.236 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.236 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:53.236 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:53.236 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.236 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.237 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.237 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:53.237 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.237 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.237 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.237 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T03:48:53.237 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.237 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.237 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.238 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.238 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.238 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.238 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.238 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.238 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.238 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.238 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.239 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.239 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.239 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.239 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.239 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.239 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.239 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.239 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.240 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.240 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.240 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.240 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.240 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.240 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.240 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.240 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.241 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.241 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.241 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.241 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.241 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.241 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.241 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.242 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.242 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.242 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.242 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.242 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.242 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.242 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.242 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.243 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.243 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.243 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.243 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.243 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.243 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.243 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.243 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.244 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.244 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.244 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.244 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.244 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.244 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.244 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.244 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.244 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.245 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.245 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.245 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.245 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.245 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.245 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.245 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.245 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.246 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.246 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.246 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.246 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.246 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.246 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.246 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_destination": { 2024-06-06T03:48:53.246 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_destination", 2024-06-06T03:48:53.247 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.247 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.247 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.247 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.247 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.247 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.247 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.247 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Email address to send alerts to", 2024-06-06T03:48:53.248 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.248 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.248 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.248 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.248 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_from_name": { 2024-06-06T03:48:53.248 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_from_name", 2024-06-06T03:48:53.248 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.248 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.248 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.249 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "Ceph", 2024-06-06T03:48:53.249 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.249 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.249 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.249 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Email From: name", 2024-06-06T03:48:53.249 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.249 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.249 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.250 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.250 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_host": { 2024-06-06T03:48:53.250 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_host", 2024-06-06T03:48:53.250 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.250 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.250 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.250 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.250 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.251 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.251 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.251 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP server", 2024-06-06T03:48:53.251 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.251 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.251 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.251 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.251 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_password": { 2024-06-06T03:48:53.252 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_password", 2024-06-06T03:48:53.252 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.252 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.252 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.252 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.252 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.252 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.252 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.253 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Password to authenticate with", 2024-06-06T03:48:53.253 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.253 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.253 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.253 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.253 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_port": { 2024-06-06T03:48:53.253 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_port", 2024-06-06T03:48:53.253 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.254 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.254 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.254 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "465", 2024-06-06T03:48:53.254 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.254 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.254 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.254 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP port", 2024-06-06T03:48:53.255 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.255 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.255 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.255 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_sender": { 2024-06-06T03:48:53.255 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_sender", 2024-06-06T03:48:53.255 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.255 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.256 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.256 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.256 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.256 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.256 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.256 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP envelope sender", 2024-06-06T03:48:53.256 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.256 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.257 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.257 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.257 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_ssl": { 2024-06-06T03:48:53.257 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_ssl", 2024-06-06T03:48:53.257 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.257 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.257 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.257 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.257 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.258 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.258 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.258 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T03:48:53.258 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.258 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.258 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.258 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.259 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_user": { 2024-06-06T03:48:53.259 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_user", 2024-06-06T03:48:53.259 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.259 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.259 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.259 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.259 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.259 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.259 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.260 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "User to authenticate as", 2024-06-06T03:48:53.260 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.260 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.260 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.260 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.260 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.260 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.260 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.261 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "balancer", 2024-06-06T03:48:53.261 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.261 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.261 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.261 INFO:teuthology.orchestra.run.smithi040.stdout: "active": { 2024-06-06T03:48:53.261 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "active", 2024-06-06T03:48:53.261 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.261 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.262 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.262 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.262 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.262 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.262 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.262 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T03:48:53.262 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.262 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.263 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.263 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.263 INFO:teuthology.orchestra.run.smithi040.stdout: "begin_time": { 2024-06-06T03:48:53.263 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "begin_time", 2024-06-06T03:48:53.263 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.263 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.263 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.263 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0000", 2024-06-06T03:48:53.263 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.264 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.264 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.264 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T03:48:53.264 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T03:48:53.264 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.264 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.264 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.264 INFO:teuthology.orchestra.run.smithi040.stdout: "begin_weekday": { 2024-06-06T03:48:53.264 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "begin_weekday", 2024-06-06T03:48:53.265 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:53.265 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.265 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.265 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:53.265 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0", 2024-06-06T03:48:53.265 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "6", 2024-06-06T03:48:53.265 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.265 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T03:48:53.266 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T03:48:53.266 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.266 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.266 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_max_iterations": { 2024-06-06T03:48:53.266 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_max_iterations", 2024-06-06T03:48:53.266 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:53.266 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.266 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.267 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "25", 2024-06-06T03:48:53.267 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2024-06-06T03:48:53.267 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "250", 2024-06-06T03:48:53.267 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.267 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T03:48:53.267 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.267 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.267 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.268 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_metrics": { 2024-06-06T03:48:53.268 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_metrics", 2024-06-06T03:48:53.268 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.268 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.268 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.268 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T03:48:53.268 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.268 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.268 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.268 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T03:48:53.269 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T03:48:53.269 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.269 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.269 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_step": { 2024-06-06T03:48:53.269 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_step", 2024-06-06T03:48:53.269 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:53.269 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.269 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.270 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0.5", 2024-06-06T03:48:53.270 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0.001", 2024-06-06T03:48:53.270 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "0.999", 2024-06-06T03:48:53.270 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.270 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "aggressiveness of optimization", 2024-06-06T03:48:53.270 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T03:48:53.270 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.270 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.271 INFO:teuthology.orchestra.run.smithi040.stdout: "end_time": { 2024-06-06T03:48:53.271 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "end_time", 2024-06-06T03:48:53.271 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.271 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.271 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.271 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "2359", 2024-06-06T03:48:53.272 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.272 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.272 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.272 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T03:48:53.272 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T03:48:53.272 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.272 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.273 INFO:teuthology.orchestra.run.smithi040.stdout: "end_weekday": { 2024-06-06T03:48:53.273 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "end_weekday", 2024-06-06T03:48:53.273 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:53.273 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.273 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.273 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:53.273 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0", 2024-06-06T03:48:53.273 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "6", 2024-06-06T03:48:53.273 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.274 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T03:48:53.274 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T03:48:53.274 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.274 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.274 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.274 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.274 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.275 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.275 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.275 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.275 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.275 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.275 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.275 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.275 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.276 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.276 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.276 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.276 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.276 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.276 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.276 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.276 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.277 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.277 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.277 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.277 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.277 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.277 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.277 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.277 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.278 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.278 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.278 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.278 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.278 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.278 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.278 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.279 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.279 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.279 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.279 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.279 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.279 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.279 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.280 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.280 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.280 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.280 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.280 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.280 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.280 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.280 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.280 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.281 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.281 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.281 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.281 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.281 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.281 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.281 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.282 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.282 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.282 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.282 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.282 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.282 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.282 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.282 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.283 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.283 INFO:teuthology.orchestra.run.smithi040.stdout: "min_score": { 2024-06-06T03:48:53.283 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "min_score", 2024-06-06T03:48:53.283 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:53.283 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.283 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.283 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:53.284 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.284 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.284 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.284 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T03:48:53.284 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.284 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.284 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.285 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": { 2024-06-06T03:48:53.285 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mode", 2024-06-06T03:48:53.285 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.285 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.285 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.285 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "upmap", 2024-06-06T03:48:53.285 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.285 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.286 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.286 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-compat", 2024-06-06T03:48:53.286 INFO:teuthology.orchestra.run.smithi040.stdout: "none", 2024-06-06T03:48:53.286 INFO:teuthology.orchestra.run.smithi040.stdout: "read", 2024-06-06T03:48:53.286 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap", 2024-06-06T03:48:53.286 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap-read" 2024-06-06T03:48:53.286 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.286 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Balancer mode", 2024-06-06T03:48:53.287 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.287 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.287 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.287 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_ids": { 2024-06-06T03:48:53.287 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pool_ids", 2024-06-06T03:48:53.287 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.287 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.288 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.288 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.288 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.288 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.288 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.288 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T03:48:53.288 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.288 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.289 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.289 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:53.289 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:53.289 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.289 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.289 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.289 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T03:48:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.291 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap_max_deviation": { 2024-06-06T03:48:53.291 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "upmap_max_deviation", 2024-06-06T03:48:53.291 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.291 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.291 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.291 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2024-06-06T03:48:53.291 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2024-06-06T03:48:53.291 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.292 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.292 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T03:48:53.292 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T03:48:53.292 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.292 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.292 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.292 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap_max_optimizations": { 2024-06-06T03:48:53.292 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "upmap_max_optimizations", 2024-06-06T03:48:53.293 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:53.293 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.293 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.293 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:53.293 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.293 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.294 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.294 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T03:48:53.294 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.294 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.294 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.294 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.294 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.295 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cephadm", 2024-06-06T03:48:53.295 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.295 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.295 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.295 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_down_multiplier": { 2024-06-06T03:48:53.296 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_down_multiplier", 2024-06-06T03:48:53.296 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:53.296 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.296 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.296 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3.0", 2024-06-06T03:48:53.296 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.296 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T03:48:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_refresh_rate": { 2024-06-06T03:48:53.298 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_refresh_rate", 2024-06-06T03:48:53.298 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.298 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.298 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.298 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "20", 2024-06-06T03:48:53.298 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.299 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.299 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.299 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T03:48:53.299 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.299 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.299 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.299 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_starting_port": { 2024-06-06T03:48:53.300 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_starting_port", 2024-06-06T03:48:53.300 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.300 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.300 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.300 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "4721", 2024-06-06T03:48:53.300 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.300 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.301 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.301 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T03:48:53.301 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.301 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.301 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.302 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_ptrace": { 2024-06-06T03:48:53.302 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_ptrace", 2024-06-06T03:48:53.302 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.302 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.302 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.302 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.303 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.303 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.303 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.303 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T03:48:53.303 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T03:48:53.304 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.304 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.304 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.304 INFO:teuthology.orchestra.run.smithi040.stdout: "autotune_interval": { 2024-06-06T03:48:53.304 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "autotune_interval", 2024-06-06T03:48:53.304 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.304 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.305 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.305 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:53.305 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.305 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.305 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.305 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T03:48:53.306 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.306 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.306 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.306 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.306 INFO:teuthology.orchestra.run.smithi040.stdout: "autotune_memory_target_ratio": { 2024-06-06T03:48:53.306 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T03:48:53.306 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:53.307 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.307 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.307 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0.7", 2024-06-06T03:48:53.307 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.307 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.307 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.307 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T03:48:53.307 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.308 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.308 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.308 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm_log_destination": { 2024-06-06T03:48:53.308 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cephadm_log_destination", 2024-06-06T03:48:53.308 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.308 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.308 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.308 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.309 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.309 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.309 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.309 INFO:teuthology.orchestra.run.smithi040.stdout: "file", 2024-06-06T03:48:53.309 INFO:teuthology.orchestra.run.smithi040.stdout: "file,syslog", 2024-06-06T03:48:53.309 INFO:teuthology.orchestra.run.smithi040.stdout: "syslog" 2024-06-06T03:48:53.309 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.309 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T03:48:53.310 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.310 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.310 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.310 INFO:teuthology.orchestra.run.smithi040.stdout: "cgroups_split": { 2024-06-06T03:48:53.310 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cgroups_split", 2024-06-06T03:48:53.310 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.310 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.311 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.311 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.311 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.311 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.311 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.311 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T03:48:53.311 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.311 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.312 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.312 INFO:teuthology.orchestra.run.smithi040.stdout: "config_checks_enabled": { 2024-06-06T03:48:53.312 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_checks_enabled", 2024-06-06T03:48:53.312 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.312 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.312 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.312 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.313 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.313 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.313 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.313 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T03:48:53.313 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.313 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.313 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.314 INFO:teuthology.orchestra.run.smithi040.stdout: "config_dashboard": { 2024-06-06T03:48:53.314 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_dashboard", 2024-06-06T03:48:53.314 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.314 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.314 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.314 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.314 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.314 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.315 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.315 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T03:48:53.315 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.315 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.315 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.315 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_alertmanager": { 2024-06-06T03:48:53.315 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_alertmanager", 2024-06-06T03:48:53.315 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.316 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.316 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.316 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T03:48:53.316 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.316 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.316 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.316 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2024-06-06T03:48:53.316 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_base": { 2024-06-06T03:48:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_base", 2024-06-06T03:48:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T03:48:53.318 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.318 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.318 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.318 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Container image name, without the tag", 2024-06-06T03:48:53.318 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.318 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.318 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.318 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_elasticsearch": { 2024-06-06T03:48:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_elasticsearch", 2024-06-06T03:48:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T03:48:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.320 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "elasticsearch container image", 2024-06-06T03:48:53.320 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.320 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.320 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.320 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_grafana": { 2024-06-06T03:48:53.320 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_grafana", 2024-06-06T03:48:53.320 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.321 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.321 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.321 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T03:48:53.321 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.321 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.321 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.321 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2024-06-06T03:48:53.321 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.321 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.322 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.322 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.322 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_haproxy": { 2024-06-06T03:48:53.322 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_haproxy", 2024-06-06T03:48:53.322 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.322 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.322 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.322 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T03:48:53.323 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.323 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.323 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.323 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "HAproxy container image", 2024-06-06T03:48:53.323 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.323 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.323 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.323 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_agent": { 2024-06-06T03:48:53.324 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_agent", 2024-06-06T03:48:53.324 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.324 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.324 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.324 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T03:48:53.324 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.324 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.325 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.325 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger agent container image", 2024-06-06T03:48:53.325 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.325 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.325 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.325 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_collector": { 2024-06-06T03:48:53.325 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_collector", 2024-06-06T03:48:53.325 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.326 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.326 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.326 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T03:48:53.326 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.326 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.326 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.326 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger collector container image", 2024-06-06T03:48:53.326 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.327 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.327 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.327 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.327 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_query": { 2024-06-06T03:48:53.327 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_query", 2024-06-06T03:48:53.327 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.327 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.327 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.328 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T03:48:53.328 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.328 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.328 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.328 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger query container image", 2024-06-06T03:48:53.328 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.328 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.328 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.329 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_keepalived": { 2024-06-06T03:48:53.329 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_keepalived", 2024-06-06T03:48:53.329 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.329 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.329 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.329 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T03:48:53.329 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.330 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.330 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.330 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Keepalived container image", 2024-06-06T03:48:53.330 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.330 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.331 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.331 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.331 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_loki": { 2024-06-06T03:48:53.331 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_loki", 2024-06-06T03:48:53.331 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.331 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.331 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.332 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T03:48:53.332 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.332 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.332 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.332 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Loki container image", 2024-06-06T03:48:53.332 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.333 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.333 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.333 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_node_exporter": { 2024-06-06T03:48:53.333 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_node_exporter", 2024-06-06T03:48:53.333 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.333 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.333 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.334 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T03:48:53.334 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.334 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.334 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.334 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2024-06-06T03:48:53.334 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.334 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.335 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.335 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_nvmeof": { 2024-06-06T03:48:53.335 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_nvmeof", 2024-06-06T03:48:53.335 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.335 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.335 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.336 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T03:48:53.336 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.336 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.336 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.336 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Nvme-of container image", 2024-06-06T03:48:53.336 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.336 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.337 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.337 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_prometheus": { 2024-06-06T03:48:53.337 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_prometheus", 2024-06-06T03:48:53.337 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.337 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.337 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.337 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T03:48:53.338 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.338 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.338 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.338 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2024-06-06T03:48:53.338 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.338 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.339 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.339 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_promtail": { 2024-06-06T03:48:53.339 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_promtail", 2024-06-06T03:48:53.339 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.339 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.340 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.340 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T03:48:53.340 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.340 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.340 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.340 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Promtail container image", 2024-06-06T03:48:53.341 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.341 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.341 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.341 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_samba": { 2024-06-06T03:48:53.341 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_samba", 2024-06-06T03:48:53.341 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.341 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.342 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.342 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T03:48:53.342 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.342 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.342 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.342 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Samba/SMB container image", 2024-06-06T03:48:53.342 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.342 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.343 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.343 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_snmp_gateway": { 2024-06-06T03:48:53.343 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_snmp_gateway", 2024-06-06T03:48:53.343 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.343 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.343 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.343 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T03:48:53.343 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.344 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.344 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.344 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SNMP Gateway container image", 2024-06-06T03:48:53.344 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.344 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.344 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.345 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.345 INFO:teuthology.orchestra.run.smithi040.stdout: "container_init": { 2024-06-06T03:48:53.345 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_init", 2024-06-06T03:48:53.345 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.345 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.345 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.346 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.346 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.346 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.346 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.346 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T03:48:53.346 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.346 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.346 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.347 INFO:teuthology.orchestra.run.smithi040.stdout: "daemon_cache_timeout": { 2024-06-06T03:48:53.347 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "daemon_cache_timeout", 2024-06-06T03:48:53.347 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.347 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.347 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.347 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:53.348 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.348 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.348 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.348 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T03:48:53.348 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.348 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.348 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.349 INFO:teuthology.orchestra.run.smithi040.stdout: "default_cephadm_command_timeout": { 2024-06-06T03:48:53.349 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T03:48:53.349 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.349 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.349 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.349 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "900", 2024-06-06T03:48:53.349 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.349 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.350 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.350 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T03:48:53.350 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.350 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.350 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.350 INFO:teuthology.orchestra.run.smithi040.stdout: "default_registry": { 2024-06-06T03:48:53.350 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "default_registry", 2024-06-06T03:48:53.351 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.351 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.351 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.351 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io", 2024-06-06T03:48:53.351 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.351 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.351 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.351 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T03:48:53.352 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.352 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.352 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.352 INFO:teuthology.orchestra.run.smithi040.stdout: "device_cache_timeout": { 2024-06-06T03:48:53.352 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_cache_timeout", 2024-06-06T03:48:53.352 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.352 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.353 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.353 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1800", 2024-06-06T03:48:53.353 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.353 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.353 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.353 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache device inventory", 2024-06-06T03:48:53.353 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.353 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.354 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.354 INFO:teuthology.orchestra.run.smithi040.stdout: "device_enhanced_scan": { 2024-06-06T03:48:53.354 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_enhanced_scan", 2024-06-06T03:48:53.354 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.354 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.354 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.354 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.355 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.355 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.355 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.355 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T03:48:53.355 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.355 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.355 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.356 INFO:teuthology.orchestra.run.smithi040.stdout: "facts_cache_timeout": { 2024-06-06T03:48:53.356 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "facts_cache_timeout", 2024-06-06T03:48:53.356 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.356 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.356 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.356 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:53.356 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.356 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.357 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.357 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache host facts data", 2024-06-06T03:48:53.357 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.357 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.357 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.357 INFO:teuthology.orchestra.run.smithi040.stdout: "grafana_dashboards_path": { 2024-06-06T03:48:53.357 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "grafana_dashboards_path", 2024-06-06T03:48:53.358 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.358 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.358 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.358 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T03:48:53.358 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.358 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.358 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.358 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T03:48:53.359 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.359 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.359 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.359 INFO:teuthology.orchestra.run.smithi040.stdout: "host_check_interval": { 2024-06-06T03:48:53.359 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "host_check_interval", 2024-06-06T03:48:53.359 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.359 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.360 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.360 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:53.360 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.360 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.360 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.360 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to perform a host check", 2024-06-06T03:48:53.360 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.360 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.361 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.361 INFO:teuthology.orchestra.run.smithi040.stdout: "hw_monitoring": { 2024-06-06T03:48:53.361 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "hw_monitoring", 2024-06-06T03:48:53.361 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.361 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.361 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.361 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.362 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.362 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.362 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.362 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T03:48:53.362 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.362 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.362 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.363 INFO:teuthology.orchestra.run.smithi040.stdout: "inventory_list_all": { 2024-06-06T03:48:53.363 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "inventory_list_all", 2024-06-06T03:48:53.363 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.363 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.363 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.363 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.363 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.363 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.363 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.364 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T03:48:53.364 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.364 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.364 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.364 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.364 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.364 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.365 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.365 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.365 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.365 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.365 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.365 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.365 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.365 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.365 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.366 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.366 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.366 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.366 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.366 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.366 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.366 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.366 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.367 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.367 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.367 INFO:teuthology.orchestra.run.smithi040.stdout: "log_refresh_metadata": { 2024-06-06T03:48:53.367 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_refresh_metadata", 2024-06-06T03:48:53.367 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.367 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.367 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.367 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T03:48:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.369 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.369 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.369 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.369 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.369 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.369 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.369 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.369 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.370 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T03:48:53.370 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.370 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.370 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.370 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.370 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.370 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.370 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.371 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.371 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.371 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.371 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.371 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.371 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.371 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.371 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.371 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.372 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.372 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.372 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.372 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.372 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.372 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.372 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.372 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.373 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.373 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.373 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.373 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.373 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.373 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.373 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.373 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.374 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.374 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.374 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.374 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.374 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.374 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.374 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.374 INFO:teuthology.orchestra.run.smithi040.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T03:48:53.375 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T03:48:53.375 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.375 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.375 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.375 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.375 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.375 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.375 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.375 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T03:48:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T03:48:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T03:48:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "*", 2024-06-06T03:48:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T03:48:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "max_count_per_host": { 2024-06-06T03:48:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_count_per_host", 2024-06-06T03:48:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.379 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.379 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.379 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "max number of daemons per service per host", 2024-06-06T03:48:53.379 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.379 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.379 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.379 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.379 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd_draining_count": { 2024-06-06T03:48:53.379 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_osd_draining_count", 2024-06-06T03:48:53.380 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.380 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.380 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.380 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:53.380 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.380 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.380 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.380 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T03:48:53.380 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.381 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.381 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.381 INFO:teuthology.orchestra.run.smithi040.stdout: "migration_current": { 2024-06-06T03:48:53.381 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "migration_current", 2024-06-06T03:48:53.381 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.381 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.381 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.382 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.382 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.382 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.382 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.382 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "internal - do not modify", 2024-06-06T03:48:53.382 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.382 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.382 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.382 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.383 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": { 2024-06-06T03:48:53.383 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mode", 2024-06-06T03:48:53.383 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.383 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.383 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.383 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "root", 2024-06-06T03:48:53.383 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.383 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.384 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.384 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm-package", 2024-06-06T03:48:53.384 INFO:teuthology.orchestra.run.smithi040.stdout: "root" 2024-06-06T03:48:53.384 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.384 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T03:48:53.384 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.384 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.384 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.384 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.385 INFO:teuthology.orchestra.run.smithi040.stdout: "oob_default_addr": { 2024-06-06T03:48:53.385 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "oob_default_addr", 2024-06-06T03:48:53.385 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.385 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.385 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.385 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "169.254.1.1", 2024-06-06T03:48:53.385 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.385 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.386 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.386 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T03:48:53.386 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.386 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.386 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.386 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.386 INFO:teuthology.orchestra.run.smithi040.stdout: "prometheus_alerts_path": { 2024-06-06T03:48:53.386 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus_alerts_path", 2024-06-06T03:48:53.387 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.387 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.387 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.387 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T03:48:53.387 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.387 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.387 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.388 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T03:48:53.388 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.388 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.388 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.388 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.388 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_insecure": { 2024-06-06T03:48:53.388 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_insecure", 2024-06-06T03:48:53.389 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.389 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.389 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.389 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.389 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.389 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.389 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.390 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T03:48:53.390 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.390 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.390 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.390 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_password": { 2024-06-06T03:48:53.390 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_password", 2024-06-06T03:48:53.391 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.391 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.391 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.391 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.391 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.391 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.391 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.391 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T03:48:53.392 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.392 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.392 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.392 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_url": { 2024-06-06T03:48:53.392 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_url", 2024-06-06T03:48:53.392 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.392 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.392 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.393 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.393 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.393 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.393 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.393 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T03:48:53.393 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.393 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.393 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.394 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_username": { 2024-06-06T03:48:53.394 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_username", 2024-06-06T03:48:53.394 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.394 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.394 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.394 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.394 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T03:48:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: "secure_monitoring_stack": { 2024-06-06T03:48:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "secure_monitoring_stack", 2024-06-06T03:48:53.396 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.396 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.396 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.396 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.396 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.396 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.396 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.396 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T03:48:53.397 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.397 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.397 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.397 INFO:teuthology.orchestra.run.smithi040.stdout: "service_discovery_port": { 2024-06-06T03:48:53.397 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "service_discovery_port", 2024-06-06T03:48:53.397 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.397 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.398 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.398 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8765", 2024-06-06T03:48:53.398 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.398 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.398 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.398 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "cephadm service discovery port", 2024-06-06T03:48:53.398 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.398 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.398 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.399 INFO:teuthology.orchestra.run.smithi040.stdout: "ssh_config_file": { 2024-06-06T03:48:53.399 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssh_config_file", 2024-06-06T03:48:53.399 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.399 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.399 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.399 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.399 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.400 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.400 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.400 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T03:48:53.400 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.400 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.400 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.400 INFO:teuthology.orchestra.run.smithi040.stdout: "use_agent": { 2024-06-06T03:48:53.401 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "use_agent", 2024-06-06T03:48:53.401 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.401 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.401 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.401 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.401 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.401 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.401 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.401 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T03:48:53.402 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.402 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.402 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.402 INFO:teuthology.orchestra.run.smithi040.stdout: "use_repo_digest": { 2024-06-06T03:48:53.402 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "use_repo_digest", 2024-06-06T03:48:53.402 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.402 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.403 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.403 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.403 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.403 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.403 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.403 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T03:48:53.403 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.403 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.404 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.404 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_failed_host_check": { 2024-06-06T03:48:53.404 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_failed_host_check", 2024-06-06T03:48:53.404 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.404 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.404 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.404 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.404 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.405 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.405 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.405 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T03:48:53.405 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.405 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.405 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.405 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_stray_daemons": { 2024-06-06T03:48:53.406 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_stray_daemons", 2024-06-06T03:48:53.406 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.406 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.406 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.406 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.406 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.406 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.407 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.407 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T03:48:53.407 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.407 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.407 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.407 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_stray_hosts": { 2024-06-06T03:48:53.407 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_stray_hosts", 2024-06-06T03:48:53.407 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.408 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.408 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.408 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.408 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.408 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.408 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.408 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T03:48:53.408 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.409 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.409 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.409 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.409 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.409 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.409 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crash", 2024-06-06T03:48:53.409 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.409 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.410 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.410 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.410 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.410 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.410 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.410 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.410 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.410 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.411 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.411 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.411 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.411 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.411 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.411 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.411 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.412 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.412 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.412 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.412 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.412 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.412 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.412 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.412 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.413 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.413 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.413 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.413 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.413 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.413 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.413 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.413 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.414 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.414 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.414 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.414 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.414 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.414 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.414 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.414 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.415 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.415 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.415 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.415 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.415 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.415 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.415 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.415 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.415 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.416 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.416 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.416 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.416 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.416 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.416 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.416 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.416 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.417 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.417 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.417 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.417 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.417 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.417 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.417 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.417 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.418 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.418 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.418 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.418 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.418 INFO:teuthology.orchestra.run.smithi040.stdout: "retain_interval": { 2024-06-06T03:48:53.418 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "retain_interval", 2024-06-06T03:48:53.418 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.419 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.419 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.419 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "31536000", 2024-06-06T03:48:53.419 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.419 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.419 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.419 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T03:48:53.419 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.419 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.420 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.420 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.420 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_recent_interval": { 2024-06-06T03:48:53.420 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_recent_interval", 2024-06-06T03:48:53.420 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.420 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.420 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.420 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1209600", 2024-06-06T03:48:53.421 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.421 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.421 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.421 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T03:48:53.421 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.421 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.421 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.421 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.422 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.422 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.422 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.422 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "dashboard", 2024-06-06T03:48:53.422 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.422 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.422 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.422 INFO:teuthology.orchestra.run.smithi040.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T03:48:53.422 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T03:48:53.423 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.423 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.423 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.423 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:53.423 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.423 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.423 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.423 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.424 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.424 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.424 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.424 INFO:teuthology.orchestra.run.smithi040.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T03:48:53.424 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T03:48:53.424 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.424 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.425 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.425 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.425 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.425 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.425 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.425 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.425 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.425 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.426 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.426 INFO:teuthology.orchestra.run.smithi040.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T03:48:53.426 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T03:48:53.426 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.426 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.426 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.426 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.427 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.427 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.427 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.427 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.427 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.427 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.427 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.428 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.428 INFO:teuthology.orchestra.run.smithi040.stdout: "AUDIT_API_ENABLED": { 2024-06-06T03:48:53.428 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T03:48:53.428 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.428 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.428 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.428 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.428 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.428 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.429 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.429 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.429 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.429 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.429 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.429 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.429 INFO:teuthology.orchestra.run.smithi040.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T03:48:53.429 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T03:48:53.430 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.430 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.430 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.430 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.430 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.430 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.430 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.430 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.431 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.431 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.431 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.431 INFO:teuthology.orchestra.run.smithi040.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T03:48:53.431 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T03:48:53.431 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.431 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.431 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.432 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.432 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.432 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.432 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.432 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.432 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.432 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.432 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.433 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.433 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T03:48:53.433 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T03:48:53.433 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.433 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.433 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.433 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.433 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.434 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.434 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.434 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.434 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.434 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.434 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.434 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.435 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T03:48:53.435 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T03:48:53.435 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.435 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.435 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.435 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.435 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.435 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.435 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.436 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.436 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.436 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.436 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.436 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T03:48:53.436 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T03:48:53.436 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.437 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.437 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.437 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.437 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.437 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.437 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.437 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.437 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.437 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.438 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.438 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.438 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T03:48:53.438 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T03:48:53.438 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.438 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.438 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.438 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.439 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.439 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.439 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.439 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.439 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.439 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.439 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.439 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.439 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T03:48:53.440 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T03:48:53.440 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.440 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.440 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.440 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.440 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.440 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.440 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.440 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.441 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.441 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.441 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.441 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.441 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T03:48:53.441 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T03:48:53.441 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.441 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.442 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.442 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.442 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.442 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.442 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.442 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.442 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.442 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.443 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.443 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T03:48:53.443 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T03:48:53.443 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.443 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.443 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.443 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.443 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.444 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.444 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.444 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.444 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.444 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.444 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.444 INFO:teuthology.orchestra.run.smithi040.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T03:48:53.444 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T03:48:53.445 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.445 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.445 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.445 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.445 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.445 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.445 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.445 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.446 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.446 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.446 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.446 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T03:48:53.446 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T03:48:53.446 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.446 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.446 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.447 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2024-06-06T03:48:53.447 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.447 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.447 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.447 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.447 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.447 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.447 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.448 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T03:48:53.448 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T03:48:53.448 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.448 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.448 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.448 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.448 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.448 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_URL": { 2024-06-06T03:48:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_URL", 2024-06-06T03:48:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.450 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.450 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.450 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.450 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.450 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.450 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.450 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.450 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.451 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.451 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.451 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T03:48:53.451 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T03:48:53.451 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.451 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.451 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2024-06-06T03:48:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.453 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T03:48:53.453 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T03:48:53.453 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.453 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.453 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.453 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.453 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.453 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.453 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.454 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.454 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.454 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.454 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.454 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T03:48:53.454 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T03:48:53.454 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.455 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.455 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.455 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.455 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.455 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.455 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.455 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.455 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.456 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.456 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.456 INFO:teuthology.orchestra.run.smithi040.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T03:48:53.456 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T03:48:53.456 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.456 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.456 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T03:48:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T03:48:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T03:48:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T03:48:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T03:48:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T03:48:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T03:48:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T03:48:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T03:48:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T03:48:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T03:48:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T03:48:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T03:48:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T03:48:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T03:48:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T03:48:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T03:48:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T03:48:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T03:48:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T03:48:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.474 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.474 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.474 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.474 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.474 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.474 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.474 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.475 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.475 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.475 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T03:48:53.475 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T03:48:53.475 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.475 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.475 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.475 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.475 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.476 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.476 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.476 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.476 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.476 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.476 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.476 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T03:48:53.477 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T03:48:53.477 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.477 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.477 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.477 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T03:48:53.477 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.477 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.477 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.478 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.478 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.478 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.478 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.478 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T03:48:53.478 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T03:48:53.478 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.479 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.479 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.479 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:53.479 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.479 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.479 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.479 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.479 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.480 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.480 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.480 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T03:48:53.480 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T03:48:53.480 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.480 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.480 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.481 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8", 2024-06-06T03:48:53.481 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.481 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.481 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.481 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.481 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.481 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.481 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.482 INFO:teuthology.orchestra.run.smithi040.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T03:48:53.482 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T03:48:53.482 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.482 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.482 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.482 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "45", 2024-06-06T03:48:53.482 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.482 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.483 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.483 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.483 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.483 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.483 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.483 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T03:48:53.483 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T03:48:53.484 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.484 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.484 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.484 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.484 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.484 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.484 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.484 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.484 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.485 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.485 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.485 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T03:48:53.485 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T03:48:53.485 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.485 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.485 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.486 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2024-06-06T03:48:53.486 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.486 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.486 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.486 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.486 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.486 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.486 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.487 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T03:48:53.487 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T03:48:53.487 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.487 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.487 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.487 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.487 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.488 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.488 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.488 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.488 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.488 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.488 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.488 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T03:48:53.488 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T03:48:53.489 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.489 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.489 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.489 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.489 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.489 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.489 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.489 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.490 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.490 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.490 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.490 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T03:48:53.490 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T03:48:53.490 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.490 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.491 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.491 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:53.491 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.491 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.491 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.491 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.491 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.491 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.492 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.492 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T03:48:53.492 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T03:48:53.492 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.492 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.492 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.492 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:53.492 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.493 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.493 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.493 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.493 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.493 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.493 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.493 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T03:48:53.494 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T03:48:53.494 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.494 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.494 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.494 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2024-06-06T03:48:53.494 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.494 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.494 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.495 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.495 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.495 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.495 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.495 INFO:teuthology.orchestra.run.smithi040.stdout: "cross_origin_url": { 2024-06-06T03:48:53.495 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cross_origin_url", 2024-06-06T03:48:53.495 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.495 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.496 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.496 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.496 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.496 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.496 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.496 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.496 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.496 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.497 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.497 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.497 INFO:teuthology.orchestra.run.smithi040.stdout: "crt_file": { 2024-06-06T03:48:53.497 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crt_file", 2024-06-06T03:48:53.497 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.497 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.497 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.497 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.498 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.498 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.498 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.498 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.498 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.498 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.498 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.498 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.499 INFO:teuthology.orchestra.run.smithi040.stdout: "debug": { 2024-06-06T03:48:53.499 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "debug", 2024-06-06T03:48:53.499 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.499 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.499 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.499 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.499 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.499 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.500 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.500 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable/disable debug options", 2024-06-06T03:48:53.500 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.500 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.500 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.500 INFO:teuthology.orchestra.run.smithi040.stdout: "jwt_token_ttl": { 2024-06-06T03:48:53.500 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "jwt_token_ttl", 2024-06-06T03:48:53.501 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.501 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.501 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.501 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "28800", 2024-06-06T03:48:53.501 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.501 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.501 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.501 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.502 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.502 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.502 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.502 INFO:teuthology.orchestra.run.smithi040.stdout: "key_file": { 2024-06-06T03:48:53.502 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "key_file", 2024-06-06T03:48:53.502 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.502 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.503 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.503 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.503 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.503 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.503 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.503 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.503 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.503 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.503 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.504 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.504 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.504 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.504 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.504 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.504 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.504 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.504 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.505 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.505 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.505 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.505 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.505 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.505 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.505 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.505 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.505 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.506 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.506 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.506 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.506 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.506 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.506 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.506 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.506 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.507 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.507 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.507 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.507 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.507 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.507 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.507 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.507 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.508 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.508 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.508 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.508 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.508 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.508 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.508 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.508 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.508 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.509 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.509 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.509 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.509 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.509 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.509 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.509 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.510 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.510 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.510 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.510 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.510 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.510 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.510 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.510 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.510 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.511 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.511 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.511 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.511 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.511 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.511 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.511 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.511 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.512 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.512 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.512 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.512 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.512 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.512 INFO:teuthology.orchestra.run.smithi040.stdout: "motd": { 2024-06-06T03:48:53.512 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "motd", 2024-06-06T03:48:53.512 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.512 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.513 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.513 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.513 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.513 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.513 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.513 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "The message of the day", 2024-06-06T03:48:53.513 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.513 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.514 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.514 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.514 INFO:teuthology.orchestra.run.smithi040.stdout: "redirect_resolve_ip_addr": { 2024-06-06T03:48:53.514 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T03:48:53.514 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.514 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.514 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.514 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.515 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.515 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.515 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.515 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.515 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.515 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.515 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.515 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.515 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2024-06-06T03:48:53.516 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2024-06-06T03:48:53.516 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.516 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.516 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.516 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "::", 2024-06-06T03:48:53.516 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.516 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.516 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.517 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.517 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.517 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.517 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.517 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.517 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2024-06-06T03:48:53.517 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2024-06-06T03:48:53.517 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.517 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.518 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.518 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8080", 2024-06-06T03:48:53.518 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.518 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.518 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.518 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.518 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.518 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.519 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.519 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.519 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl": { 2024-06-06T03:48:53.519 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl", 2024-06-06T03:48:53.519 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.519 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.519 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.519 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.519 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.520 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.520 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.520 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.520 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.520 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.520 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.520 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.520 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl_server_port": { 2024-06-06T03:48:53.521 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl_server_port", 2024-06-06T03:48:53.521 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.521 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.521 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.521 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8443", 2024-06-06T03:48:53.521 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.521 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.521 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.521 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.522 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.522 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.522 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.522 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.522 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_behaviour": { 2024-06-06T03:48:53.522 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_behaviour", 2024-06-06T03:48:53.522 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.522 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.523 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.523 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "redirect", 2024-06-06T03:48:53.523 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.523 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.523 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.523 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.523 INFO:teuthology.orchestra.run.smithi040.stdout: "redirect" 2024-06-06T03:48:53.523 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.524 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.524 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.524 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.524 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.524 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.524 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_error_status_code": { 2024-06-06T03:48:53.524 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_error_status_code", 2024-06-06T03:48:53.524 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.524 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.525 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.525 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "500", 2024-06-06T03:48:53.525 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "400", 2024-06-06T03:48:53.525 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "599", 2024-06-06T03:48:53.525 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.525 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.525 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.526 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.526 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.526 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.526 INFO:teuthology.orchestra.run.smithi040.stdout: "url_prefix": { 2024-06-06T03:48:53.526 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "url_prefix", 2024-06-06T03:48:53.526 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.526 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.526 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.527 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.527 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.527 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.527 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.527 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.527 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.527 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.527 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.528 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.528 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.528 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.528 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.528 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "devicehealth", 2024-06-06T03:48:53.528 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.528 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.528 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.528 INFO:teuthology.orchestra.run.smithi040.stdout: "enable_monitoring": { 2024-06-06T03:48:53.529 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enable_monitoring", 2024-06-06T03:48:53.529 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.529 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.529 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.529 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.529 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.529 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.529 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.530 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "monitor device health metrics", 2024-06-06T03:48:53.530 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.530 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.530 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.530 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.530 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.530 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.530 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.531 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.531 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.531 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.531 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.531 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.531 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.531 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.531 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.532 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.532 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.532 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.532 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.532 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.532 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.532 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.532 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.533 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.533 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.533 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.533 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.533 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.533 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.533 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.533 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.534 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.534 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.534 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.534 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.534 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.534 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.534 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.534 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.535 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.535 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.535 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.535 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.535 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.535 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.535 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.535 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.535 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.536 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.536 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.536 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.536 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.536 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.536 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.536 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.536 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.537 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.537 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.537 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.537 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.537 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.537 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.537 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.537 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.538 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.538 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.538 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.538 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.538 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.538 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.538 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.538 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.539 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.539 INFO:teuthology.orchestra.run.smithi040.stdout: "mark_out_threshold": { 2024-06-06T03:48:53.539 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mark_out_threshold", 2024-06-06T03:48:53.539 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.539 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.539 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.539 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "2419200", 2024-06-06T03:48:53.539 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.540 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.540 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.540 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T03:48:53.540 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.540 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.540 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.540 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.540 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": { 2024-06-06T03:48:53.540 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pool_name", 2024-06-06T03:48:53.541 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.541 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.541 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.541 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "device_health_metrics", 2024-06-06T03:48:53.541 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.541 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.541 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.541 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T03:48:53.542 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.542 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.542 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.542 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.542 INFO:teuthology.orchestra.run.smithi040.stdout: "retention_period": { 2024-06-06T03:48:53.542 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "retention_period", 2024-06-06T03:48:53.542 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.542 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.543 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.543 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15552000", 2024-06-06T03:48:53.543 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.543 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.543 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.543 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long to retain device health metrics", 2024-06-06T03:48:53.543 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.543 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.543 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.544 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.544 INFO:teuthology.orchestra.run.smithi040.stdout: "scrape_frequency": { 2024-06-06T03:48:53.544 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "scrape_frequency", 2024-06-06T03:48:53.544 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.544 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.544 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.544 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "86400", 2024-06-06T03:48:53.544 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.545 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.545 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.545 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T03:48:53.545 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.545 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.545 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.545 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.545 INFO:teuthology.orchestra.run.smithi040.stdout: "self_heal": { 2024-06-06T03:48:53.546 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "self_heal", 2024-06-06T03:48:53.546 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.546 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.546 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.546 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.546 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.546 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.546 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.547 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T03:48:53.547 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.547 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.547 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.547 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.547 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:53.547 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:53.547 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.548 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.548 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.548 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:53.548 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.548 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.548 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.548 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T03:48:53.548 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.548 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.549 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.549 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.549 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_threshold": { 2024-06-06T03:48:53.549 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_threshold", 2024-06-06T03:48:53.549 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.549 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.549 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.549 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "7257600", 2024-06-06T03:48:53.550 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.550 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.550 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.550 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T03:48:53.550 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.550 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.550 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.550 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.551 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.551 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.551 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.551 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "diskprediction_local", 2024-06-06T03:48:53.551 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.551 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.551 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.552 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.552 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.552 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.552 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.552 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.552 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.552 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.552 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.553 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.553 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.553 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.553 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.553 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.553 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.553 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.553 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.554 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.554 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.554 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.554 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.554 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.554 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.554 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.555 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.555 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.555 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.555 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.555 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.555 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.555 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.555 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.556 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.556 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.556 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.556 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.556 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.556 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.556 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.557 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.557 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.557 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.557 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.557 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.557 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.557 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.557 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.557 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.558 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.558 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.558 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.558 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.558 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.558 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.558 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.559 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.559 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.559 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.559 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.559 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.559 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.559 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.559 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.560 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.560 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.560 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.560 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.560 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.560 INFO:teuthology.orchestra.run.smithi040.stdout: "predict_interval": { 2024-06-06T03:48:53.560 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "predict_interval", 2024-06-06T03:48:53.560 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.561 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.561 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.561 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "86400", 2024-06-06T03:48:53.561 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.561 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.561 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.561 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.561 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.561 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.562 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.562 INFO:teuthology.orchestra.run.smithi040.stdout: "predictor_model": { 2024-06-06T03:48:53.562 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "predictor_model", 2024-06-06T03:48:53.562 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.562 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.562 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.562 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "prophetstor", 2024-06-06T03:48:53.563 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.563 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.563 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.563 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.563 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.563 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.563 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.563 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:53.564 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:53.564 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.564 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.564 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.564 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:53.564 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.564 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.564 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.564 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.565 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.565 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.565 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.565 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.565 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.565 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "influx", 2024-06-06T03:48:53.566 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": false, 2024-06-06T03:48:53.566 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "influxdb python module not found", 2024-06-06T03:48:53.566 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.566 INFO:teuthology.orchestra.run.smithi040.stdout: "batch_size": { 2024-06-06T03:48:53.566 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "batch_size", 2024-06-06T03:48:53.566 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.566 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.566 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.567 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5000", 2024-06-06T03:48:53.567 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.567 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.567 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.567 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T03:48:53.567 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.567 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.567 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.567 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.568 INFO:teuthology.orchestra.run.smithi040.stdout: "database": { 2024-06-06T03:48:53.568 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "database", 2024-06-06T03:48:53.568 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.568 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.568 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.568 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "ceph", 2024-06-06T03:48:53.568 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.568 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.569 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.569 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T03:48:53.569 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.569 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.569 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.569 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": { 2024-06-06T03:48:53.569 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "hostname", 2024-06-06T03:48:53.570 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.570 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.570 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.570 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.570 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.570 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.570 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.570 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB server hostname", 2024-06-06T03:48:53.571 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.571 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.571 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.571 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:53.571 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:53.571 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.571 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.571 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.572 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "30", 2024-06-06T03:48:53.572 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "5", 2024-06-06T03:48:53.572 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.572 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.572 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T03:48:53.572 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.572 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.572 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.573 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.573 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.573 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.573 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.573 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.573 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.573 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.573 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.574 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.574 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.574 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.574 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.574 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.574 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.574 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.574 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.574 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.575 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.575 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.575 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.575 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.575 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.575 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.575 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.576 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.576 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.576 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.576 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.576 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.576 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.576 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.576 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.576 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.577 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.577 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.577 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.577 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.577 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.577 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.577 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.577 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.578 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.578 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.578 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.578 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.578 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.578 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.578 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.578 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.579 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.579 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.579 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.579 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.579 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.579 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.579 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.579 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.580 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.580 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.580 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.580 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.580 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.580 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.580 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.580 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.581 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.581 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.581 INFO:teuthology.orchestra.run.smithi040.stdout: "password": { 2024-06-06T03:48:53.581 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "password", 2024-06-06T03:48:53.581 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.581 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.581 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.581 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.581 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.582 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.582 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.582 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "password of InfluxDB server user", 2024-06-06T03:48:53.582 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.582 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.582 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.582 INFO:teuthology.orchestra.run.smithi040.stdout: "port": { 2024-06-06T03:48:53.583 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "port", 2024-06-06T03:48:53.583 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.583 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.583 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.583 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8086", 2024-06-06T03:48:53.583 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.583 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.583 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.583 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB server port", 2024-06-06T03:48:53.584 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.584 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.584 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.584 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl": { 2024-06-06T03:48:53.584 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl", 2024-06-06T03:48:53.584 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.584 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.585 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.585 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "false", 2024-06-06T03:48:53.585 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.585 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.585 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.585 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T03:48:53.585 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.585 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.585 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.586 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.586 INFO:teuthology.orchestra.run.smithi040.stdout: "threads": { 2024-06-06T03:48:53.586 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "threads", 2024-06-06T03:48:53.586 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.586 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.586 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.586 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2024-06-06T03:48:53.586 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2024-06-06T03:48:53.587 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "32", 2024-06-06T03:48:53.587 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.587 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T03:48:53.587 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.587 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.587 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.587 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.587 INFO:teuthology.orchestra.run.smithi040.stdout: "username": { 2024-06-06T03:48:53.587 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "username", 2024-06-06T03:48:53.588 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.588 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.588 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.588 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.588 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.588 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.588 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.588 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "username of InfluxDB server user", 2024-06-06T03:48:53.588 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.589 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.589 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.589 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.589 INFO:teuthology.orchestra.run.smithi040.stdout: "verify_ssl": { 2024-06-06T03:48:53.589 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "verify_ssl", 2024-06-06T03:48:53.589 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.589 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.589 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.589 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "true", 2024-06-06T03:48:53.590 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.590 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.590 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.590 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T03:48:53.590 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.590 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.590 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.591 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.591 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.591 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "insights", 2024-06-06T03:48:53.591 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.591 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.591 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.591 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.592 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.592 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.592 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.592 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.592 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.592 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.592 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.592 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.593 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.593 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.593 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.593 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.593 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.593 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.593 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.593 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.594 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.594 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.594 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.594 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.594 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.594 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.594 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.595 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.595 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.595 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.595 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.595 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.595 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.595 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.595 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.595 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.596 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.596 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.596 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.596 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.596 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.596 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.596 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.597 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.597 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.597 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.597 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.597 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.597 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.597 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.597 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.597 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.598 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.598 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.598 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.598 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.598 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.598 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.598 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.599 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.599 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.599 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.599 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.599 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.599 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.599 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.599 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.600 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.600 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.600 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.600 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.600 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.600 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "iostat", 2024-06-06T03:48:53.600 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.601 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.601 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.601 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.601 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.601 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.601 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.601 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.601 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.601 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.602 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.602 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.602 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.602 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.602 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.602 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.602 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.602 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.603 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.603 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.603 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.603 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.603 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.603 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.603 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.603 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.604 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.604 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.604 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.604 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.604 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.604 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.604 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.604 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.605 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.605 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.605 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.605 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.605 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.605 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.605 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.605 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.606 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.606 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.606 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.606 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.606 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.606 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.606 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.606 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.607 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.607 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.607 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.607 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.607 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.607 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.607 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.608 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.608 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.608 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.608 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.608 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.608 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.608 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.608 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.608 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.609 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.609 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.609 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.609 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.609 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.609 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "k8sevents", 2024-06-06T03:48:53.609 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.610 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.610 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.610 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_event_retention_days": { 2024-06-06T03:48:53.610 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ceph_event_retention_days", 2024-06-06T03:48:53.610 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.610 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.610 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.610 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "7", 2024-06-06T03:48:53.611 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.611 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.611 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.611 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T03:48:53.611 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.611 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.611 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.611 INFO:teuthology.orchestra.run.smithi040.stdout: "config_check_secs": { 2024-06-06T03:48:53.612 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_check_secs", 2024-06-06T03:48:53.612 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.612 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.612 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.612 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:53.612 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "10", 2024-06-06T03:48:53.612 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.612 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.613 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T03:48:53.613 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.613 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.613 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.613 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.613 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.613 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.613 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.614 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.614 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.614 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.614 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.614 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.614 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.614 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.614 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.615 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.615 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.615 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.615 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.615 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.615 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.615 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.615 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.616 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.616 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.616 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.616 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.616 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.616 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.616 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.616 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.617 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.617 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.617 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.617 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.617 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.617 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.617 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.618 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.618 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.618 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.618 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.618 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.618 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.618 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.618 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.619 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.619 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.619 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.619 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.619 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.619 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.619 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.619 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.619 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.620 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.620 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.620 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.620 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.620 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.620 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.620 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.621 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.621 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.621 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.621 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.621 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.621 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.621 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.621 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.622 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "localpool", 2024-06-06T03:48:53.622 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.622 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.622 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.622 INFO:teuthology.orchestra.run.smithi040.stdout: "failure_domain": { 2024-06-06T03:48:53.623 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "failure_domain", 2024-06-06T03:48:53.623 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.623 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.623 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.623 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "host", 2024-06-06T03:48:53.623 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.623 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.623 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.623 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "failure domain for any created local pool", 2024-06-06T03:48:53.624 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T03:48:53.624 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.624 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.624 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.624 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.624 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.624 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.624 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.625 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.625 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.625 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.625 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.625 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.625 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.625 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.625 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.626 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.626 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.626 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.626 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.626 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.626 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.626 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.627 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.627 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.627 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.627 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.627 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.627 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.627 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.627 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.628 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.628 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.628 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.628 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.628 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.628 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.628 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.629 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.629 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.629 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.629 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.629 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.629 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.629 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.629 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.629 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.630 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.630 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.630 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.630 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.630 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.630 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.630 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.631 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.631 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.631 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.631 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.631 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.631 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.631 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.631 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.632 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.632 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.632 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.632 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.632 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.632 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.632 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.632 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": { 2024-06-06T03:48:53.633 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "min_size", 2024-06-06T03:48:53.633 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.633 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.633 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.633 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.633 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.633 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.633 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.634 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default min_size for any created local pool", 2024-06-06T03:48:53.634 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T03:48:53.634 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.634 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.634 INFO:teuthology.orchestra.run.smithi040.stdout: "num_rep": { 2024-06-06T03:48:53.634 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "num_rep", 2024-06-06T03:48:53.634 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.634 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.635 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.635 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3", 2024-06-06T03:48:53.635 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.635 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.635 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.635 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default replica count for any created local pool", 2024-06-06T03:48:53.635 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.635 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.636 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.636 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": { 2024-06-06T03:48:53.636 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pg_num", 2024-06-06T03:48:53.636 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.636 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.636 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.637 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "128", 2024-06-06T03:48:53.637 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.637 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.637 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.637 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T03:48:53.637 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.637 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.637 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.638 INFO:teuthology.orchestra.run.smithi040.stdout: "prefix": { 2024-06-06T03:48:53.638 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prefix", 2024-06-06T03:48:53.638 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.638 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.638 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.638 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.638 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.638 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.639 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.639 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "name prefix for any created local pool", 2024-06-06T03:48:53.639 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.639 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.639 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.639 INFO:teuthology.orchestra.run.smithi040.stdout: "subtree": { 2024-06-06T03:48:53.639 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "subtree", 2024-06-06T03:48:53.640 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.640 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.640 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.640 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "rack", 2024-06-06T03:48:53.640 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.640 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.640 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.640 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T03:48:53.640 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T03:48:53.641 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.641 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.641 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.641 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.641 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mds_autoscaler", 2024-06-06T03:48:53.641 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.642 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.642 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.642 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.642 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.642 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.642 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.642 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.642 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.643 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.643 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.643 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.643 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.643 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.643 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.643 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.643 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.643 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.644 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.644 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.644 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.644 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.644 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.644 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.644 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.645 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.645 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.645 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.645 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.645 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.645 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.645 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.645 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.645 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.646 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.646 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.646 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.646 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.646 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.646 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.646 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.647 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.647 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.647 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.647 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.647 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.647 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.647 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.647 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.648 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.648 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.648 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.648 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.648 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.648 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.648 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.648 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.649 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.649 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.649 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.649 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.649 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.649 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.649 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.649 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.649 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.650 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.650 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.650 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.650 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.650 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.650 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.650 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mirroring", 2024-06-06T03:48:53.651 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.651 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.651 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.651 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.651 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.651 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.651 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.651 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.652 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.652 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.652 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.652 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.652 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.652 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.652 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.652 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.652 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.653 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.653 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.653 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.653 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.653 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.653 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.653 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.654 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.654 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.654 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.654 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.654 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.654 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.654 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.654 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.655 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.655 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.655 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.655 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.655 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.655 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.655 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.659 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.659 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.659 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.659 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.659 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.659 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.659 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.659 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.659 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.660 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "nfs", 2024-06-06T03:48:53.660 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.660 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.660 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.660 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.660 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.660 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.660 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.661 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.661 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.661 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.661 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.661 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.661 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.661 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.661 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.662 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.662 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.662 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.662 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.662 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.662 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.662 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.662 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.663 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.663 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.663 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.663 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.663 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.663 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.663 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.663 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.664 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.664 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.664 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.664 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.664 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.664 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.664 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.665 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.665 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.665 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.665 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.665 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.665 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.665 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.665 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.666 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.666 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.666 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.666 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.666 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.666 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.666 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.666 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.667 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.667 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.667 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.667 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.667 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.667 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.667 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.668 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.668 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.668 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.668 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.668 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.668 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.668 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.668 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.668 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.669 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.669 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.669 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.669 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "orchestrator", 2024-06-06T03:48:53.669 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.669 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.669 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.670 INFO:teuthology.orchestra.run.smithi040.stdout: "fail_fs": { 2024-06-06T03:48:53.670 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "fail_fs", 2024-06-06T03:48:53.670 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.670 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.670 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.670 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.670 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.670 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.671 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.671 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T03:48:53.671 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.671 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.671 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.671 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.671 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.671 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator": { 2024-06-06T03:48:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "orchestrator", 2024-06-06T03:48:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm", 2024-06-06T03:48:53.681 INFO:teuthology.orchestra.run.smithi040.stdout: "rook", 2024-06-06T03:48:53.681 INFO:teuthology.orchestra.run.smithi040.stdout: "test_orchestrator" 2024-06-06T03:48:53.681 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.681 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Orchestrator backend", 2024-06-06T03:48:53.681 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.681 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.681 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.681 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.682 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.682 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.682 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.682 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "osd_perf_query", 2024-06-06T03:48:53.682 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.682 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.682 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.682 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.683 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.683 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.683 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.685 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.685 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.685 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.686 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.686 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.686 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.686 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.686 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.686 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.686 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.686 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.687 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.687 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.687 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.687 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.687 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.687 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.687 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.688 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.688 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.688 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.688 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.688 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.688 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.688 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.688 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.688 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.689 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.689 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.689 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.689 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.689 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.689 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.689 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.690 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.690 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.690 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.690 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.690 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.690 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.690 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.690 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.691 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.691 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.691 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.691 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.691 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.691 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.691 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.691 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.691 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.692 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.692 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.692 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.692 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.692 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.692 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.692 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.692 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.693 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.693 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.693 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.693 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.693 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.693 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.693 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.694 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "osd_support", 2024-06-06T03:48:53.694 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.694 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.694 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.694 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.694 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.694 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.694 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.694 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.695 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.695 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.695 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.695 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.695 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.695 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.695 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.695 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.696 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.696 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.696 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.696 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.696 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.696 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.696 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.697 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.697 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.697 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.697 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.697 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.697 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.697 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.697 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.697 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.698 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.698 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.698 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.698 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.698 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.698 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.698 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.698 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.699 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.699 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.699 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.699 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.699 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.699 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.699 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.699 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.699 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.700 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.700 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.700 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.700 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.700 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.700 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.700 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.700 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.701 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.701 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.701 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.701 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.701 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.701 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.701 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.701 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.701 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.702 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.702 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.702 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.702 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.702 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.702 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.702 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.702 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.703 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pg_autoscaler", 2024-06-06T03:48:53.703 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.703 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.703 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.703 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.703 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.703 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.704 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.704 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.704 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.704 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.704 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.704 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.704 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.704 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.704 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.705 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.705 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.705 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.705 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.705 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.705 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.705 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.706 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.706 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.706 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.706 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.706 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.706 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.706 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.706 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.706 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.707 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.707 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.707 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.707 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.707 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.707 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.707 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.708 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.708 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.708 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.708 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.708 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.708 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.708 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.708 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.708 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.709 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.709 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.709 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.709 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.709 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.709 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.709 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.709 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.710 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.710 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.710 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.710 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.710 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.710 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.710 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.710 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.711 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.711 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.711 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.711 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.711 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.711 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.711 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.711 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.712 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:53.712 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:53.712 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.712 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.712 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.712 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:53.712 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.713 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.713 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.713 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.713 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.713 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.713 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.713 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.713 INFO:teuthology.orchestra.run.smithi040.stdout: "threshold": { 2024-06-06T03:48:53.713 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "threshold", 2024-06-06T03:48:53.714 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:53.714 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.714 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.714 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3.0", 2024-06-06T03:48:53.714 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1.0", 2024-06-06T03:48:53.714 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.714 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.714 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "scaling threshold", 2024-06-06T03:48:53.715 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T03:48:53.715 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.715 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.715 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.715 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.715 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.715 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.715 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "progress", 2024-06-06T03:48:53.715 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.716 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.716 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.716 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_pg_recovery_event": { 2024-06-06T03:48:53.716 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_pg_recovery_event", 2024-06-06T03:48:53.716 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.716 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.716 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.716 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.717 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.717 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.717 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.717 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T03:48:53.717 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.717 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.717 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.717 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.717 INFO:teuthology.orchestra.run.smithi040.stdout: "enabled": { 2024-06-06T03:48:53.718 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enabled", 2024-06-06T03:48:53.718 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.718 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.718 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.718 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.718 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.718 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.719 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.719 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.719 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.719 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.719 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.719 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.719 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.719 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.719 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "max_completed_events": { 2024-06-06T03:48:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_completed_events", 2024-06-06T03:48:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "50", 2024-06-06T03:48:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "number of past completed events to remember", 2024-06-06T03:48:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2024-06-06T03:48:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long the module is going to sleep", 2024-06-06T03:48:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus", 2024-06-06T03:48:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "cache": { 2024-06-06T03:48:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cache", 2024-06-06T03:48:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.733 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.733 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.733 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.733 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.733 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.733 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.733 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.733 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.733 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.734 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.734 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.734 INFO:teuthology.orchestra.run.smithi040.stdout: "exclude_perf_counters": { 2024-06-06T03:48:53.734 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "exclude_perf_counters", 2024-06-06T03:48:53.734 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.734 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.734 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.734 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.735 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.735 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.735 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.735 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T03:48:53.735 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T03:48:53.735 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.735 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.735 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.736 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.736 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.736 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.736 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.736 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.736 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.736 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.736 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.736 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.737 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.737 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.737 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.737 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.737 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.737 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.737 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.737 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.738 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.738 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.738 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.738 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.738 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.738 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.738 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.738 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.739 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.739 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.739 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.739 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.739 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.739 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.739 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.739 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.739 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.740 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.740 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.740 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.740 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.740 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.740 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.740 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.740 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.740 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.741 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.741 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.741 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.741 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.741 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.741 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.741 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.741 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.742 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.742 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.742 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.742 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.742 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.742 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.742 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.742 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.743 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.743 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.743 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.743 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.743 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.743 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.743 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.743 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.744 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.744 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.744 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.744 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_stats_pools": { 2024-06-06T03:48:53.744 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_stats_pools", 2024-06-06T03:48:53.744 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.744 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.744 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.745 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.745 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.745 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.745 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.745 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.745 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.745 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.746 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.746 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.746 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T03:48:53.746 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T03:48:53.746 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.746 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.746 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.746 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "300", 2024-06-06T03:48:53.747 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.747 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.747 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.747 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.747 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.747 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.747 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.747 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.748 INFO:teuthology.orchestra.run.smithi040.stdout: "scrape_interval": { 2024-06-06T03:48:53.748 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "scrape_interval", 2024-06-06T03:48:53.748 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:53.748 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.748 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.748 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15.0", 2024-06-06T03:48:53.748 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.748 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.749 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.749 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.749 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.749 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.749 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.749 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2024-06-06T03:48:53.750 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2024-06-06T03:48:53.750 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.750 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.750 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.750 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "::", 2024-06-06T03:48:53.750 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.750 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.750 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T03:48:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2024-06-06T03:48:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2024-06-06T03:48:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "9283", 2024-06-06T03:48:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T03:48:53.753 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.753 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.753 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.753 INFO:teuthology.orchestra.run.smithi040.stdout: "stale_cache_strategy": { 2024-06-06T03:48:53.753 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "stale_cache_strategy", 2024-06-06T03:48:53.753 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.754 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.754 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.754 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "log", 2024-06-06T03:48:53.754 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.754 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.754 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.754 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.754 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.754 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.755 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.755 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_behaviour": { 2024-06-06T03:48:53.755 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_behaviour", 2024-06-06T03:48:53.755 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.755 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.755 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.756 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "default", 2024-06-06T03:48:53.756 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.756 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.756 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.756 INFO:teuthology.orchestra.run.smithi040.stdout: "default", 2024-06-06T03:48:53.756 INFO:teuthology.orchestra.run.smithi040.stdout: "error" 2024-06-06T03:48:53.756 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.756 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.756 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.757 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.757 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.757 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.757 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_error_status_code": { 2024-06-06T03:48:53.757 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_error_status_code", 2024-06-06T03:48:53.757 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.757 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.757 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.758 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "500", 2024-06-06T03:48:53.758 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "400", 2024-06-06T03:48:53.758 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "599", 2024-06-06T03:48:53.758 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.758 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.758 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.758 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.758 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.759 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.759 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.759 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.759 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.759 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_support", 2024-06-06T03:48:53.759 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.759 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.759 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.760 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.760 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.760 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.760 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.760 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.760 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.760 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.760 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.761 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.761 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.761 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.761 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.761 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.761 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.761 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.762 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.762 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.762 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.762 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.762 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.762 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.762 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.762 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.763 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.763 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.763 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.763 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.763 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.763 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.763 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.763 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.764 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.764 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.764 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.764 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.764 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.764 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.764 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.764 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.765 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.765 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.765 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.765 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.765 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.765 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.765 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.765 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.765 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.766 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.766 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.766 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.766 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.766 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.766 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.766 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.766 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.767 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.767 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.767 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.767 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.767 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.767 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.767 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.767 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.768 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.768 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.768 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.768 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.768 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.768 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.768 INFO:teuthology.orchestra.run.smithi040.stdout: "max_concurrent_snap_create": { 2024-06-06T03:48:53.768 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_concurrent_snap_create", 2024-06-06T03:48:53.769 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.769 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.769 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.769 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:53.769 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.769 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.769 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.769 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.770 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.770 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.770 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.770 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mirror_snapshot_schedule": { 2024-06-06T03:48:53.770 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T03:48:53.770 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.770 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.772 INFO:teuthology.orchestra.run.smithi040.stdout: "trash_purge_schedule": { 2024-06-06T03:48:53.772 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "trash_purge_schedule", 2024-06-06T03:48:53.772 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.772 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.772 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.772 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.772 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.774 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "restful", 2024-06-06T03:48:53.774 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.774 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.774 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.774 INFO:teuthology.orchestra.run.smithi040.stdout: "enable_auth": { 2024-06-06T03:48:53.774 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enable_auth", 2024-06-06T03:48:53.774 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: "key_file": { 2024-06-06T03:48:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "key_file", 2024-06-06T03:48:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.777 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.777 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.777 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.777 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.777 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.777 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.777 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.777 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.778 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.778 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.778 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.778 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.778 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.778 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.778 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.778 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.779 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.779 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.779 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.779 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.779 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.779 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.779 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.779 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.780 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.780 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.780 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.780 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.780 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.780 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.780 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.780 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.780 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.782 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.782 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.782 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.782 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.782 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.782 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.782 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.783 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.783 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.783 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.783 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.783 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.783 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.783 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.783 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.783 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.784 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.784 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.784 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.784 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.784 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.784 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.784 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.786 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.786 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2024-06-06T03:48:53.786 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2024-06-06T03:48:53.786 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.786 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.786 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.786 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.787 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.787 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.787 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.787 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.787 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.787 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.787 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.787 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2024-06-06T03:48:53.788 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2024-06-06T03:48:53.788 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.788 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.788 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.788 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.788 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.788 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.788 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rgw", 2024-06-06T03:48:53.790 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.790 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.790 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.790 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.790 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.790 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.790 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.790 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.794 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.794 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.794 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.794 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.794 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.794 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.794 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.796 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.796 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.796 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.796 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.796 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.796 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.796 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.796 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.796 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.797 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.797 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.797 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.797 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.797 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.797 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.797 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.797 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.798 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.798 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.798 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.798 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.798 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.798 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.798 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.799 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rook", 2024-06-06T03:48:53.799 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.799 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.799 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.799 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.799 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.799 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.799 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.799 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.800 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.800 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.800 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.800 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.800 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.800 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.800 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.800 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.800 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.801 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.801 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.801 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.801 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.801 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.801 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.801 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.801 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.802 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.802 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.802 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.802 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.802 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.802 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.802 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.802 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.803 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.803 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.803 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.803 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.803 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.803 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.803 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.803 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.804 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.804 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.804 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.804 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.804 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.804 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.804 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.804 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.804 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.805 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.805 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.805 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.805 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.805 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.805 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.805 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.805 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.806 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.806 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.806 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.806 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.806 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.806 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.806 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.806 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.806 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.807 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.807 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.807 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.807 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.807 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.807 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.807 INFO:teuthology.orchestra.run.smithi040.stdout: "storage_class": { 2024-06-06T03:48:53.807 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "storage_class", 2024-06-06T03:48:53.808 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.808 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.808 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.808 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "local", 2024-06-06T03:48:53.808 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.808 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.808 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.808 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T03:48:53.808 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.809 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.809 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.809 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.809 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.809 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.809 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "selftest", 2024-06-06T03:48:53.809 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.809 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.810 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.810 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.810 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.810 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.810 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.810 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.810 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.810 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.811 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.811 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.811 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.811 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.811 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.811 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.811 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.811 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.812 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.812 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.812 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.812 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.812 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.812 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.812 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.812 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.813 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.813 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.813 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.813 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.813 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.813 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.813 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.813 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.813 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.814 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.814 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.814 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.814 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.814 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.814 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.814 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.815 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.815 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.815 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.815 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.815 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.815 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.815 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.815 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.815 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.816 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.816 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.816 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.816 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.816 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.816 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.816 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.816 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.817 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.817 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.817 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.817 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.817 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.817 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.817 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.817 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.818 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.818 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.818 INFO:teuthology.orchestra.run.smithi040.stdout: "roption1": { 2024-06-06T03:48:53.818 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "roption1", 2024-06-06T03:48:53.818 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.818 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.818 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.819 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.819 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.819 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.819 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.819 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.819 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.819 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.819 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.819 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.820 INFO:teuthology.orchestra.run.smithi040.stdout: "roption2": { 2024-06-06T03:48:53.820 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "roption2", 2024-06-06T03:48:53.820 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.820 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.820 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.820 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "xyz", 2024-06-06T03:48:53.820 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.820 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.820 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.821 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.821 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.821 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.821 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.821 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption1": { 2024-06-06T03:48:53.821 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption1", 2024-06-06T03:48:53.821 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.822 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.822 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.822 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.822 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.822 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.822 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.822 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.822 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.823 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.823 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.823 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption2": { 2024-06-06T03:48:53.823 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption2", 2024-06-06T03:48:53.823 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.823 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.823 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.823 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.824 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.824 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.824 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.824 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.824 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.824 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.824 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.825 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption3": { 2024-06-06T03:48:53.825 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption3", 2024-06-06T03:48:53.825 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:53.825 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.825 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.825 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.825 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.825 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.826 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.826 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.826 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.826 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.826 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.826 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.826 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption4": { 2024-06-06T03:48:53.826 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption4", 2024-06-06T03:48:53.826 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.827 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.827 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.827 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.827 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.827 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.827 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.827 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.828 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.828 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.828 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.828 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.828 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption5": { 2024-06-06T03:48:53.828 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption5", 2024-06-06T03:48:53.828 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.828 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.828 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.829 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.829 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.829 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.829 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.829 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.829 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.829 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.829 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.830 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.830 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption6": { 2024-06-06T03:48:53.830 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption6", 2024-06-06T03:48:53.830 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.830 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.830 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.830 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.830 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.831 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.831 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.831 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.831 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.831 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.831 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.831 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.831 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption7": { 2024-06-06T03:48:53.831 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption7", 2024-06-06T03:48:53.832 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.832 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.832 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.832 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.832 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2024-06-06T03:48:53.832 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "42", 2024-06-06T03:48:53.832 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.832 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.833 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.833 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.833 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.833 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.833 INFO:teuthology.orchestra.run.smithi040.stdout: "testkey": { 2024-06-06T03:48:53.833 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testkey", 2024-06-06T03:48:53.833 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.833 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.834 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.834 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.834 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.834 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.834 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.834 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.834 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.834 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.835 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.835 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.835 INFO:teuthology.orchestra.run.smithi040.stdout: "testlkey": { 2024-06-06T03:48:53.835 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testlkey", 2024-06-06T03:48:53.835 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.835 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.835 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.835 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.836 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.836 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.836 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.836 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.836 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.836 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.836 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.836 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.837 INFO:teuthology.orchestra.run.smithi040.stdout: "testnewline": { 2024-06-06T03:48:53.837 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testnewline", 2024-06-06T03:48:53.837 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.837 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.837 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.837 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.837 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.837 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.837 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.838 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.838 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.838 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.838 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.838 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.838 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.838 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.838 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.839 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snap_schedule", 2024-06-06T03:48:53.839 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.839 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.839 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.839 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_m_granularity": { 2024-06-06T03:48:53.839 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_m_granularity", 2024-06-06T03:48:53.839 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.839 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.839 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.840 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.840 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.840 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.840 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.840 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T03:48:53.840 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.840 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.840 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.841 INFO:teuthology.orchestra.run.smithi040.stdout: "dump_on_update": { 2024-06-06T03:48:53.841 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "dump_on_update", 2024-06-06T03:48:53.841 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.841 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.841 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.841 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.841 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.841 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.842 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.842 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "dump database to debug log on update", 2024-06-06T03:48:53.842 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.842 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.842 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.842 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.842 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.843 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.843 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.843 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.843 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.843 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.843 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.843 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.843 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.844 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.844 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.844 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.844 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.844 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.844 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.844 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.844 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.844 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.845 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.845 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.845 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.845 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.845 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.845 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.845 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.845 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.846 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.846 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.846 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.846 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.846 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.846 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.846 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.846 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.847 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.847 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.847 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.847 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.847 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.847 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.847 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.847 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.847 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.848 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.848 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.848 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.848 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.848 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.848 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.848 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.848 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.849 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.849 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.849 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.849 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.849 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.849 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.849 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.850 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.850 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.850 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.850 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.850 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.850 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.850 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.850 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.850 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.851 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.851 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "stats", 2024-06-06T03:48:53.851 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.851 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.851 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.851 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.852 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.852 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.852 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.852 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.852 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.852 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.852 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.852 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.853 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.853 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.853 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.853 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.853 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.853 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.853 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.853 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.853 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.854 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.854 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.854 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.854 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.854 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.854 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.854 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.855 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.855 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.855 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.855 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.855 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.855 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.855 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.855 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.856 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.856 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.856 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.856 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.856 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.856 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.856 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.857 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.857 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.857 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.857 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.857 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.857 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.857 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.857 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.858 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.858 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.858 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.858 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.858 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.858 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.858 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.858 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.859 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.859 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.859 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.859 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.859 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.859 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.859 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.859 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.859 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.860 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.860 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.860 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.860 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.860 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.860 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.860 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "status", 2024-06-06T03:48:53.860 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.861 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.861 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.861 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.861 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.861 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.861 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.861 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.861 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.861 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.862 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.862 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.862 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.862 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.862 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.862 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.862 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.862 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.863 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.863 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.863 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.863 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.863 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.863 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.863 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.864 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.864 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.864 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.864 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.864 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.864 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.864 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.864 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.864 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.865 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.865 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.865 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.865 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.865 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.865 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.865 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.865 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.866 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.866 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.866 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.866 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.866 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.866 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.866 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.866 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.867 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.867 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.867 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.867 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.867 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.867 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.867 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.867 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.867 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.868 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.868 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.868 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.868 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.868 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.868 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.868 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.868 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.868 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.869 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.869 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.869 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.869 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.869 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.869 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.869 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "telegraf", 2024-06-06T03:48:53.869 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.869 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.870 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.870 INFO:teuthology.orchestra.run.smithi040.stdout: "address": { 2024-06-06T03:48:53.870 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "address", 2024-06-06T03:48:53.870 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.870 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.870 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.870 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T03:48:53.870 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.871 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.871 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.871 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.871 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.871 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.871 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.871 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.871 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:53.872 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:53.872 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.872 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.872 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.872 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15", 2024-06-06T03:48:53.872 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.872 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.872 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.873 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.873 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.873 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.873 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.873 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.873 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.873 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.873 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.874 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.874 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.874 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.874 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.874 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.874 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.874 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.874 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.875 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.875 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.875 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.875 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.875 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.875 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.875 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.875 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.875 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.876 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.876 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.876 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.876 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.876 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.876 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.876 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.876 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.877 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.877 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.877 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.877 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.877 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.877 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.877 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.877 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.878 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.878 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.878 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.878 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.878 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.878 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.878 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.878 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.879 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.879 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.879 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.879 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.879 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.879 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.879 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.879 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.879 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.880 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.880 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.880 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.880 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.880 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.880 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.880 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.880 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.881 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.881 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.881 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.881 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.881 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.881 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.881 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.881 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.882 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "telemetry", 2024-06-06T03:48:53.882 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.882 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.882 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.882 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_basic": { 2024-06-06T03:48:53.882 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_basic", 2024-06-06T03:48:53.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.883 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.883 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.883 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.883 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.883 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.883 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.883 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T03:48:53.883 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.883 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.884 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.884 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.884 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_crash": { 2024-06-06T03:48:53.884 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_crash", 2024-06-06T03:48:53.884 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.884 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.884 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.884 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.885 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.885 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.885 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.885 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T03:48:53.885 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.885 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.885 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.885 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_device": { 2024-06-06T03:48:53.886 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_device", 2024-06-06T03:48:53.886 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.886 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.886 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.886 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.886 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.886 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.886 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.887 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T03:48:53.887 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.887 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.887 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.887 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.887 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_ident": { 2024-06-06T03:48:53.887 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_ident", 2024-06-06T03:48:53.887 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.888 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.888 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.888 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.888 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.888 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.888 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.888 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T03:48:53.888 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.888 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_perf": { 2024-06-06T03:48:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_perf", 2024-06-06T03:48:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T03:48:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "contact": { 2024-06-06T03:48:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "contact", 2024-06-06T03:48:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "description": { 2024-06-06T03:48:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "description", 2024-06-06T03:48:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "device_url": { 2024-06-06T03:48:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_url", 2024-06-06T03:48:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T03:48:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "enabled": { 2024-06-06T03:48:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enabled", 2024-06-06T03:48:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "24", 2024-06-06T03:48:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "8", 2024-06-06T03:48:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: "last_opt_revision": { 2024-06-06T03:48:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "last_opt_revision", 2024-06-06T03:48:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.900 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.900 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.900 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1", 2024-06-06T03:48:53.900 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.900 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.900 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.900 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.900 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.901 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.901 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.901 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.901 INFO:teuthology.orchestra.run.smithi040.stdout: "leaderboard": { 2024-06-06T03:48:53.901 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "leaderboard", 2024-06-06T03:48:53.901 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.901 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.901 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.902 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.902 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.902 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.902 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.902 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.902 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.902 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.902 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.903 INFO:teuthology.orchestra.run.smithi040.stdout: "leaderboard_description": { 2024-06-06T03:48:53.903 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "leaderboard_description", 2024-06-06T03:48:53.903 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.903 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.903 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.903 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.903 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.903 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.904 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.904 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.904 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.904 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.904 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.904 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.904 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.904 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.905 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.905 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.905 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.905 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.905 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.905 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.905 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.905 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.905 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.906 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.906 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.906 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.906 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.906 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.906 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.906 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.906 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.907 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.907 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.907 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.907 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.907 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.907 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.907 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.907 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.908 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.908 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.908 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.908 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.908 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.908 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.908 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.909 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.909 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.909 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.909 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.909 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.909 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.909 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.909 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.909 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.910 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.910 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.910 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.910 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.910 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.910 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.910 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.910 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.911 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.911 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.911 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.911 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.911 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.911 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.911 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.911 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.912 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.912 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.912 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.912 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.912 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.912 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.912 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.912 INFO:teuthology.orchestra.run.smithi040.stdout: "organization": { 2024-06-06T03:48:53.913 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "organization", 2024-06-06T03:48:53.913 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.913 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.913 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.913 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.913 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.913 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.913 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.913 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.914 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.914 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.914 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.914 INFO:teuthology.orchestra.run.smithi040.stdout: "proxy": { 2024-06-06T03:48:53.914 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "proxy", 2024-06-06T03:48:53.914 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.915 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.915 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.915 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.915 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.915 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.915 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.915 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.915 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.915 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.916 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.916 INFO:teuthology.orchestra.run.smithi040.stdout: "url": { 2024-06-06T03:48:53.916 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "url", 2024-06-06T03:48:53.916 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.916 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.916 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.916 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T03:48:53.916 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.917 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.917 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.917 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.917 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.917 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.917 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.917 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.917 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.918 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.918 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.918 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "test_orchestrator", 2024-06-06T03:48:53.918 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.918 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.918 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.918 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.918 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.919 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.919 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.919 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.919 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.919 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.919 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.919 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.919 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.920 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.920 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.920 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.920 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.920 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.920 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.920 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.920 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.920 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.921 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.921 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.921 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.921 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.921 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.921 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.921 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.921 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.921 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.922 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.922 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.922 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.922 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.922 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.922 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.922 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.922 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.923 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.923 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.923 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.923 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.923 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.923 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.923 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.923 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.924 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.924 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.924 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.924 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.924 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.924 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.924 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.924 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.924 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.925 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.925 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.925 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.925 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.925 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.925 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.925 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.925 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.925 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.926 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.926 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.926 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.926 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.926 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.926 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.926 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.926 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.927 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.927 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.927 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "volumes", 2024-06-06T03:48:53.927 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.927 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.927 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.927 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.928 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.928 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.928 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.928 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.928 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.928 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.928 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.928 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.928 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.929 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.929 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.929 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.929 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.929 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.929 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.929 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.929 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.929 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.930 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.930 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.930 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.930 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.930 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.930 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.930 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.930 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.931 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.931 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.931 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.931 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.931 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.931 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.931 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.931 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.932 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.932 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.932 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.932 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.932 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.932 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.932 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.932 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.932 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.933 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.933 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.933 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.933 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.933 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.933 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.933 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.933 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.934 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.934 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.934 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.934 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.934 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.934 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.934 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.934 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.935 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.935 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.935 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.935 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.935 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.935 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.935 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.935 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.936 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.936 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.936 INFO:teuthology.orchestra.run.smithi040.stdout: "max_concurrent_clones": { 2024-06-06T03:48:53.936 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_concurrent_clones", 2024-06-06T03:48:53.936 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.936 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.936 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.936 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "4", 2024-06-06T03:48:53.937 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.937 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.937 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.937 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T03:48:53.937 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.937 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.937 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.937 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.938 INFO:teuthology.orchestra.run.smithi040.stdout: "periodic_async_work": { 2024-06-06T03:48:53.938 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "periodic_async_work", 2024-06-06T03:48:53.938 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.938 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.938 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.938 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.938 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.938 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.939 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.939 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Periodically check for async work", 2024-06-06T03:48:53.939 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.939 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.939 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.939 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.939 INFO:teuthology.orchestra.run.smithi040.stdout: "snapshot_clone_delay": { 2024-06-06T03:48:53.939 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snapshot_clone_delay", 2024-06-06T03:48:53.940 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.940 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.940 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.940 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:53.940 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.940 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.940 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.940 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T03:48:53.941 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.941 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.941 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.941 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.941 INFO:teuthology.orchestra.run.smithi040.stdout: "snapshot_clone_no_wait": { 2024-06-06T03:48:53.941 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T03:48:53.941 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.941 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.942 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.942 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.942 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.942 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.942 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.942 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T03:48:53.942 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.943 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.943 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.943 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.943 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.943 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.943 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix", 2024-06-06T03:48:53.943 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.944 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.944 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.944 INFO:teuthology.orchestra.run.smithi040.stdout: "discovery_interval": { 2024-06-06T03:48:53.944 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "discovery_interval", 2024-06-06T03:48:53.944 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:53.944 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.944 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.944 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "100", 2024-06-06T03:48:53.945 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.945 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.945 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.945 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.945 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.945 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.945 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.945 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.946 INFO:teuthology.orchestra.run.smithi040.stdout: "identifier": { 2024-06-06T03:48:53.946 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "identifier", 2024-06-06T03:48:53.946 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.946 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.946 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.946 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.946 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.947 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.947 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.947 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.947 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.947 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.947 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.947 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.947 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:53.947 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:53.948 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.948 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.948 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.948 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:53.948 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.948 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.948 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.948 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.949 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.949 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.949 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.949 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.949 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.949 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.949 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.949 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.949 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.950 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.950 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.950 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.950 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.950 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.950 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.950 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.950 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.951 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.951 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.951 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.951 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.951 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.951 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.951 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.951 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.951 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.952 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.952 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.952 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.952 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.952 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.952 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.952 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.952 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.953 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.953 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.953 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.953 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.953 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.953 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.953 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.953 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.953 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.954 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.954 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.954 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.954 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.954 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.954 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.954 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.954 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.955 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.955 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.955 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.955 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.955 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.955 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.955 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.955 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.955 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.956 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.956 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.956 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.956 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.956 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.956 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.956 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.956 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.957 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.957 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.957 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.957 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.957 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.957 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.957 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_host": { 2024-06-06T03:48:53.957 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_host", 2024-06-06T03:48:53.957 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.958 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.958 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.958 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.958 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.958 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.958 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.958 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.958 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.959 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.959 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.959 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.959 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_port": { 2024-06-06T03:48:53.959 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_port", 2024-06-06T03:48:53.959 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.959 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.959 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.959 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10051", 2024-06-06T03:48:53.960 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.960 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.960 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.960 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.960 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.960 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.960 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.960 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.961 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_sender": { 2024-06-06T03:48:53.961 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_sender", 2024-06-06T03:48:53.961 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.961 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.961 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:53.961 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T03:48:53.961 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.961 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.961 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.962 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.962 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.962 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.962 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.962 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.962 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.962 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.962 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-06-06T03:48:53.963 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.963 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.963 INFO:teuthology.orchestra.run.smithi040.stdout: "modules": [ 2024-06-06T03:48:53.963 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm", 2024-06-06T03:48:53.963 INFO:teuthology.orchestra.run.smithi040.stdout: "dashboard", 2024-06-06T03:48:53.963 INFO:teuthology.orchestra.run.smithi040.stdout: "iostat", 2024-06-06T03:48:53.963 INFO:teuthology.orchestra.run.smithi040.stdout: "nfs", 2024-06-06T03:48:53.963 INFO:teuthology.orchestra.run.smithi040.stdout: "restful", 2024-06-06T03:48:53.964 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_schedule" 2024-06-06T03:48:53.964 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.964 INFO:teuthology.orchestra.run.smithi040.stdout: "available_modules": [ 2024-06-06T03:48:53.964 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.964 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "alerts", 2024-06-06T03:48:53.964 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.964 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.964 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.964 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:53.965 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:53.965 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:53.965 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.965 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.965 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:53.965 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.965 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.965 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.966 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T03:48:53.966 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.966 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.966 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.966 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:53.966 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:53.966 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.967 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.967 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.967 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.967 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.967 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.967 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.967 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.967 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.967 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.968 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.968 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.968 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.968 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.968 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.968 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.968 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.968 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.969 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:53.969 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:53.969 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.969 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.969 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.969 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.969 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.969 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.970 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.970 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.970 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.970 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.970 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.970 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:53.970 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:53.971 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.971 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.971 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.971 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:53.971 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.971 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.971 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:53.971 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:53.971 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:53.972 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:53.972 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:53.972 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:53.972 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:53.972 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:53.972 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.972 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.972 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.972 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.973 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:53.973 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:53.973 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.973 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.973 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.973 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:53.973 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.974 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.974 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.974 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:53.974 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.974 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.974 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.974 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.975 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_destination": { 2024-06-06T03:48:53.975 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_destination", 2024-06-06T03:48:53.975 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.975 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.975 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.975 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.975 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.975 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.975 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.976 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Email address to send alerts to", 2024-06-06T03:48:53.976 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.976 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.976 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.976 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.976 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_from_name": { 2024-06-06T03:48:53.976 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_from_name", 2024-06-06T03:48:53.976 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.977 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.977 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.977 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "Ceph", 2024-06-06T03:48:53.977 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.977 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.977 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.977 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Email From: name", 2024-06-06T03:48:53.977 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.977 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.978 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.978 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.978 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_host": { 2024-06-06T03:48:53.978 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_host", 2024-06-06T03:48:53.978 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.978 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.978 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.978 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.979 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.979 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.979 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.979 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP server", 2024-06-06T03:48:53.979 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.979 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.979 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.979 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.979 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_password": { 2024-06-06T03:48:53.980 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_password", 2024-06-06T03:48:53.980 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.980 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.980 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.980 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.980 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.980 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.980 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.981 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Password to authenticate with", 2024-06-06T03:48:53.981 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.981 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.981 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.981 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.981 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_port": { 2024-06-06T03:48:53.981 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_port", 2024-06-06T03:48:53.981 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:53.981 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.982 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.982 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "465", 2024-06-06T03:48:53.982 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.982 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.982 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.982 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP port", 2024-06-06T03:48:53.982 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.982 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.983 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.983 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.983 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_sender": { 2024-06-06T03:48:53.983 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_sender", 2024-06-06T03:48:53.983 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.983 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.983 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.983 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.984 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.984 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.984 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.984 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP envelope sender", 2024-06-06T03:48:53.984 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.984 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.984 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.984 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.984 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_ssl": { 2024-06-06T03:48:53.985 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_ssl", 2024-06-06T03:48:53.985 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.985 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.985 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.985 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.985 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.985 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.985 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.986 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T03:48:53.986 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.986 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.986 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.986 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.986 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_user": { 2024-06-06T03:48:53.986 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_user", 2024-06-06T03:48:53.986 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.987 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.987 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.987 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:53.987 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.987 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.987 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.987 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "User to authenticate as", 2024-06-06T03:48:53.988 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.988 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.988 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.988 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.988 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:53.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.988 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:53.988 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "balancer", 2024-06-06T03:48:53.989 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:53.989 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:53.989 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:53.989 INFO:teuthology.orchestra.run.smithi040.stdout: "active": { 2024-06-06T03:48:53.989 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "active", 2024-06-06T03:48:53.989 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:53.989 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.989 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.989 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:53.990 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.990 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.990 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.990 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T03:48:53.990 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.990 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.990 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.990 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.991 INFO:teuthology.orchestra.run.smithi040.stdout: "begin_time": { 2024-06-06T03:48:53.991 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "begin_time", 2024-06-06T03:48:53.991 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.991 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.991 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.991 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0000", 2024-06-06T03:48:53.991 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.991 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.991 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.992 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T03:48:53.992 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T03:48:53.992 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.992 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.992 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.992 INFO:teuthology.orchestra.run.smithi040.stdout: "begin_weekday": { 2024-06-06T03:48:53.992 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "begin_weekday", 2024-06-06T03:48:53.993 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:53.993 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.993 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.993 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:53.993 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0", 2024-06-06T03:48:53.993 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "6", 2024-06-06T03:48:53.993 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.993 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T03:48:53.993 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T03:48:53.994 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.994 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.994 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.994 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_max_iterations": { 2024-06-06T03:48:53.994 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_max_iterations", 2024-06-06T03:48:53.994 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:53.994 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.994 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.995 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "25", 2024-06-06T03:48:53.995 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2024-06-06T03:48:53.995 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "250", 2024-06-06T03:48:53.995 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.995 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T03:48:53.995 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:53.995 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.995 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.996 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.996 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_metrics": { 2024-06-06T03:48:53.996 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_metrics", 2024-06-06T03:48:53.996 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.996 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.996 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.996 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T03:48:53.996 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:53.996 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:53.997 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.997 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T03:48:53.997 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T03:48:53.997 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.997 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.997 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.997 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_step": { 2024-06-06T03:48:53.997 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_step", 2024-06-06T03:48:53.998 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:53.998 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:53.998 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:53.998 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0.5", 2024-06-06T03:48:53.998 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0.001", 2024-06-06T03:48:53.998 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "0.999", 2024-06-06T03:48:53.998 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:53.998 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "aggressiveness of optimization", 2024-06-06T03:48:53.999 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T03:48:53.999 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:53.999 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:53.999 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:53.999 INFO:teuthology.orchestra.run.smithi040.stdout: "end_time": { 2024-06-06T03:48:53.999 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "end_time", 2024-06-06T03:48:53.999 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:53.999 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.000 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.000 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "2359", 2024-06-06T03:48:54.000 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.000 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.000 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.000 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T03:48:54.000 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T03:48:54.000 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.000 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.001 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.001 INFO:teuthology.orchestra.run.smithi040.stdout: "end_weekday": { 2024-06-06T03:48:54.001 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "end_weekday", 2024-06-06T03:48:54.001 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:54.001 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.001 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.001 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:54.001 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0", 2024-06-06T03:48:54.002 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "6", 2024-06-06T03:48:54.002 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.002 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T03:48:54.002 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T03:48:54.002 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.002 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.002 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.002 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.003 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.003 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.003 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.003 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.003 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.003 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.003 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.003 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.005 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.005 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.005 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.005 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.005 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.005 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.005 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.005 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.005 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.006 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.006 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.006 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.006 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.006 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.006 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.006 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.006 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.007 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.007 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.007 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.007 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.007 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.007 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.007 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.007 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.007 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.008 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.008 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.008 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.008 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.008 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.008 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.008 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.008 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.009 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.009 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.009 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.009 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.009 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.009 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.009 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.009 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.009 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.010 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.010 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.010 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.010 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.010 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.010 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.010 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.010 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.011 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.011 INFO:teuthology.orchestra.run.smithi040.stdout: "min_score": { 2024-06-06T03:48:54.011 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "min_score", 2024-06-06T03:48:54.011 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:54.011 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.011 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.011 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:54.011 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.012 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.012 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.012 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T03:48:54.012 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.012 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.012 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.012 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": { 2024-06-06T03:48:54.013 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mode", 2024-06-06T03:48:54.013 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.013 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.013 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.013 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "upmap", 2024-06-06T03:48:54.013 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.013 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.013 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.013 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-compat", 2024-06-06T03:48:54.014 INFO:teuthology.orchestra.run.smithi040.stdout: "none", 2024-06-06T03:48:54.014 INFO:teuthology.orchestra.run.smithi040.stdout: "read", 2024-06-06T03:48:54.014 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap", 2024-06-06T03:48:54.014 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap-read" 2024-06-06T03:48:54.014 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.014 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Balancer mode", 2024-06-06T03:48:54.014 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.014 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.015 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.015 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_ids": { 2024-06-06T03:48:54.015 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pool_ids", 2024-06-06T03:48:54.015 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.015 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.015 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.015 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.016 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.016 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.016 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.016 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T03:48:54.016 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.016 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.016 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.016 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.016 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:54.017 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:54.017 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.017 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.017 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.017 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:54.017 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.017 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.017 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.018 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T03:48:54.018 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.018 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.018 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.018 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap_max_deviation": { 2024-06-06T03:48:54.018 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "upmap_max_deviation", 2024-06-06T03:48:54.018 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.019 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.019 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.019 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2024-06-06T03:48:54.019 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2024-06-06T03:48:54.019 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.019 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.019 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T03:48:54.019 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T03:48:54.019 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.020 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.020 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap_max_optimizations": { 2024-06-06T03:48:54.020 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "upmap_max_optimizations", 2024-06-06T03:48:54.020 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:54.020 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.020 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.020 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:54.021 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.021 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.021 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.021 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T03:48:54.021 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.021 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.021 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.021 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.021 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.022 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.022 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cephadm", 2024-06-06T03:48:54.022 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.022 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.022 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.022 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_down_multiplier": { 2024-06-06T03:48:54.022 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_down_multiplier", 2024-06-06T03:48:54.023 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:54.023 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.023 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.023 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3.0", 2024-06-06T03:48:54.023 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.023 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.023 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.023 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T03:48:54.023 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.024 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.024 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.024 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_refresh_rate": { 2024-06-06T03:48:54.024 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_refresh_rate", 2024-06-06T03:48:54.024 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.024 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.024 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.025 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "20", 2024-06-06T03:48:54.025 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.025 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.025 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.025 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T03:48:54.025 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.025 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.025 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.025 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.026 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_starting_port": { 2024-06-06T03:48:54.026 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_starting_port", 2024-06-06T03:48:54.026 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.026 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.026 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.026 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "4721", 2024-06-06T03:48:54.026 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.026 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.027 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.027 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T03:48:54.027 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.027 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.027 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.027 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_ptrace": { 2024-06-06T03:48:54.027 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_ptrace", 2024-06-06T03:48:54.027 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.028 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.028 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.028 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.028 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.028 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.028 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.028 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T03:48:54.028 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T03:48:54.029 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.029 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.029 INFO:teuthology.orchestra.run.smithi040.stdout: "autotune_interval": { 2024-06-06T03:48:54.029 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "autotune_interval", 2024-06-06T03:48:54.029 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.029 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.029 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.030 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:54.030 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.030 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.030 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.030 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T03:48:54.030 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.030 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.030 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.031 INFO:teuthology.orchestra.run.smithi040.stdout: "autotune_memory_target_ratio": { 2024-06-06T03:48:54.031 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T03:48:54.031 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:54.031 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.031 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.031 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0.7", 2024-06-06T03:48:54.031 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.031 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.031 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.032 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T03:48:54.032 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.032 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.032 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.032 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.032 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm_log_destination": { 2024-06-06T03:48:54.032 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cephadm_log_destination", 2024-06-06T03:48:54.032 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.033 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.033 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.033 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.033 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.033 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.033 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.033 INFO:teuthology.orchestra.run.smithi040.stdout: "file", 2024-06-06T03:48:54.033 INFO:teuthology.orchestra.run.smithi040.stdout: "file,syslog", 2024-06-06T03:48:54.034 INFO:teuthology.orchestra.run.smithi040.stdout: "syslog" 2024-06-06T03:48:54.034 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.034 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T03:48:54.034 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.034 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.034 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.034 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.034 INFO:teuthology.orchestra.run.smithi040.stdout: "cgroups_split": { 2024-06-06T03:48:54.035 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cgroups_split", 2024-06-06T03:48:54.035 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.035 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.035 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.035 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.035 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.035 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.035 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.036 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T03:48:54.036 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.036 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.036 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.036 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.036 INFO:teuthology.orchestra.run.smithi040.stdout: "config_checks_enabled": { 2024-06-06T03:48:54.036 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_checks_enabled", 2024-06-06T03:48:54.036 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.036 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.037 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.037 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.037 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.037 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.037 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.037 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T03:48:54.037 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.037 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.038 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.038 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.038 INFO:teuthology.orchestra.run.smithi040.stdout: "config_dashboard": { 2024-06-06T03:48:54.038 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_dashboard", 2024-06-06T03:48:54.038 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.038 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.038 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.038 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.039 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.039 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.039 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.039 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T03:48:54.039 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.039 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.039 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.039 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.040 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_alertmanager": { 2024-06-06T03:48:54.040 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_alertmanager", 2024-06-06T03:48:54.040 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.040 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.040 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.040 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T03:48:54.040 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.040 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.041 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.041 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2024-06-06T03:48:54.041 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.041 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.041 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.041 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.041 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_base": { 2024-06-06T03:48:54.041 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_base", 2024-06-06T03:48:54.041 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.042 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.042 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.042 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T03:48:54.042 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.042 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.042 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.042 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Container image name, without the tag", 2024-06-06T03:48:54.042 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.043 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.043 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.043 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.043 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_elasticsearch": { 2024-06-06T03:48:54.043 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_elasticsearch", 2024-06-06T03:48:54.043 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.043 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.043 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.044 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T03:48:54.044 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.044 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.044 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.044 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "elasticsearch container image", 2024-06-06T03:48:54.044 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.044 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.044 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.045 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.045 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_grafana": { 2024-06-06T03:48:54.045 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_grafana", 2024-06-06T03:48:54.045 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.045 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.045 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.045 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T03:48:54.045 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.046 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.046 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.046 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2024-06-06T03:48:54.046 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.046 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.046 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.046 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_haproxy": { 2024-06-06T03:48:54.046 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_haproxy", 2024-06-06T03:48:54.047 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.047 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.047 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.047 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T03:48:54.047 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.047 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.047 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.047 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "HAproxy container image", 2024-06-06T03:48:54.048 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.048 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.048 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.048 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_agent": { 2024-06-06T03:48:54.048 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_agent", 2024-06-06T03:48:54.048 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.048 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.048 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.049 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T03:48:54.049 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.049 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.049 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.049 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger agent container image", 2024-06-06T03:48:54.049 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.049 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.049 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.050 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_collector": { 2024-06-06T03:48:54.050 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_collector", 2024-06-06T03:48:54.050 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.050 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.050 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.050 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T03:48:54.050 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.051 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.051 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.051 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger collector container image", 2024-06-06T03:48:54.051 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.051 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.051 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.051 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_query": { 2024-06-06T03:48:54.051 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_query", 2024-06-06T03:48:54.052 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.052 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.052 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.052 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T03:48:54.052 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.052 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.052 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.052 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger query container image", 2024-06-06T03:48:54.053 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.053 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.053 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.053 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.053 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_keepalived": { 2024-06-06T03:48:54.053 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_keepalived", 2024-06-06T03:48:54.053 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.053 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.053 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.054 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T03:48:54.054 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.054 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.054 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.054 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Keepalived container image", 2024-06-06T03:48:54.054 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.054 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.054 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.055 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.055 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_loki": { 2024-06-06T03:48:54.055 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_loki", 2024-06-06T03:48:54.055 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.055 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.055 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.055 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T03:48:54.055 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.056 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.056 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.056 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Loki container image", 2024-06-06T03:48:54.056 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.056 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.056 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.056 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.056 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_node_exporter": { 2024-06-06T03:48:54.057 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_node_exporter", 2024-06-06T03:48:54.057 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.057 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.057 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.057 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T03:48:54.057 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.057 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.057 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.057 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2024-06-06T03:48:54.058 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.058 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.058 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.058 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.058 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_nvmeof": { 2024-06-06T03:48:54.058 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_nvmeof", 2024-06-06T03:48:54.058 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.059 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.059 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.059 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T03:48:54.059 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.059 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.059 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.059 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Nvme-of container image", 2024-06-06T03:48:54.059 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.059 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.060 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.060 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.060 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_prometheus": { 2024-06-06T03:48:54.060 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_prometheus", 2024-06-06T03:48:54.060 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.060 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.060 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.060 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T03:48:54.061 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.061 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.061 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.061 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2024-06-06T03:48:54.061 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.061 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.061 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.061 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.061 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_promtail": { 2024-06-06T03:48:54.062 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_promtail", 2024-06-06T03:48:54.062 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.062 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.062 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.062 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T03:48:54.062 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.062 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.063 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.063 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Promtail container image", 2024-06-06T03:48:54.063 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.063 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.063 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.063 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_samba": { 2024-06-06T03:48:54.063 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_samba", 2024-06-06T03:48:54.063 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.064 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.064 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.064 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T03:48:54.064 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.064 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.064 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.064 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Samba/SMB container image", 2024-06-06T03:48:54.064 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.064 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.065 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.065 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.065 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_snmp_gateway": { 2024-06-06T03:48:54.065 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_snmp_gateway", 2024-06-06T03:48:54.065 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.065 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.065 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.065 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T03:48:54.065 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.065 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.066 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.066 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SNMP Gateway container image", 2024-06-06T03:48:54.066 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.066 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.066 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.066 INFO:teuthology.orchestra.run.smithi040.stdout: "container_init": { 2024-06-06T03:48:54.066 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_init", 2024-06-06T03:48:54.067 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.067 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.067 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.067 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.067 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.067 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.067 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.067 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T03:48:54.068 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.068 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.068 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.068 INFO:teuthology.orchestra.run.smithi040.stdout: "daemon_cache_timeout": { 2024-06-06T03:48:54.068 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "daemon_cache_timeout", 2024-06-06T03:48:54.068 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.068 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.068 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.069 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:54.069 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.069 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.069 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.069 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T03:48:54.069 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.069 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.069 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.070 INFO:teuthology.orchestra.run.smithi040.stdout: "default_cephadm_command_timeout": { 2024-06-06T03:48:54.070 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T03:48:54.070 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.070 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.070 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.070 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "900", 2024-06-06T03:48:54.070 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.071 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.071 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.071 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T03:48:54.071 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.071 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.071 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.071 INFO:teuthology.orchestra.run.smithi040.stdout: "default_registry": { 2024-06-06T03:48:54.072 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "default_registry", 2024-06-06T03:48:54.072 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.072 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.072 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.072 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io", 2024-06-06T03:48:54.072 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.072 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.072 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.072 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T03:48:54.073 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.073 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.073 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.073 INFO:teuthology.orchestra.run.smithi040.stdout: "device_cache_timeout": { 2024-06-06T03:48:54.073 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_cache_timeout", 2024-06-06T03:48:54.073 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.073 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.074 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.074 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1800", 2024-06-06T03:48:54.074 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.074 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.074 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.074 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache device inventory", 2024-06-06T03:48:54.074 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.074 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.075 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.075 INFO:teuthology.orchestra.run.smithi040.stdout: "device_enhanced_scan": { 2024-06-06T03:48:54.075 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_enhanced_scan", 2024-06-06T03:48:54.075 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.075 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.075 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.075 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.076 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.076 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.076 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.076 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T03:48:54.076 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.076 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.076 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.076 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.076 INFO:teuthology.orchestra.run.smithi040.stdout: "facts_cache_timeout": { 2024-06-06T03:48:54.077 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "facts_cache_timeout", 2024-06-06T03:48:54.077 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.077 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.077 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.077 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:54.077 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.077 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.077 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.077 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache host facts data", 2024-06-06T03:48:54.078 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.078 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.078 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.078 INFO:teuthology.orchestra.run.smithi040.stdout: "grafana_dashboards_path": { 2024-06-06T03:48:54.078 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "grafana_dashboards_path", 2024-06-06T03:48:54.078 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.078 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.079 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.079 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T03:48:54.079 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.079 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.079 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.079 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T03:48:54.079 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.079 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.080 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.080 INFO:teuthology.orchestra.run.smithi040.stdout: "host_check_interval": { 2024-06-06T03:48:54.080 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "host_check_interval", 2024-06-06T03:48:54.080 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.080 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.080 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.080 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:54.081 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.081 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.081 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.081 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to perform a host check", 2024-06-06T03:48:54.081 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.081 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.081 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.081 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.081 INFO:teuthology.orchestra.run.smithi040.stdout: "hw_monitoring": { 2024-06-06T03:48:54.082 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "hw_monitoring", 2024-06-06T03:48:54.082 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.082 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.082 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.082 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.082 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.082 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.082 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.083 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T03:48:54.083 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.083 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.083 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.083 INFO:teuthology.orchestra.run.smithi040.stdout: "inventory_list_all": { 2024-06-06T03:48:54.083 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "inventory_list_all", 2024-06-06T03:48:54.083 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.084 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.084 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.084 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.084 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.084 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.084 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.084 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T03:48:54.084 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.084 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.085 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.085 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.085 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.085 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.085 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.085 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.085 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.085 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.086 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.086 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.086 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.086 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.086 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.086 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.086 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.086 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.086 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.087 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.087 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.087 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.087 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.087 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.087 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.087 INFO:teuthology.orchestra.run.smithi040.stdout: "log_refresh_metadata": { 2024-06-06T03:48:54.087 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_refresh_metadata", 2024-06-06T03:48:54.088 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.088 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.088 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.088 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.088 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.088 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.088 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.088 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T03:48:54.089 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.089 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.089 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.089 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.089 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.089 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.089 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.089 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.090 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.090 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.090 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.090 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.090 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T03:48:54.090 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.090 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.090 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.091 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.091 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.091 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.091 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.091 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.091 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.091 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.091 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.091 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.092 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.092 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.092 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.092 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.092 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.092 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.092 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.093 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.093 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.093 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.093 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.093 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.093 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.093 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.093 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.094 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.094 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.094 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.094 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.094 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.094 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.094 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.094 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.095 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.095 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.095 INFO:teuthology.orchestra.run.smithi040.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T03:48:54.095 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T03:48:54.095 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.095 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.097 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.097 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.097 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.097 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.098 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.098 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T03:48:54.098 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.098 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.098 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.098 INFO:teuthology.orchestra.run.smithi040.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T03:48:54.098 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T03:48:54.099 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.099 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.099 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.099 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "*", 2024-06-06T03:48:54.099 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.099 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.099 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.099 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T03:48:54.100 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.100 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.100 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.100 INFO:teuthology.orchestra.run.smithi040.stdout: "max_count_per_host": { 2024-06-06T03:48:54.100 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_count_per_host", 2024-06-06T03:48:54.100 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.100 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.101 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.101 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:54.101 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.101 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.101 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.101 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "max number of daemons per service per host", 2024-06-06T03:48:54.101 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.101 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.102 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.102 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd_draining_count": { 2024-06-06T03:48:54.102 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_osd_draining_count", 2024-06-06T03:48:54.102 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.102 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.102 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.102 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:54.103 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.103 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.103 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.103 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T03:48:54.103 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.103 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.103 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.104 INFO:teuthology.orchestra.run.smithi040.stdout: "migration_current": { 2024-06-06T03:48:54.104 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "migration_current", 2024-06-06T03:48:54.104 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.104 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.104 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.104 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.104 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.104 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.104 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.105 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "internal - do not modify", 2024-06-06T03:48:54.105 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.105 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.105 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.105 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": { 2024-06-06T03:48:54.105 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mode", 2024-06-06T03:48:54.105 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.105 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.106 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.106 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "root", 2024-06-06T03:48:54.106 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.106 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.106 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.106 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm-package", 2024-06-06T03:48:54.106 INFO:teuthology.orchestra.run.smithi040.stdout: "root" 2024-06-06T03:48:54.106 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.107 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T03:48:54.107 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.107 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.107 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.107 INFO:teuthology.orchestra.run.smithi040.stdout: "oob_default_addr": { 2024-06-06T03:48:54.107 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "oob_default_addr", 2024-06-06T03:48:54.107 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.108 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.108 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.108 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "169.254.1.1", 2024-06-06T03:48:54.108 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.108 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.108 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.108 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T03:48:54.108 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.108 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.109 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.109 INFO:teuthology.orchestra.run.smithi040.stdout: "prometheus_alerts_path": { 2024-06-06T03:48:54.109 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus_alerts_path", 2024-06-06T03:48:54.109 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.109 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.109 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.109 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T03:48:54.109 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.110 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.110 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.110 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T03:48:54.110 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.110 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.110 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.111 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_insecure": { 2024-06-06T03:48:54.111 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_insecure", 2024-06-06T03:48:54.111 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.111 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.111 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.111 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.111 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.111 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.112 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.112 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T03:48:54.112 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.112 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.112 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.112 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_password": { 2024-06-06T03:48:54.112 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_password", 2024-06-06T03:48:54.113 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.113 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.113 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.113 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.113 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.113 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.113 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.113 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T03:48:54.113 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.114 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.114 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.114 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_url": { 2024-06-06T03:48:54.114 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_url", 2024-06-06T03:48:54.114 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.114 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.115 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.115 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.115 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.115 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.115 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.115 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T03:48:54.115 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.115 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.115 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.116 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_username": { 2024-06-06T03:48:54.116 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_username", 2024-06-06T03:48:54.116 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.116 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.116 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.116 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.116 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.116 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.117 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.117 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T03:48:54.117 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.117 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.117 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.117 INFO:teuthology.orchestra.run.smithi040.stdout: "secure_monitoring_stack": { 2024-06-06T03:48:54.117 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "secure_monitoring_stack", 2024-06-06T03:48:54.118 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.118 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.118 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.118 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.118 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.118 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.118 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.118 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T03:48:54.119 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.119 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.119 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.119 INFO:teuthology.orchestra.run.smithi040.stdout: "service_discovery_port": { 2024-06-06T03:48:54.119 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "service_discovery_port", 2024-06-06T03:48:54.119 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.119 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.120 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.120 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8765", 2024-06-06T03:48:54.120 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.120 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.120 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.120 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "cephadm service discovery port", 2024-06-06T03:48:54.120 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.120 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.121 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.121 INFO:teuthology.orchestra.run.smithi040.stdout: "ssh_config_file": { 2024-06-06T03:48:54.121 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssh_config_file", 2024-06-06T03:48:54.121 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.121 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.121 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.121 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.121 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.122 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.122 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.122 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T03:48:54.122 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.122 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.122 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.122 INFO:teuthology.orchestra.run.smithi040.stdout: "use_agent": { 2024-06-06T03:48:54.123 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "use_agent", 2024-06-06T03:48:54.123 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.123 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.123 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.123 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.123 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.123 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.123 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.123 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T03:48:54.124 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.124 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.124 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.124 INFO:teuthology.orchestra.run.smithi040.stdout: "use_repo_digest": { 2024-06-06T03:48:54.124 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "use_repo_digest", 2024-06-06T03:48:54.124 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.124 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.124 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.125 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.125 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.125 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.125 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.125 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T03:48:54.125 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.125 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.125 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.126 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_failed_host_check": { 2024-06-06T03:48:54.126 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_failed_host_check", 2024-06-06T03:48:54.126 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.126 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.126 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.126 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.126 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.127 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.127 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.127 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T03:48:54.127 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.127 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.127 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.127 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_stray_daemons": { 2024-06-06T03:48:54.127 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_stray_daemons", 2024-06-06T03:48:54.128 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.128 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.128 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.128 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.128 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.128 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.128 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.128 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T03:48:54.129 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.129 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.129 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.129 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_stray_hosts": { 2024-06-06T03:48:54.129 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_stray_hosts", 2024-06-06T03:48:54.129 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.129 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.129 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.130 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.130 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.130 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.130 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.130 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T03:48:54.130 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.130 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.131 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.131 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.131 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.131 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crash", 2024-06-06T03:48:54.131 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.131 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.132 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.132 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.132 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.132 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.132 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.132 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.132 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.132 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.132 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.133 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.133 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.133 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.133 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.133 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.133 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.133 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.133 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.134 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.134 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.134 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.134 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.134 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.134 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.134 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.135 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.135 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.135 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.135 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.135 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.135 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.135 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.135 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.136 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.136 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.136 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.136 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.136 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.136 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.137 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.137 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.137 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.137 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.137 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.137 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.137 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.137 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.138 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.138 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.138 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.138 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.138 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.138 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.138 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.139 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.139 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.139 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.139 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.139 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.139 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.139 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.140 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.140 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.140 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.140 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.140 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.140 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.140 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: "retain_interval": { 2024-06-06T03:48:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "retain_interval", 2024-06-06T03:48:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "31536000", 2024-06-06T03:48:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T03:48:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_recent_interval": { 2024-06-06T03:48:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_recent_interval", 2024-06-06T03:48:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1209600", 2024-06-06T03:48:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.144 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T03:48:54.144 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.144 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.144 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.145 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "dashboard", 2024-06-06T03:48:54.145 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.145 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.145 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.145 INFO:teuthology.orchestra.run.smithi040.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T03:48:54.145 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T03:48:54.145 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.145 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.146 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.146 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:54.146 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.146 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.146 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.146 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.146 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.146 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.147 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.147 INFO:teuthology.orchestra.run.smithi040.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T03:48:54.147 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T03:48:54.147 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.147 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.147 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.147 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.148 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.148 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.148 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.148 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.148 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.148 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.148 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.148 INFO:teuthology.orchestra.run.smithi040.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T03:48:54.149 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T03:48:54.149 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.149 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.149 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.149 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.149 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.149 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.149 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.150 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.150 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.150 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.150 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.150 INFO:teuthology.orchestra.run.smithi040.stdout: "AUDIT_API_ENABLED": { 2024-06-06T03:48:54.150 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T03:48:54.150 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.151 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.151 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.151 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.151 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.151 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.151 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.151 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.151 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.152 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.152 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.152 INFO:teuthology.orchestra.run.smithi040.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T03:48:54.152 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T03:48:54.152 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.152 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.152 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.153 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.153 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.153 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.153 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.153 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.153 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.153 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.153 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.154 INFO:teuthology.orchestra.run.smithi040.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T03:48:54.154 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T03:48:54.154 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.154 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.154 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.154 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.154 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.155 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.155 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.155 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.155 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.155 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.155 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.155 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T03:48:54.155 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T03:48:54.156 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.156 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.156 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.156 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.156 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.156 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.156 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.156 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.157 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.157 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.157 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.157 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T03:48:54.157 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T03:48:54.157 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.157 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.157 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.158 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.158 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.158 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.158 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.158 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.158 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.158 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.158 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.159 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T03:48:54.159 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T03:48:54.159 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.159 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.159 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.159 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.159 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.159 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.160 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.160 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.160 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.160 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.160 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.160 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T03:48:54.160 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T03:48:54.161 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.161 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.161 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.161 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.161 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.161 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.161 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.161 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.162 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.162 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.162 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.162 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T03:48:54.162 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T03:48:54.162 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.162 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.163 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.163 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.163 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.163 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.163 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.163 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.163 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.163 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.163 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.164 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T03:48:54.164 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T03:48:54.164 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.164 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.164 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.164 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.164 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.165 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.165 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.165 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.165 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.165 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.165 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.165 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T03:48:54.165 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T03:48:54.166 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.166 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.166 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.166 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.166 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.166 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.166 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.166 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.167 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.167 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.167 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.167 INFO:teuthology.orchestra.run.smithi040.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T03:48:54.167 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T03:48:54.167 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.167 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.167 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.168 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.168 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.168 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.168 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.168 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.168 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.168 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.168 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.169 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T03:48:54.169 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T03:48:54.169 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.169 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.169 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.169 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2024-06-06T03:48:54.169 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.169 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.170 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.170 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.170 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.170 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.170 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.170 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T03:48:54.170 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T03:48:54.171 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.171 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.171 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.171 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.171 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.171 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.171 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.172 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.172 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.172 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.172 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.172 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_URL": { 2024-06-06T03:48:54.172 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_URL", 2024-06-06T03:48:54.172 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.173 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.173 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.173 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.173 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.173 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.173 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.173 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.173 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.174 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.174 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.174 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T03:48:54.174 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T03:48:54.174 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.174 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.175 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.175 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2024-06-06T03:48:54.175 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.175 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.175 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.175 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.175 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.175 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.176 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.176 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T03:48:54.176 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T03:48:54.176 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.176 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.176 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.176 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.177 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.177 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.177 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.177 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.177 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.177 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.177 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.178 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T03:48:54.178 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T03:48:54.178 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.178 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.178 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.178 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.178 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.178 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.179 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.179 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.179 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.179 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.179 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.179 INFO:teuthology.orchestra.run.smithi040.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T03:48:54.179 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T03:48:54.179 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.180 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.180 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.180 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.180 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.180 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.180 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.180 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.180 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.181 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.181 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.181 INFO:teuthology.orchestra.run.smithi040.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T03:48:54.181 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T03:48:54.181 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.181 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.181 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.181 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.182 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.182 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.182 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.182 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.182 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.182 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.182 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.183 INFO:teuthology.orchestra.run.smithi040.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T03:48:54.183 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T03:48:54.183 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.183 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.183 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.183 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.183 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.183 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.184 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.184 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.184 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.184 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.184 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.184 INFO:teuthology.orchestra.run.smithi040.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T03:48:54.184 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T03:48:54.185 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.185 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.185 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.185 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.185 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.185 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.185 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.185 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.185 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.186 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.186 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.186 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T03:48:54.186 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T03:48:54.186 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.186 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.186 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.187 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.187 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.187 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.187 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.187 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.187 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.187 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.187 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.188 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T03:48:54.188 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T03:48:54.188 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.188 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.188 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.188 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.188 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.189 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.189 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.189 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.189 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.189 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.189 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.189 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T03:48:54.189 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T03:48:54.189 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.190 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.190 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.190 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.190 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.190 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.190 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.190 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.190 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.190 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.191 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.191 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T03:48:54.191 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T03:48:54.191 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.191 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.191 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.191 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.192 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.192 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.192 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.192 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.192 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.192 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.192 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.193 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T03:48:54.193 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T03:48:54.193 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.193 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.193 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.193 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.193 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.193 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.194 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.194 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.194 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.194 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.194 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.194 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T03:48:54.194 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T03:48:54.195 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.195 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.195 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.195 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.195 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.195 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.195 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.195 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.196 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.196 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.196 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.196 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.196 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T03:48:54.196 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T03:48:54.196 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.196 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.197 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.197 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.197 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.197 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.197 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.197 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.197 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.197 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.197 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.198 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T03:48:54.198 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T03:48:54.198 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.198 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.198 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.198 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.198 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.199 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.199 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.199 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.199 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.199 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.199 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.199 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.199 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T03:48:54.199 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T03:48:54.200 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.200 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.200 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.200 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T03:48:54.200 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.200 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.200 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.200 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.201 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.201 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.201 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.201 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.201 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T03:48:54.201 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T03:48:54.201 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.201 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.202 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.202 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:54.202 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.202 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.202 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.202 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.202 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.202 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.203 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.203 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.203 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T03:48:54.203 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T03:48:54.203 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.203 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.203 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.203 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8", 2024-06-06T03:48:54.204 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.204 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.204 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.204 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.204 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.204 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.204 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.204 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.205 INFO:teuthology.orchestra.run.smithi040.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T03:48:54.205 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T03:48:54.205 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.205 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.205 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.205 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "45", 2024-06-06T03:48:54.205 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.205 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.205 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.206 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.206 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.206 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.206 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.206 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.206 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T03:48:54.206 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T03:48:54.206 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.207 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.207 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.207 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.207 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.207 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.207 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.207 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.207 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.207 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.208 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.208 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.208 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T03:48:54.208 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T03:48:54.208 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.208 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.208 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.208 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2024-06-06T03:48:54.209 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.209 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.209 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.209 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.209 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.209 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.209 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.209 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.210 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T03:48:54.210 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T03:48:54.210 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.210 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.210 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.210 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.210 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.210 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.211 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.211 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.211 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.211 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.211 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.211 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.211 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T03:48:54.211 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T03:48:54.211 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.212 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.212 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.212 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.212 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.212 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.212 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.212 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.212 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.213 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.213 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.213 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.213 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T03:48:54.213 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T03:48:54.213 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.213 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.213 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.214 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:54.214 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.214 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.214 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.214 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.214 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.214 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.214 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.215 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.215 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T03:48:54.215 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T03:48:54.215 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.215 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.215 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.215 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:54.215 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.215 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.216 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.216 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.216 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.216 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.216 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.216 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.216 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T03:48:54.216 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T03:48:54.217 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.217 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.217 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.217 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2024-06-06T03:48:54.217 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.217 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.217 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.217 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.217 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.218 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.218 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.218 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.218 INFO:teuthology.orchestra.run.smithi040.stdout: "cross_origin_url": { 2024-06-06T03:48:54.218 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cross_origin_url", 2024-06-06T03:48:54.218 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.218 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.218 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.219 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.219 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.219 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.219 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.219 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.219 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.219 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.219 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.220 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.220 INFO:teuthology.orchestra.run.smithi040.stdout: "crt_file": { 2024-06-06T03:48:54.220 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crt_file", 2024-06-06T03:48:54.220 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.220 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.220 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.220 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.220 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.220 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.221 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.221 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.221 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.221 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.221 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.221 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.221 INFO:teuthology.orchestra.run.smithi040.stdout: "debug": { 2024-06-06T03:48:54.221 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "debug", 2024-06-06T03:48:54.221 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.222 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.222 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.222 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.222 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.222 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.222 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.222 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable/disable debug options", 2024-06-06T03:48:54.222 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.223 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.223 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.223 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.223 INFO:teuthology.orchestra.run.smithi040.stdout: "jwt_token_ttl": { 2024-06-06T03:48:54.223 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "jwt_token_ttl", 2024-06-06T03:48:54.223 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.223 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.223 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.224 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "28800", 2024-06-06T03:48:54.224 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.224 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.224 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.224 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.224 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.224 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.224 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.224 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.225 INFO:teuthology.orchestra.run.smithi040.stdout: "key_file": { 2024-06-06T03:48:54.225 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "key_file", 2024-06-06T03:48:54.225 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.225 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.225 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.225 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.225 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.225 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.226 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.226 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.226 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.226 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.226 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.226 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.226 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.226 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.226 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.228 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.228 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.228 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.228 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.228 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.228 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.228 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.228 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.229 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.229 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.229 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.229 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.229 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.229 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.229 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.229 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.232 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.232 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.232 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.232 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.232 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.232 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.232 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.232 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.234 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.234 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.234 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.234 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.234 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.234 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.234 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.234 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.235 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.235 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.235 INFO:teuthology.orchestra.run.smithi040.stdout: "motd": { 2024-06-06T03:48:54.235 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "motd", 2024-06-06T03:48:54.235 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.235 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.235 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.235 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.236 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.236 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.236 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.236 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "The message of the day", 2024-06-06T03:48:54.236 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.236 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.236 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.236 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.236 INFO:teuthology.orchestra.run.smithi040.stdout: "redirect_resolve_ip_addr": { 2024-06-06T03:48:54.237 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T03:48:54.237 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.237 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.237 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.237 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.237 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.237 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.237 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.238 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.238 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.238 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.238 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.238 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.238 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2024-06-06T03:48:54.238 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2024-06-06T03:48:54.238 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.239 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.239 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.239 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "::", 2024-06-06T03:48:54.239 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.239 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.239 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.239 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.239 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.240 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.240 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.240 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.240 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2024-06-06T03:48:54.240 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2024-06-06T03:48:54.240 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.240 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.240 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.241 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8080", 2024-06-06T03:48:54.241 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.241 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.241 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.241 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.241 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.241 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.241 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.242 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl": { 2024-06-06T03:48:54.242 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl", 2024-06-06T03:48:54.242 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.242 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.242 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.242 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.242 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.242 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.243 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.243 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.243 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.243 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.243 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.243 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.243 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl_server_port": { 2024-06-06T03:48:54.243 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl_server_port", 2024-06-06T03:48:54.244 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.244 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.244 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.244 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8443", 2024-06-06T03:48:54.244 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.244 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.244 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.244 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.244 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.245 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.245 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.245 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_behaviour": { 2024-06-06T03:48:54.245 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_behaviour", 2024-06-06T03:48:54.245 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.245 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.245 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.245 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "redirect", 2024-06-06T03:48:54.246 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.246 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.246 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.246 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.246 INFO:teuthology.orchestra.run.smithi040.stdout: "redirect" 2024-06-06T03:48:54.246 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.246 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.246 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.247 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.247 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.247 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.247 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_error_status_code": { 2024-06-06T03:48:54.247 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_error_status_code", 2024-06-06T03:48:54.247 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.247 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.247 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.247 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "500", 2024-06-06T03:48:54.248 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "400", 2024-06-06T03:48:54.248 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "599", 2024-06-06T03:48:54.248 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.248 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.248 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.248 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.248 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.248 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.249 INFO:teuthology.orchestra.run.smithi040.stdout: "url_prefix": { 2024-06-06T03:48:54.249 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "url_prefix", 2024-06-06T03:48:54.249 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.249 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.249 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.249 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.249 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.249 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.249 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.250 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.250 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.250 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.250 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.250 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.250 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.250 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.250 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.251 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "devicehealth", 2024-06-06T03:48:54.251 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.251 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.251 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.251 INFO:teuthology.orchestra.run.smithi040.stdout: "enable_monitoring": { 2024-06-06T03:48:54.251 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enable_monitoring", 2024-06-06T03:48:54.251 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.251 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.251 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.252 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.252 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.252 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.252 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.252 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "monitor device health metrics", 2024-06-06T03:48:54.252 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.252 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.252 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.253 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.253 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.253 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.253 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.253 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.253 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.253 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.253 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.253 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.254 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.254 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.254 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.254 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.254 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.254 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.254 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.254 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.255 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.255 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.255 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.255 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.255 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.255 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.255 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.256 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.256 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.256 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.256 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.256 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.256 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.256 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.256 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.256 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.256 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.257 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.257 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.257 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.257 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.257 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.257 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.257 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.257 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.258 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.258 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.258 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.258 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.258 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.258 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.258 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.258 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.259 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.259 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.259 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.259 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.259 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.259 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.259 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.259 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.259 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.260 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.260 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.260 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.260 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.260 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.260 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.260 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.260 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.260 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.261 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.261 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.261 INFO:teuthology.orchestra.run.smithi040.stdout: "mark_out_threshold": { 2024-06-06T03:48:54.261 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mark_out_threshold", 2024-06-06T03:48:54.261 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.261 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.261 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.261 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "2419200", 2024-06-06T03:48:54.261 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.262 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.262 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.262 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T03:48:54.262 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.262 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.262 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.262 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.262 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": { 2024-06-06T03:48:54.263 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pool_name", 2024-06-06T03:48:54.263 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.263 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.263 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.263 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "device_health_metrics", 2024-06-06T03:48:54.263 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.263 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.263 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.263 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T03:48:54.264 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.264 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.264 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.264 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.264 INFO:teuthology.orchestra.run.smithi040.stdout: "retention_period": { 2024-06-06T03:48:54.264 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "retention_period", 2024-06-06T03:48:54.264 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.264 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.265 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.265 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15552000", 2024-06-06T03:48:54.265 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.265 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.265 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.265 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long to retain device health metrics", 2024-06-06T03:48:54.265 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.265 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.265 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.266 INFO:teuthology.orchestra.run.smithi040.stdout: "scrape_frequency": { 2024-06-06T03:48:54.266 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "scrape_frequency", 2024-06-06T03:48:54.266 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.266 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.266 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.266 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "86400", 2024-06-06T03:48:54.266 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.266 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T03:48:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: "self_heal": { 2024-06-06T03:48:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "self_heal", 2024-06-06T03:48:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.268 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.268 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.268 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.268 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.268 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.268 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.268 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T03:48:54.268 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.268 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.269 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.269 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:54.269 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:54.269 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.269 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.269 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.269 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:54.270 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.270 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.270 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.270 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T03:48:54.270 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.270 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.270 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.271 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_threshold": { 2024-06-06T03:48:54.271 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_threshold", 2024-06-06T03:48:54.271 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.271 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.271 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.271 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "7257600", 2024-06-06T03:48:54.271 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.271 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.271 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.272 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T03:48:54.272 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.272 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.272 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.272 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.272 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.272 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.273 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "diskprediction_local", 2024-06-06T03:48:54.273 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.273 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.273 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.273 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.273 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.273 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.273 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.274 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.274 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.274 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.274 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.274 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.274 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.274 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.274 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.274 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.277 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.277 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.277 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.277 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.277 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.277 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.277 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.278 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.278 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.278 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.278 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.278 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.278 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.278 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.278 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.278 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.279 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.279 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.279 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.279 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.279 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.279 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.279 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.279 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.280 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.280 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.280 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.280 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.280 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.280 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.280 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.281 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.281 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.281 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.281 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.281 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.281 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.281 INFO:teuthology.orchestra.run.smithi040.stdout: "predict_interval": { 2024-06-06T03:48:54.282 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "predict_interval", 2024-06-06T03:48:54.282 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.282 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.282 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.282 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "86400", 2024-06-06T03:48:54.282 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.282 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.283 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.283 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.283 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.283 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.283 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.283 INFO:teuthology.orchestra.run.smithi040.stdout: "predictor_model": { 2024-06-06T03:48:54.283 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "predictor_model", 2024-06-06T03:48:54.284 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.284 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.284 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.284 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "prophetstor", 2024-06-06T03:48:54.284 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.284 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.284 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.284 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.285 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.285 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.285 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.285 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.285 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:54.285 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:54.285 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.285 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.286 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.286 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2024-06-06T03:48:54.286 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.286 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.286 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.286 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.286 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.286 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.287 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.287 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.287 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.287 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.287 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "influx", 2024-06-06T03:48:54.287 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": false, 2024-06-06T03:48:54.287 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "influxdb python module not found", 2024-06-06T03:48:54.287 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "batch_size": { 2024-06-06T03:48:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "batch_size", 2024-06-06T03:48:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5000", 2024-06-06T03:48:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T03:48:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "database": { 2024-06-06T03:48:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "database", 2024-06-06T03:48:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "ceph", 2024-06-06T03:48:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T03:48:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": { 2024-06-06T03:48:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "hostname", 2024-06-06T03:48:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB server hostname", 2024-06-06T03:48:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "30", 2024-06-06T03:48:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "5", 2024-06-06T03:48:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T03:48:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.302 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.302 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.302 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.302 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.302 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.302 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.302 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.303 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.303 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.303 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.303 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.303 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.303 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.303 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.304 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.304 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.304 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.304 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.304 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.304 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.304 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.304 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.305 INFO:teuthology.orchestra.run.smithi040.stdout: "password": { 2024-06-06T03:48:54.305 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "password", 2024-06-06T03:48:54.305 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.305 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.305 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.305 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.305 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "password of InfluxDB server user", 2024-06-06T03:48:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "port": { 2024-06-06T03:48:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "port", 2024-06-06T03:48:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8086", 2024-06-06T03:48:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB server port", 2024-06-06T03:48:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl": { 2024-06-06T03:48:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl", 2024-06-06T03:48:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "false", 2024-06-06T03:48:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T03:48:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "threads": { 2024-06-06T03:48:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "threads", 2024-06-06T03:48:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2024-06-06T03:48:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2024-06-06T03:48:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "32", 2024-06-06T03:48:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T03:48:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "username": { 2024-06-06T03:48:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "username", 2024-06-06T03:48:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "username of InfluxDB server user", 2024-06-06T03:48:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "verify_ssl": { 2024-06-06T03:48:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "verify_ssl", 2024-06-06T03:48:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "true", 2024-06-06T03:48:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T03:48:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "insights", 2024-06-06T03:48:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.321 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.321 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.321 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.321 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.321 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.321 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.321 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.321 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.322 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.322 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.322 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.322 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.322 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.322 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.322 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.322 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.323 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.323 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.323 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.323 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.323 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.323 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.323 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.325 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.325 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.325 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "iostat", 2024-06-06T03:48:54.325 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.325 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.325 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.326 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.326 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.326 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.326 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.326 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.326 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.326 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.326 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.328 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.328 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.328 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.328 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.328 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.328 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.328 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.329 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.329 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.329 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.329 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.329 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.329 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.329 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.329 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.330 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.330 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.330 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.330 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.330 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.330 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.330 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.331 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.331 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.331 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.331 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.331 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.331 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.331 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.331 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.332 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.332 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.332 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.332 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.332 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.332 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.332 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.333 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.333 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.333 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.333 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.333 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.333 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.333 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.333 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.334 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.334 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.334 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.334 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.334 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.334 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.335 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "k8sevents", 2024-06-06T03:48:54.335 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.335 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.335 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.335 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_event_retention_days": { 2024-06-06T03:48:54.335 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ceph_event_retention_days", 2024-06-06T03:48:54.335 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.336 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.336 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.336 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "7", 2024-06-06T03:48:54.336 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.336 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.336 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.336 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T03:48:54.336 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.337 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.337 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.337 INFO:teuthology.orchestra.run.smithi040.stdout: "config_check_secs": { 2024-06-06T03:48:54.337 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_check_secs", 2024-06-06T03:48:54.337 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.337 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.337 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.338 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:54.338 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "10", 2024-06-06T03:48:54.338 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.338 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.338 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T03:48:54.338 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.338 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.339 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.339 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.339 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.339 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.339 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.339 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.339 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.339 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.340 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.340 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.340 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.340 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.340 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.340 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.340 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.340 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.341 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.341 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.341 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.341 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.341 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.341 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.342 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.342 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.342 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.342 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.342 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.342 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.342 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.343 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.343 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.343 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.343 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.343 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.343 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.344 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.344 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.344 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.344 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.344 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.344 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.344 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.344 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.344 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.345 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.345 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.345 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.345 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.345 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.345 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.345 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.345 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.346 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.346 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.346 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.346 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.346 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.346 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.347 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.347 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.347 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.347 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.347 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.347 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.347 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.347 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.347 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.348 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.348 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.348 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "localpool", 2024-06-06T03:48:54.348 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.348 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.348 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.349 INFO:teuthology.orchestra.run.smithi040.stdout: "failure_domain": { 2024-06-06T03:48:54.349 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "failure_domain", 2024-06-06T03:48:54.349 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.349 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.349 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.349 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "host", 2024-06-06T03:48:54.349 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.349 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.350 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.350 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "failure domain for any created local pool", 2024-06-06T03:48:54.350 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T03:48:54.350 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.350 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.350 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.350 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.351 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.351 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.351 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.351 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.351 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.351 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.351 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.351 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.352 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.352 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.352 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.352 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.352 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.352 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.352 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.352 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.353 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.353 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.353 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.353 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.353 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.353 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.353 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.354 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.354 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.354 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.354 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.354 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.354 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.354 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.355 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.355 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.355 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.355 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.355 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.355 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.355 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.355 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.356 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.356 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.356 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.356 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.356 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.356 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.356 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.356 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.357 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.357 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.357 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.357 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.357 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.357 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.357 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.358 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.358 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.358 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.358 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.358 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.358 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.358 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.358 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.359 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.359 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": { 2024-06-06T03:48:54.359 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "min_size", 2024-06-06T03:48:54.359 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.359 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.359 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.359 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.359 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.360 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.360 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.360 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default min_size for any created local pool", 2024-06-06T03:48:54.360 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T03:48:54.360 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.360 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.361 INFO:teuthology.orchestra.run.smithi040.stdout: "num_rep": { 2024-06-06T03:48:54.361 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "num_rep", 2024-06-06T03:48:54.361 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.361 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.361 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.361 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3", 2024-06-06T03:48:54.361 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.361 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.362 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.362 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default replica count for any created local pool", 2024-06-06T03:48:54.362 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.362 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.362 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.362 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": { 2024-06-06T03:48:54.362 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pg_num", 2024-06-06T03:48:54.363 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.363 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.363 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.363 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "128", 2024-06-06T03:48:54.363 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.363 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.363 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.363 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T03:48:54.363 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: "prefix": { 2024-06-06T03:48:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prefix", 2024-06-06T03:48:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.365 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.365 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.365 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.365 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.365 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "name prefix for any created local pool", 2024-06-06T03:48:54.365 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.365 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.365 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.366 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.366 INFO:teuthology.orchestra.run.smithi040.stdout: "subtree": { 2024-06-06T03:48:54.366 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "subtree", 2024-06-06T03:48:54.366 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.366 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.366 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.366 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "rack", 2024-06-06T03:48:54.366 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.367 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.367 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.367 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T03:48:54.367 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T03:48:54.367 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.367 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.367 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.367 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.368 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.368 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mds_autoscaler", 2024-06-06T03:48:54.368 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.368 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.368 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.368 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.368 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.368 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.369 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.369 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.369 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.369 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.369 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.369 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.369 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.369 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.370 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.370 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.370 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.370 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.370 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.370 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.370 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.370 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.371 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.371 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.371 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.371 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.371 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.371 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.371 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.371 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.372 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.372 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.372 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.372 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.372 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.372 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.372 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.373 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.373 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.373 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.373 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.373 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.373 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.373 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.373 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.373 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.374 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.374 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.374 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.374 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.374 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.374 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.374 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.374 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.375 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.375 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.375 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.375 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.375 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.375 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.375 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.376 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.376 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.376 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.376 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.376 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.376 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.376 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.376 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.377 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.377 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.377 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.377 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.377 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.377 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.377 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.377 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mirroring", 2024-06-06T03:48:54.377 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.378 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.378 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.378 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.378 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.378 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.378 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.378 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.378 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.379 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.379 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.379 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.379 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.379 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.379 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.379 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.379 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.380 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.380 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.380 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.380 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.380 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.380 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.380 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.380 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.382 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.382 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.382 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.382 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.382 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.382 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.382 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.382 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.383 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.383 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.383 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.383 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.383 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.383 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.383 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.383 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.384 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.384 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.384 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.384 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.384 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.384 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.384 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.384 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.385 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.385 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.385 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.385 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.385 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.385 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.385 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.386 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.386 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.386 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.386 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.386 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.386 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.386 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.386 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.387 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.387 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.387 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "nfs", 2024-06-06T03:48:54.387 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.387 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.387 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.387 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.387 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.388 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.388 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.388 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.388 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.388 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.388 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.388 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.388 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.388 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.389 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.389 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.389 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.389 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.389 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.389 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.389 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.389 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.390 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.390 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.390 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.390 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.390 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.390 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.390 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.391 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.391 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.391 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.391 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.391 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.391 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.391 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.391 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.392 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.392 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.392 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.392 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.392 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.392 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.392 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.392 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.394 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.394 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.394 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.394 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.394 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.394 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.394 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.395 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.395 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.395 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.395 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.395 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.395 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.395 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.395 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.396 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.396 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.396 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.396 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "orchestrator", 2024-06-06T03:48:54.396 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.396 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.396 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.396 INFO:teuthology.orchestra.run.smithi040.stdout: "fail_fs": { 2024-06-06T03:48:54.397 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "fail_fs", 2024-06-06T03:48:54.397 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.397 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.397 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.397 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.397 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.397 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.397 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.398 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T03:48:54.398 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.398 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.398 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.398 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.398 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.398 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.399 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.399 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.399 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.399 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.399 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.399 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.399 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.399 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.401 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.401 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.401 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.401 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.401 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.401 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.401 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.402 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.402 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.402 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.402 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.402 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.402 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.402 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.403 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.403 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.403 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.403 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.403 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.403 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.403 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.403 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.404 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.404 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.404 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.404 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.404 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.404 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.404 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.404 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.404 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.405 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.405 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.405 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.405 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.405 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.405 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.405 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.405 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.406 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.406 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.406 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.406 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.406 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.406 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.406 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.407 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.407 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator": { 2024-06-06T03:48:54.407 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "orchestrator", 2024-06-06T03:48:54.407 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.407 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.407 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.407 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.407 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.408 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.408 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.408 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm", 2024-06-06T03:48:54.408 INFO:teuthology.orchestra.run.smithi040.stdout: "rook", 2024-06-06T03:48:54.408 INFO:teuthology.orchestra.run.smithi040.stdout: "test_orchestrator" 2024-06-06T03:48:54.408 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.408 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Orchestrator backend", 2024-06-06T03:48:54.408 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.408 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "osd_perf_query", 2024-06-06T03:48:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.410 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.410 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.410 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.410 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.410 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.410 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.410 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.410 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.412 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.412 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.412 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.412 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.412 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.412 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.412 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.413 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.413 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.413 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.413 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.413 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.413 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.413 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.413 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.414 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.414 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.414 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.414 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.414 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.414 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.414 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.414 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.415 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.415 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.415 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.415 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.415 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.415 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.415 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.415 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.416 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.416 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.416 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.416 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.416 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.416 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.416 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.416 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.417 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.417 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.417 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.417 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.417 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.417 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.417 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.417 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.418 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.418 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.418 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.418 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.418 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.418 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "osd_support", 2024-06-06T03:48:54.418 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.419 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.419 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.419 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.419 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.419 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.419 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.419 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.419 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.421 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.421 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.421 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.421 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.421 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.421 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.421 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.422 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.422 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.422 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.422 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.422 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.422 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.422 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.422 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.423 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.423 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.423 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.423 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.423 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.423 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.423 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.423 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.424 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.424 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.424 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.424 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.424 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.424 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.424 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.424 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.424 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.426 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.426 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.426 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.426 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.426 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.426 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.426 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.426 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.427 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.427 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.427 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.427 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.427 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.427 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.427 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.428 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pg_autoscaler", 2024-06-06T03:48:54.428 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.428 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.428 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.428 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.428 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.428 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.428 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.429 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.429 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.429 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.429 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.429 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.429 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.429 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.429 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.430 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.430 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.430 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.430 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.430 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.430 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.430 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.430 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.431 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.431 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.431 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.431 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.431 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.431 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.431 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.432 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.432 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.432 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.432 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.432 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.432 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.432 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.433 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.433 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.433 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.433 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.433 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.433 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.433 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.433 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.434 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.434 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.434 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.434 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.434 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.434 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.434 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.434 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.435 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.435 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.435 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.435 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.435 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.435 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.435 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.436 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.436 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.436 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.436 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.436 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.436 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.436 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.436 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.437 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.437 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:54.437 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:54.437 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.437 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.437 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.437 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:54.438 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.438 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.438 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.438 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.438 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.438 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.438 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.438 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: "threshold": { 2024-06-06T03:48:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "threshold", 2024-06-06T03:48:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3.0", 2024-06-06T03:48:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1.0", 2024-06-06T03:48:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.440 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "scaling threshold", 2024-06-06T03:48:54.440 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T03:48:54.440 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.440 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.440 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.440 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.440 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.440 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "progress", 2024-06-06T03:48:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_pg_recovery_event": { 2024-06-06T03:48:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_pg_recovery_event", 2024-06-06T03:48:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.442 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.442 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.442 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.442 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.442 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T03:48:54.442 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.442 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.443 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.443 INFO:teuthology.orchestra.run.smithi040.stdout: "enabled": { 2024-06-06T03:48:54.443 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enabled", 2024-06-06T03:48:54.443 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.443 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.443 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.443 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.443 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.445 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.445 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.445 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.445 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.445 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.445 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.445 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.445 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.446 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.446 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.446 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.446 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.446 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.446 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.446 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.446 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.448 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.448 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.448 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.448 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.448 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.448 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.448 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.450 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.450 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.450 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.450 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.450 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.450 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.450 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.450 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.451 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.451 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.451 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.451 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.451 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.451 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.451 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.453 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.453 INFO:teuthology.orchestra.run.smithi040.stdout: "max_completed_events": { 2024-06-06T03:48:54.453 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_completed_events", 2024-06-06T03:48:54.453 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.453 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.453 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.453 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "50", 2024-06-06T03:48:54.454 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.454 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.454 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.454 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "number of past completed events to remember", 2024-06-06T03:48:54.454 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.454 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.454 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2024-06-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2024-06-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2024-06-06T03:48:54.456 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.456 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.456 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.456 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long the module is going to sleep", 2024-06-06T03:48:54.456 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.456 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.456 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.456 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus", 2024-06-06T03:48:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: "cache": { 2024-06-06T03:48:54.458 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cache", 2024-06-06T03:48:54.458 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.458 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.458 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.458 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.458 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.458 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.458 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.459 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.459 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.459 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.459 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.459 INFO:teuthology.orchestra.run.smithi040.stdout: "exclude_perf_counters": { 2024-06-06T03:48:54.459 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "exclude_perf_counters", 2024-06-06T03:48:54.459 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.460 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.460 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.460 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.460 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.460 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.460 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.460 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T03:48:54.460 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T03:48:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.462 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.462 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.462 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.462 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.462 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.462 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.462 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.462 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.463 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.463 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.463 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.463 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.463 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.463 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.463 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.464 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.464 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.464 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.464 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.464 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.464 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.464 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.464 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.465 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.465 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.465 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.465 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.465 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.465 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.465 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.465 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.466 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.466 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.466 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.466 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.466 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.466 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.466 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.466 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.467 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.467 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.467 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.467 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.467 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.467 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.467 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.467 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.468 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.468 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.468 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.468 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.468 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.468 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.468 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.469 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.469 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.469 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.469 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.469 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.469 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.469 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_stats_pools": { 2024-06-06T03:48:54.470 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_stats_pools", 2024-06-06T03:48:54.470 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.470 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.470 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.470 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.470 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.470 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T03:48:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T03:48:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "300", 2024-06-06T03:48:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "scrape_interval": { 2024-06-06T03:48:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "scrape_interval", 2024-06-06T03:48:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15.0", 2024-06-06T03:48:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2024-06-06T03:48:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2024-06-06T03:48:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "::", 2024-06-06T03:48:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T03:48:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2024-06-06T03:48:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2024-06-06T03:48:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "9283", 2024-06-06T03:48:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T03:48:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "stale_cache_strategy": { 2024-06-06T03:48:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "stale_cache_strategy", 2024-06-06T03:48:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "log", 2024-06-06T03:48:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_behaviour": { 2024-06-06T03:48:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_behaviour", 2024-06-06T03:48:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "default", 2024-06-06T03:48:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "default", 2024-06-06T03:48:54.482 INFO:teuthology.orchestra.run.smithi040.stdout: "error" 2024-06-06T03:48:54.482 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.482 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.482 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.482 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.482 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.482 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_error_status_code": { 2024-06-06T03:48:54.483 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_error_status_code", 2024-06-06T03:48:54.483 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.483 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.483 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.483 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "500", 2024-06-06T03:48:54.483 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "400", 2024-06-06T03:48:54.483 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "599", 2024-06-06T03:48:54.483 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.483 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.484 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.484 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.484 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.484 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.484 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.484 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.484 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.484 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_support", 2024-06-06T03:48:54.485 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.485 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.485 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.485 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.485 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.485 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.485 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.485 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.486 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.486 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.486 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.486 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.486 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.486 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.486 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.486 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.487 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.487 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.487 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.487 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.487 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.487 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.487 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.488 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.488 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.488 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.488 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.488 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.488 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.488 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.488 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.489 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.489 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.489 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.489 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.489 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.489 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.489 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.489 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.489 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.490 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.490 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.490 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.490 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.490 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.490 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.490 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.490 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.491 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.491 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.491 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.491 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.491 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.491 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.491 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.491 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.492 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.492 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.492 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.492 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.492 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.492 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.492 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.493 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.493 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.493 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.493 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.493 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.493 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.493 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.493 INFO:teuthology.orchestra.run.smithi040.stdout: "max_concurrent_snap_create": { 2024-06-06T03:48:54.494 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_concurrent_snap_create", 2024-06-06T03:48:54.494 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.494 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.494 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.494 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2024-06-06T03:48:54.494 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.494 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.494 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.495 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.495 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.495 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.495 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.495 INFO:teuthology.orchestra.run.smithi040.stdout: "mirror_snapshot_schedule": { 2024-06-06T03:48:54.495 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T03:48:54.495 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.496 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.496 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.496 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.496 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.496 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.496 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.496 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.496 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.496 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.497 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.497 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.497 INFO:teuthology.orchestra.run.smithi040.stdout: "trash_purge_schedule": { 2024-06-06T03:48:54.497 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "trash_purge_schedule", 2024-06-06T03:48:54.497 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.497 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.497 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.497 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.498 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.498 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.498 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.498 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.498 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.498 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.498 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.498 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.499 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.499 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.499 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "restful", 2024-06-06T03:48:54.499 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.499 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.499 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.499 INFO:teuthology.orchestra.run.smithi040.stdout: "enable_auth": { 2024-06-06T03:48:54.500 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enable_auth", 2024-06-06T03:48:54.500 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.500 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.500 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.500 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.500 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.500 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.500 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.500 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.501 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.501 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.501 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.501 INFO:teuthology.orchestra.run.smithi040.stdout: "key_file": { 2024-06-06T03:48:54.501 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "key_file", 2024-06-06T03:48:54.501 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.501 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.502 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.502 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.502 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.502 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.502 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.502 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.502 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.503 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.503 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.503 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.503 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.503 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.503 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.503 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.504 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.504 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.505 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.505 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.505 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.506 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.506 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.506 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.506 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.506 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.506 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.506 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.506 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.507 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.507 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.507 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.507 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.507 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.507 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.507 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.507 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.508 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.508 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.508 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.508 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.508 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.508 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.508 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.508 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.508 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.509 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.509 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.509 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.509 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.509 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.509 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.509 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.509 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.510 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.510 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.510 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.510 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.510 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.510 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.510 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.511 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.511 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.511 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.511 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.511 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.511 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.511 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.511 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.512 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.512 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.512 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.512 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.512 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.512 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.512 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.512 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.513 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.513 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.513 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.513 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.513 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.513 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2024-06-06T03:48:54.513 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2024-06-06T03:48:54.513 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.513 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.514 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.514 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.514 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.514 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.514 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.514 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.514 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.514 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.515 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.515 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.515 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2024-06-06T03:48:54.515 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2024-06-06T03:48:54.515 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.515 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.515 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.515 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.516 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.516 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.516 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.516 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.516 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.516 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.516 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.516 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.517 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.517 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.517 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.517 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rgw", 2024-06-06T03:48:54.517 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.517 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.517 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.517 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.518 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.518 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.518 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.518 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.518 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.518 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.518 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.518 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.519 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.519 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.519 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.519 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.519 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.519 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.519 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.519 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.520 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.520 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.520 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.520 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.520 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.520 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.520 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.520 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.521 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.521 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.521 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.521 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.521 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.521 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.521 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.521 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.522 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.522 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.522 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.522 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.522 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.522 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.522 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.522 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.523 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.523 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.523 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.523 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.523 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.523 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.523 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.523 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.523 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.524 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.524 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.524 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.524 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.524 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.524 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.524 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.524 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.524 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.525 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.525 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.525 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.525 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.525 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.525 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.525 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.525 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.525 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.526 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.526 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.526 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.526 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.526 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.526 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rook", 2024-06-06T03:48:54.526 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.526 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.527 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.527 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.527 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.527 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.527 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.527 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.527 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.527 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.528 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.528 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.528 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.528 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.528 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.528 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.528 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.528 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.529 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.529 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.529 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.529 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.529 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.529 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.529 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.529 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.530 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.530 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.530 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.530 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.530 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.530 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.530 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.531 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.531 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.531 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.531 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.531 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.531 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.531 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.531 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.532 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.532 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.532 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.532 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.532 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.532 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.532 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.532 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.533 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.533 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.533 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.533 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.533 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.533 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.533 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.533 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.534 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.534 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.534 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.534 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.534 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.534 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.534 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.535 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.535 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.535 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.535 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.535 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.535 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.535 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.535 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.536 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.536 INFO:teuthology.orchestra.run.smithi040.stdout: "storage_class": { 2024-06-06T03:48:54.536 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "storage_class", 2024-06-06T03:48:54.536 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.536 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.536 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.536 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "local", 2024-06-06T03:48:54.536 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.537 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.537 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.537 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T03:48:54.537 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.537 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.537 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.537 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.537 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.538 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.538 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.538 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "selftest", 2024-06-06T03:48:54.538 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.538 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.538 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.538 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.538 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.539 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.539 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.539 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.539 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.539 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.539 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.539 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.539 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.540 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.540 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.540 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.540 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.540 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.540 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.540 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.540 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.541 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.541 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.541 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.541 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.541 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.541 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.541 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.541 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.541 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.542 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.542 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.542 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.542 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.542 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.542 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.542 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.543 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.543 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.543 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.543 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.543 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.543 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.543 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.543 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.544 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.544 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.544 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.544 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.544 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.544 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.544 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.544 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.544 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.545 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.545 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.545 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.545 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.545 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.545 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.545 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.545 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.545 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.546 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.546 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.546 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.546 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.546 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.546 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.546 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.546 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.547 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.547 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.547 INFO:teuthology.orchestra.run.smithi040.stdout: "roption1": { 2024-06-06T03:48:54.547 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "roption1", 2024-06-06T03:48:54.547 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.547 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.547 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.547 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.548 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.548 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.548 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.548 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.548 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.548 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.548 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.548 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.549 INFO:teuthology.orchestra.run.smithi040.stdout: "roption2": { 2024-06-06T03:48:54.549 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "roption2", 2024-06-06T03:48:54.549 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.549 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.549 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.549 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "xyz", 2024-06-06T03:48:54.549 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.549 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.549 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.550 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.550 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.550 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.550 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.550 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.550 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption1": { 2024-06-06T03:48:54.550 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption1", 2024-06-06T03:48:54.550 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.551 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.551 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.551 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.551 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.551 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.551 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.551 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.551 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.551 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.552 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.552 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.552 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption2": { 2024-06-06T03:48:54.552 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption2", 2024-06-06T03:48:54.552 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.552 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.552 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.552 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.553 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.553 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.553 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.553 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.553 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.553 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.553 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.553 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.554 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption3": { 2024-06-06T03:48:54.554 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption3", 2024-06-06T03:48:54.554 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2024-06-06T03:48:54.554 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.554 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.554 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.554 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.554 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.555 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.555 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.555 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.555 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.555 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.555 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.555 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption4": { 2024-06-06T03:48:54.555 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption4", 2024-06-06T03:48:54.555 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.556 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.556 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.556 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.556 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.556 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.556 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.556 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.556 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.557 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.557 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.557 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption5": { 2024-06-06T03:48:54.557 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption5", 2024-06-06T03:48:54.557 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.557 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.557 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.558 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.558 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.558 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.558 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.558 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.558 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.558 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.558 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.559 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption6": { 2024-06-06T03:48:54.559 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption6", 2024-06-06T03:48:54.559 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.559 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.559 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.559 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.559 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.559 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.560 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.560 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.560 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.560 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.560 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.560 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption7": { 2024-06-06T03:48:54.560 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption7", 2024-06-06T03:48:54.561 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.561 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.561 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.561 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.561 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2024-06-06T03:48:54.561 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "42", 2024-06-06T03:48:54.561 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.561 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.561 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.562 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.562 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.562 INFO:teuthology.orchestra.run.smithi040.stdout: "testkey": { 2024-06-06T03:48:54.562 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testkey", 2024-06-06T03:48:54.562 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.562 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.562 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.563 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.563 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.563 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.563 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.563 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.563 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.563 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.563 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.564 INFO:teuthology.orchestra.run.smithi040.stdout: "testlkey": { 2024-06-06T03:48:54.564 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testlkey", 2024-06-06T03:48:54.564 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.564 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.564 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.564 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.564 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.564 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.565 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.565 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.565 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.565 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.565 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.565 INFO:teuthology.orchestra.run.smithi040.stdout: "testnewline": { 2024-06-06T03:48:54.565 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testnewline", 2024-06-06T03:48:54.566 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.566 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.566 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.566 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.566 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.566 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.566 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.566 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.567 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.567 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.567 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.567 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.567 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.567 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.567 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.567 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snap_schedule", 2024-06-06T03:48:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_m_granularity": { 2024-06-06T03:48:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_m_granularity", 2024-06-06T03:48:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T03:48:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: "dump_on_update": { 2024-06-06T03:48:54.570 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "dump_on_update", 2024-06-06T03:48:54.570 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.570 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.570 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.570 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.570 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.570 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.570 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.571 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "dump database to debug log on update", 2024-06-06T03:48:54.571 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.571 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.571 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.571 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.571 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.571 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.572 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.572 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.572 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.572 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.572 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.572 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.572 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.572 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.573 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.573 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.573 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.573 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.573 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.573 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.573 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.573 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.574 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.574 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.574 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.574 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.574 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.574 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.574 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.575 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.575 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.575 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.575 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.575 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.575 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.575 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.576 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.576 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.576 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.576 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.576 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.576 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.576 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.576 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.577 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.577 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.577 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.577 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.577 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.577 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.577 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.577 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.577 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.578 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.578 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.578 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.578 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.578 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.578 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.579 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.579 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.579 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.579 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.579 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.579 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.579 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.579 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.580 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.580 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.580 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.580 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.580 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.580 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "stats", 2024-06-06T03:48:54.580 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.581 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.581 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.581 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.581 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.581 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.581 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.581 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.581 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.582 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.582 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.582 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.582 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.582 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.582 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.582 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.582 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.583 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.583 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.583 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.583 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.583 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.583 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.583 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.584 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.584 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.584 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.584 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.584 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.584 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.584 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.584 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.585 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.585 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.585 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.585 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.585 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.585 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.585 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.585 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.585 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.586 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.586 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.586 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.586 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.586 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.586 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.586 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.586 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.587 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.587 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.587 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.587 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.587 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.587 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.587 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.587 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.588 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.588 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.588 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.588 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.588 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.588 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.588 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.589 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.589 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.589 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.589 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.589 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.589 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.589 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.589 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.590 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.590 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "status", 2024-06-06T03:48:54.590 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.590 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.590 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.590 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.590 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.591 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.591 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.591 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.591 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.591 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.591 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.591 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.591 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.592 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.592 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.592 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.592 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.592 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.592 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.592 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.592 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.592 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.593 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.593 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.593 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.593 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.593 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.593 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.593 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.593 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.594 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.594 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.594 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.594 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.594 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.594 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.594 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.595 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.595 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.595 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.595 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.595 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.595 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.595 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.595 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.596 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.596 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.596 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.596 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.596 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.596 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.596 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.596 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.597 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.597 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.597 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.597 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.597 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.597 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.597 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.598 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.598 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.598 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.598 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.598 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.598 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.598 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.598 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.599 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.599 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "telegraf", 2024-06-06T03:48:54.599 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.599 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.599 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.600 INFO:teuthology.orchestra.run.smithi040.stdout: "address": { 2024-06-06T03:48:54.600 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "address", 2024-06-06T03:48:54.600 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.600 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.600 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.600 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T03:48:54.600 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.600 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.601 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.601 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.601 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.601 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.601 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.601 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:54.601 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:54.602 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.602 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.602 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.602 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15", 2024-06-06T03:48:54.602 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.602 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.602 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.602 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.603 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.603 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.603 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.603 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.603 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.603 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.603 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.604 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.604 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.604 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.604 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.604 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.604 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.604 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.604 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.604 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.605 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.605 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.605 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.605 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.605 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.605 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.605 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.605 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.605 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.606 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.606 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.606 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.606 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.606 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.606 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.606 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.606 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.607 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.607 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.607 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.607 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.607 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.607 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.607 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.608 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.608 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.608 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.608 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.608 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.608 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.608 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.608 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.609 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.609 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.609 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.609 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.609 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.609 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.609 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.609 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.610 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.610 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.610 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.610 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.610 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.610 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.610 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.611 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.611 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.611 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.611 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.611 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.611 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.611 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.611 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.612 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "telemetry", 2024-06-06T03:48:54.612 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.612 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.612 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.612 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_basic": { 2024-06-06T03:48:54.612 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_basic", 2024-06-06T03:48:54.612 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.612 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.613 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.613 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.613 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.613 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.613 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.613 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T03:48:54.613 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.613 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.613 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.614 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_crash": { 2024-06-06T03:48:54.614 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_crash", 2024-06-06T03:48:54.614 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.614 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.614 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.614 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.614 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.614 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.615 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.615 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T03:48:54.615 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.615 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.615 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.615 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_device": { 2024-06-06T03:48:54.615 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_device", 2024-06-06T03:48:54.615 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.616 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.616 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.616 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.616 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.616 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.616 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.616 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T03:48:54.616 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.617 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.617 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.617 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_ident": { 2024-06-06T03:48:54.617 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_ident", 2024-06-06T03:48:54.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.617 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.617 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.618 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.618 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.618 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.618 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.618 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T03:48:54.618 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.618 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.619 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.619 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_perf": { 2024-06-06T03:48:54.619 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_perf", 2024-06-06T03:48:54.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.619 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.619 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.619 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.620 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.620 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.620 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.620 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T03:48:54.620 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.620 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.620 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.620 INFO:teuthology.orchestra.run.smithi040.stdout: "contact": { 2024-06-06T03:48:54.621 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "contact", 2024-06-06T03:48:54.621 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.621 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.621 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.621 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.621 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.621 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.621 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.622 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.622 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.622 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.622 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.622 INFO:teuthology.orchestra.run.smithi040.stdout: "description": { 2024-06-06T03:48:54.622 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "description", 2024-06-06T03:48:54.622 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.623 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.623 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.623 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.623 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.623 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.623 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.623 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.623 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.624 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.624 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.624 INFO:teuthology.orchestra.run.smithi040.stdout: "device_url": { 2024-06-06T03:48:54.624 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_url", 2024-06-06T03:48:54.624 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.624 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.624 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.624 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T03:48:54.625 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.625 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.625 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.625 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.625 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.625 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.625 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.626 INFO:teuthology.orchestra.run.smithi040.stdout: "enabled": { 2024-06-06T03:48:54.626 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enabled", 2024-06-06T03:48:54.626 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.626 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.626 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.626 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.626 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.626 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.627 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.627 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.627 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.627 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.627 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.627 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:54.627 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:54.627 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.628 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.628 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.628 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "24", 2024-06-06T03:48:54.628 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "8", 2024-06-06T03:48:54.628 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.628 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.628 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.628 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.629 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.629 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.629 INFO:teuthology.orchestra.run.smithi040.stdout: "last_opt_revision": { 2024-06-06T03:48:54.629 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "last_opt_revision", 2024-06-06T03:48:54.629 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.629 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.629 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.629 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1", 2024-06-06T03:48:54.630 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.630 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.630 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.630 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.630 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.630 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.630 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.631 INFO:teuthology.orchestra.run.smithi040.stdout: "leaderboard": { 2024-06-06T03:48:54.631 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "leaderboard", 2024-06-06T03:48:54.631 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.631 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.631 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.631 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.631 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.631 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.631 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.632 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.632 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.632 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.632 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.632 INFO:teuthology.orchestra.run.smithi040.stdout: "leaderboard_description": { 2024-06-06T03:48:54.632 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "leaderboard_description", 2024-06-06T03:48:54.632 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.633 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.633 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.633 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.633 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.633 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.633 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.633 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.633 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.634 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.634 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.634 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.634 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.634 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.634 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.635 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.635 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.635 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.635 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.635 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.635 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.635 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.635 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.635 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.636 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.636 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.636 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.636 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.636 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.636 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.636 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.637 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.637 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.637 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.637 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.637 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.637 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.637 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.637 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.637 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.638 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.638 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.638 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.638 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.638 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.638 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.639 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.639 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.639 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.639 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.639 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.639 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.639 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.639 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.640 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.640 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.640 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.640 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.640 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.640 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.640 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.640 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.640 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.641 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.641 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.641 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.641 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.641 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.641 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.641 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.642 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.642 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.642 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.642 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.642 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.642 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.642 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "organization": { 2024-06-06T03:48:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "organization", 2024-06-06T03:48:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "proxy": { 2024-06-06T03:48:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "proxy", 2024-06-06T03:48:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "url": { 2024-06-06T03:48:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "url", 2024-06-06T03:48:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T03:48:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "test_orchestrator", 2024-06-06T03:48:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.656 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.656 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.656 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.656 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.656 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.656 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.657 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.657 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.657 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.657 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.657 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.657 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.657 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.658 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.658 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.658 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.658 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.658 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.658 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.659 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.659 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.659 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.659 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.659 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "volumes", 2024-06-06T03:48:54.659 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.659 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.659 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.660 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.660 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.660 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.660 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.660 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.660 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.660 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.660 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.661 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.661 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.661 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.661 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.661 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.661 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.661 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.661 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.662 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.662 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.662 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.662 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.662 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.662 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.663 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.663 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.663 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.663 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.663 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.663 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.664 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.664 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.664 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.664 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.664 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.664 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.664 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.664 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.665 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.665 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.665 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.665 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.665 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.665 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.665 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.666 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.666 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.666 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.666 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.666 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.666 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.666 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.666 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.666 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.667 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.667 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.667 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.667 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.667 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.667 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.667 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.668 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.668 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.668 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.668 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.668 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.668 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.668 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.668 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.668 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi040.stdout: "max_concurrent_clones": { 2024-06-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_concurrent_clones", 2024-06-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "4", 2024-06-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.670 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T03:48:54.670 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.670 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.670 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.670 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.670 INFO:teuthology.orchestra.run.smithi040.stdout: "periodic_async_work": { 2024-06-06T03:48:54.670 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "periodic_async_work", 2024-06-06T03:48:54.670 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.671 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.671 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.671 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.671 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.671 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.671 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.671 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Periodically check for async work", 2024-06-06T03:48:54.671 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.672 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.672 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.672 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.672 INFO:teuthology.orchestra.run.smithi040.stdout: "snapshot_clone_delay": { 2024-06-06T03:48:54.672 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snapshot_clone_delay", 2024-06-06T03:48:54.672 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.672 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.672 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.673 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2024-06-06T03:48:54.673 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.673 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.673 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.673 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T03:48:54.673 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.673 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.673 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.674 INFO:teuthology.orchestra.run.smithi040.stdout: "snapshot_clone_no_wait": { 2024-06-06T03:48:54.674 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T03:48:54.674 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.674 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.674 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.674 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2024-06-06T03:48:54.674 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.674 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.675 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.675 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T03:48:54.675 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.675 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.675 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.675 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.675 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.675 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.676 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix", 2024-06-06T03:48:54.676 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2024-06-06T03:48:54.676 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2024-06-06T03:48:54.676 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2024-06-06T03:48:54.676 INFO:teuthology.orchestra.run.smithi040.stdout: "discovery_interval": { 2024-06-06T03:48:54.676 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "discovery_interval", 2024-06-06T03:48:54.676 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2024-06-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "100", 2024-06-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi040.stdout: "identifier": { 2024-06-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "identifier", 2024-06-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.680 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2024-06-06T03:48:54.680 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2024-06-06T03:48:54.680 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2024-06-06T03:48:54.680 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.680 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.680 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2024-06-06T03:48:54.680 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.680 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.680 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.681 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.681 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.681 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.681 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.681 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.681 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2024-06-06T03:48:54.681 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2024-06-06T03:48:54.681 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.681 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.682 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.682 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.682 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.682 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.682 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.682 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.682 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.682 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.683 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.683 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.683 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.683 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.683 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.683 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.683 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.683 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.684 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.684 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2024-06-06T03:48:54.684 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2024-06-06T03:48:54.684 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.684 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.684 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.684 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.684 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.684 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.685 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.685 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.685 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.685 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.685 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.685 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.685 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2024-06-06T03:48:54.685 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2024-06-06T03:48:54.686 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.686 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.686 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.686 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2024-06-06T03:48:54.686 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.686 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.686 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2024-06-06T03:48:54.686 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2024-06-06T03:48:54.687 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2024-06-06T03:48:54.687 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2024-06-06T03:48:54.687 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2024-06-06T03:48:54.687 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2024-06-06T03:48:54.687 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2024-06-06T03:48:54.687 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.687 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.687 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.687 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.688 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.688 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.688 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2024-06-06T03:48:54.688 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2024-06-06T03:48:54.688 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2024-06-06T03:48:54.688 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.688 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2024-06-06T03:48:54.688 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2024-06-06T03:48:54.689 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.689 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.689 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.689 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.689 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.689 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.689 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.689 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_host": { 2024-06-06T03:48:54.690 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_host", 2024-06-06T03:48:54.690 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.690 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.690 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.690 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2024-06-06T03:48:54.690 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.690 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.690 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.691 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.691 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.691 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.691 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.691 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.691 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_port": { 2024-06-06T03:48:54.691 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_port", 2024-06-06T03:48:54.691 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2024-06-06T03:48:54.692 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.692 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.692 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10051", 2024-06-06T03:48:54.692 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.692 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.692 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.692 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.692 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.693 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.693 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.693 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_sender": { 2024-06-06T03:48:54.693 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_sender", 2024-06-06T03:48:54.693 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2024-06-06T03:48:54.693 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2024-06-06T03:48:54.693 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2024-06-06T03:48:54.693 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T03:48:54.694 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2024-06-06T03:48:54.694 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2024-06-06T03:48:54.694 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2024-06-06T03:48:54.694 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2024-06-06T03:48:54.694 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2024-06-06T03:48:54.694 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2024-06-06T03:48:54.695 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2024-06-06T03:48:54.695 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.695 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.695 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.695 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.695 INFO:teuthology.orchestra.run.smithi040.stdout: "services": { 2024-06-06T03:48:54.695 INFO:teuthology.orchestra.run.smithi040.stdout: "dashboard": "https://172.21.15.40:8443/" 2024-06-06T03:48:54.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.696 INFO:teuthology.orchestra.run.smithi040.stdout: "always_on_modules": { 2024-06-06T03:48:54.696 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus": [ 2024-06-06T03:48:54.696 INFO:teuthology.orchestra.run.smithi040.stdout: "balancer", 2024-06-06T03:48:54.696 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-06-06T03:48:54.696 INFO:teuthology.orchestra.run.smithi040.stdout: "devicehealth", 2024-06-06T03:48:54.696 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator", 2024-06-06T03:48:54.696 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscaler", 2024-06-06T03:48:54.696 INFO:teuthology.orchestra.run.smithi040.stdout: "progress", 2024-06-06T03:48:54.697 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_support", 2024-06-06T03:48:54.697 INFO:teuthology.orchestra.run.smithi040.stdout: "status", 2024-06-06T03:48:54.697 INFO:teuthology.orchestra.run.smithi040.stdout: "telemetry", 2024-06-06T03:48:54.697 INFO:teuthology.orchestra.run.smithi040.stdout: "volumes" 2024-06-06T03:48:54.697 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.697 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific": [ 2024-06-06T03:48:54.697 INFO:teuthology.orchestra.run.smithi040.stdout: "balancer", 2024-06-06T03:48:54.697 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-06-06T03:48:54.698 INFO:teuthology.orchestra.run.smithi040.stdout: "devicehealth", 2024-06-06T03:48:54.698 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator", 2024-06-06T03:48:54.698 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscaler", 2024-06-06T03:48:54.698 INFO:teuthology.orchestra.run.smithi040.stdout: "progress", 2024-06-06T03:48:54.698 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_support", 2024-06-06T03:48:54.698 INFO:teuthology.orchestra.run.smithi040.stdout: "status", 2024-06-06T03:48:54.698 INFO:teuthology.orchestra.run.smithi040.stdout: "telemetry", 2024-06-06T03:48:54.698 INFO:teuthology.orchestra.run.smithi040.stdout: "volumes" 2024-06-06T03:48:54.699 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.699 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy": [ 2024-06-06T03:48:54.699 INFO:teuthology.orchestra.run.smithi040.stdout: "balancer", 2024-06-06T03:48:54.699 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-06-06T03:48:54.699 INFO:teuthology.orchestra.run.smithi040.stdout: "devicehealth", 2024-06-06T03:48:54.699 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator", 2024-06-06T03:48:54.699 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscaler", 2024-06-06T03:48:54.699 INFO:teuthology.orchestra.run.smithi040.stdout: "progress", 2024-06-06T03:48:54.700 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_support", 2024-06-06T03:48:54.700 INFO:teuthology.orchestra.run.smithi040.stdout: "status", 2024-06-06T03:48:54.700 INFO:teuthology.orchestra.run.smithi040.stdout: "telemetry", 2024-06-06T03:48:54.700 INFO:teuthology.orchestra.run.smithi040.stdout: "volumes" 2024-06-06T03:48:54.700 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.700 INFO:teuthology.orchestra.run.smithi040.stdout: "reef": [ 2024-06-06T03:48:54.700 INFO:teuthology.orchestra.run.smithi040.stdout: "balancer", 2024-06-06T03:48:54.700 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-06-06T03:48:54.700 INFO:teuthology.orchestra.run.smithi040.stdout: "devicehealth", 2024-06-06T03:48:54.701 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator", 2024-06-06T03:48:54.701 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscaler", 2024-06-06T03:48:54.701 INFO:teuthology.orchestra.run.smithi040.stdout: "progress", 2024-06-06T03:48:54.701 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_support", 2024-06-06T03:48:54.701 INFO:teuthology.orchestra.run.smithi040.stdout: "status", 2024-06-06T03:48:54.701 INFO:teuthology.orchestra.run.smithi040.stdout: "telemetry", 2024-06-06T03:48:54.701 INFO:teuthology.orchestra.run.smithi040.stdout: "volumes" 2024-06-06T03:48:54.701 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-06-06T03:48:54.702 INFO:teuthology.orchestra.run.smithi040.stdout: "squid": [ 2024-06-06T03:48:54.702 INFO:teuthology.orchestra.run.smithi040.stdout: "balancer", 2024-06-06T03:48:54.702 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-06-06T03:48:54.702 INFO:teuthology.orchestra.run.smithi040.stdout: "devicehealth", 2024-06-06T03:48:54.702 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator", 2024-06-06T03:48:54.702 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscaler", 2024-06-06T03:48:54.702 INFO:teuthology.orchestra.run.smithi040.stdout: "progress", 2024-06-06T03:48:54.702 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_support", 2024-06-06T03:48:54.703 INFO:teuthology.orchestra.run.smithi040.stdout: "status", 2024-06-06T03:48:54.703 INFO:teuthology.orchestra.run.smithi040.stdout: "telemetry", 2024-06-06T03:48:54.703 INFO:teuthology.orchestra.run.smithi040.stdout: "volumes" 2024-06-06T03:48:54.703 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-06-06T03:48:54.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.703 INFO:teuthology.orchestra.run.smithi040.stdout: "last_failure_osd_epoch": 91, 2024-06-06T03:48:54.703 INFO:teuthology.orchestra.run.smithi040.stdout: "active_clients": [ 2024-06-06T03:48:54.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.703 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "devicehealth", 2024-06-06T03:48:54.704 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-06-06T03:48:54.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.704 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-06-06T03:48:54.704 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:0", 2024-06-06T03:48:54.704 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2166696380 2024-06-06T03:48:54.704 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.704 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-06-06T03:48:54.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.705 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "libcephsqlite", 2024-06-06T03:48:54.705 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-06-06T03:48:54.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.705 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-06-06T03:48:54.705 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:0", 2024-06-06T03:48:54.705 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 4102988406 2024-06-06T03:48:54.705 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.705 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-06-06T03:48:54.706 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.706 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_support", 2024-06-06T03:48:54.706 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-06-06T03:48:54.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.706 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-06-06T03:48:54.706 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:0", 2024-06-06T03:48:54.709 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3867902101 2024-06-06T03:48:54.709 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.709 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-06-06T03:48:54.710 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.710 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.710 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snap_schedule", 2024-06-06T03:48:54.710 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-06-06T03:48:54.710 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.710 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-06-06T03:48:54.710 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:0", 2024-06-06T03:48:54.711 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 434283748 2024-06-06T03:48:54.711 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.711 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-06-06T03:48:54.711 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.711 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.711 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snap_schedule", 2024-06-06T03:48:54.711 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-06-06T03:48:54.711 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.711 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-06-06T03:48:54.711 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:0", 2024-06-06T03:48:54.712 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2731606137 2024-06-06T03:48:54.712 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.712 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-06-06T03:48:54.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:54.712 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.712 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "volumes", 2024-06-06T03:48:54.712 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-06-06T03:48:54.712 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-06-06T03:48:54.713 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-06-06T03:48:54.713 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:0", 2024-06-06T03:48:54.713 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3360552348 2024-06-06T03:48:54.713 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.713 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-06-06T03:48:54.713 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:54.713 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-06-06T03:48:54.713 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:48:54.715 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:52 smithi104 ceph-mon[36569]: pgmap v318: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 90 KiB/s wr, 38 op/s 2024-06-06T03:48:54.715 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:52 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2817560726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T03:48:54.715 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:52 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/32819536' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T03:48:54.715 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:52 smithi131 ceph-mon[36687]: pgmap v318: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 90 KiB/s wr, 38 op/s 2024-06-06T03:48:54.715 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:52 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2817560726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T03:48:54.715 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:52 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/32819536' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T03:48:54.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.498+0000 7f81679d5640 1 -- 172.21.15.40:0/32819536 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f814c0a2d80 msgr2=0x7f814c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:54.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.498+0000 7f81679d5640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f814c0a2d80 0x7f814c0a5240 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8150005d00 tx=0x7f8150009550 comp rx=0 tx=0).stop 2024-06-06T03:48:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.498+0000 7f81679d5640 1 -- 172.21.15.40:0/32819536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8160102c80 msgr2=0x7f8160198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:48:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.498+0000 7f81679d5640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8160102c80 0x7f8160198250 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f815c006010 tx=0x7f815c0042f0 comp rx=0 tx=0).stop 2024-06-06T03:48:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.498+0000 7f81679d5640 1 -- 172.21.15.40:0/32819536 shutdown_connections 2024-06-06T03:48:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.498+0000 7f81679d5640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f814c0a2d80 0x7f814c0a5240 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.498+0000 7f81679d5640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f816010bfc0 0x7f816019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.498+0000 7f81679d5640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81601035a0 0x7f8160198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.498+0000 7f81679d5640 1 --2- 172.21.15.40:0/32819536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8160102c80 0x7f8160198250 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:48:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.498+0000 7f81679d5640 1 -- 172.21.15.40:0/32819536 >> 172.21.15.40:0/32819536 conn(0x7f8160077a00 msgr2=0x7f81600fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:48:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.498+0000 7f81679d5640 1 -- 172.21.15.40:0/32819536 shutdown_connections 2024-06-06T03:48:54.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:48:52.498+0000 7f81679d5640 1 -- 172.21.15.40:0/32819536 wait complete. 2024-06-06T03:48:54.726 DEBUG:tasks.check_counter:Getting stats from a 2024-06-06T03:48:54.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-06T03:48:54.727 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:52 smithi040 ceph-mon[32220]: pgmap v318: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 90 KiB/s wr, 38 op/s 2024-06-06T03:48:54.727 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:52 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2817560726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T03:48:54.727 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:52 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/32819536' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T03:48:54.991 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:48:55.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:54 smithi040 ceph-mon[32220]: pgmap v319: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 86 KiB/s wr, 31 op/s 2024-06-06T03:48:55.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:54 smithi131 ceph-mon[36687]: pgmap v319: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 86 KiB/s wr, 31 op/s 2024-06-06T03:48:55.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:54 smithi104 ceph-mon[36569]: pgmap v319: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 86 KiB/s wr, 31 op/s 2024-06-06T03:48:56.780 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:48:56.780 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T03:48:56.781 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1294, 2024-06-06T03:48:56.781 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1296, 2024-06-06T03:48:56.781 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 14361238, 2024-06-06T03:48:56.781 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 411493, 2024-06-06T03:48:56.781 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 7, 2024-06-06T03:48:56.781 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 3, 2024-06-06T03:48:56.781 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.123888983, 2024-06-06T03:48:56.781 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.036187081, 2024-06-06T03:48:56.782 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.039452965, 2024-06-06T03:48:56.782 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.037546984, 2024-06-06T03:48:56.782 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:48:56.782 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1294, 2024-06-06T03:48:56.782 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.049025255, 2024-06-06T03:48:56.782 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000037886 2024-06-06T03:48:56.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.782 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:48:56.783 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.783 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.783 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.783 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T03:48:56.783 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T03:48:56.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.783 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T03:48:56.784 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 829, 2024-06-06T03:48:56.784 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 802, 2024-06-06T03:48:56.784 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 39609179, 2024-06-06T03:48:56.784 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 320018, 2024-06-06T03:48:56.784 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 5, 2024-06-06T03:48:56.784 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 4, 2024-06-06T03:48:56.784 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.115165210, 2024-06-06T03:48:56.784 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.027873148, 2024-06-06T03:48:56.785 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.074304325, 2024-06-06T03:48:56.785 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.019119336, 2024-06-06T03:48:56.785 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:48:56.785 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 801, 2024-06-06T03:48:56.785 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.029251330, 2024-06-06T03:48:56.785 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000036518 2024-06-06T03:48:56.785 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.785 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:48:56.786 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.786 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.786 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.786 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 1715040, 2024-06-06T03:48:56.786 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 141552 2024-06-06T03:48:56.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.786 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T03:48:56.787 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 585, 2024-06-06T03:48:56.787 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 752, 2024-06-06T03:48:56.787 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 25324455, 2024-06-06T03:48:56.787 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 645724, 2024-06-06T03:48:56.787 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 8, 2024-06-06T03:48:56.787 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 6, 2024-06-06T03:48:56.787 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.077751775, 2024-06-06T03:48:56.787 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.022311521, 2024-06-06T03:48:56.787 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.032864531, 2024-06-06T03:48:56.788 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.016294552, 2024-06-06T03:48:56.788 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:48:56.788 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 750, 2024-06-06T03:48:56.788 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.031599141, 2024-06-06T03:48:56.788 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000042132 2024-06-06T03:48:56.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.788 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:48:56.788 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.789 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.789 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.789 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-06-06T03:48:56.789 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 461920 2024-06-06T03:48:56.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.789 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2024-06-06T03:48:56.789 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 1, 2024-06-06T03:48:56.790 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2024-06-06T03:48:56.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.790 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-MDSRank": { 2024-06-06T03:48:56.790 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-06-06T03:48:56.790 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-06-06T03:48:56.790 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 4102, 2024-06-06T03:48:56.790 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.106365266, 2024-06-06T03:48:56.790 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000025930 2024-06-06T03:48:56.791 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:56.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.791 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-PurgeQueue": { 2024-06-06T03:48:56.791 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-06-06T03:48:56.791 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-06-06T03:48:56.791 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.791 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.791 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.791 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:56.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-06-06T03:48:56.792 INFO:teuthology.orchestra.run.smithi040.stdout: "request": 0, 2024-06-06T03:48:56.792 INFO:teuthology.orchestra.run.smithi040.stdout: "reply": 0, 2024-06-06T03:48:56.792 INFO:teuthology.orchestra.run.smithi040.stdout: "reply_latency": { 2024-06-06T03:48:56.792 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.792 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.792 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.793 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_reply": 0, 2024-06-06T03:48:56.793 INFO:teuthology.orchestra.run.smithi040.stdout: "forward": 0, 2024-06-06T03:48:56.793 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_complete": 0, 2024-06-06T03:48:56.793 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_keys": 0, 2024-06-06T03:48:56.793 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_commit": 0, 2024-06-06T03:48:56.793 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_split": 0, 2024-06-06T03:48:56.793 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_merge": 0, 2024-06-06T03:48:56.794 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes": 18543, 2024-06-06T03:48:56.794 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_top": 357, 2024-06-06T03:48:56.794 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_bottom": 18186, 2024-06-06T03:48:56.794 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pin_tail": 0, 2024-06-06T03:48:56.794 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pinned": 18034, 2024-06-06T03:48:56.794 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_expired": 0, 2024-06-06T03:48:56.794 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_with_caps": 0, 2024-06-06T03:48:56.794 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": 0, 2024-06-06T03:48:56.795 INFO:teuthology.orchestra.run.smithi040.stdout: "subtrees": 78, 2024-06-06T03:48:56.795 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse": 0, 2024-06-06T03:48:56.795 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_hit": 0, 2024-06-06T03:48:56.795 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_forward": 0, 2024-06-06T03:48:56.795 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_discover": 0, 2024-06-06T03:48:56.795 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_dir_fetch": 0, 2024-06-06T03:48:56.795 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_remote_ino": 0, 2024-06-06T03:48:56.795 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_lock": 0, 2024-06-06T03:48:56.796 INFO:teuthology.orchestra.run.smithi040.stdout: "load_cent": 0, 2024-06-06T03:48:56.796 INFO:teuthology.orchestra.run.smithi040.stdout: "q": 0, 2024-06-06T03:48:56.796 INFO:teuthology.orchestra.run.smithi040.stdout: "exported": 0, 2024-06-06T03:48:56.796 INFO:teuthology.orchestra.run.smithi040.stdout: "exported_inodes": 0, 2024-06-06T03:48:56.796 INFO:teuthology.orchestra.run.smithi040.stdout: "imported": 0, 2024-06-06T03:48:56.796 INFO:teuthology.orchestra.run.smithi040.stdout: "imported_inodes": 0, 2024-06-06T03:48:56.796 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_dir_fetch": 0, 2024-06-06T03:48:56.796 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_backtrace_fetch": 0, 2024-06-06T03:48:56.797 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_peer_discover": 0, 2024-06-06T03:48:56.797 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rfiles": 0, 2024-06-06T03:48:56.797 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rbytes": 0, 2024-06-06T03:48:56.797 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rsnaps": 0, 2024-06-06T03:48:56.797 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T03:48:56.797 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_set_tag": 0, 2024-06-06T03:48:56.797 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T03:48:56.797 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_inotable_repaired": 0, 2024-06-06T03:48:56.797 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_inodes": 0, 2024-06-06T03:48:56.798 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T03:48:56.798 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T03:48:56.798 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_file_inodes": 0, 2024-06-06T03:48:56.798 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_inode_file_caps": 0, 2024-06-06T03:48:56.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T03:48:56.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_grant": 0, 2024-06-06T03:48:56.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T03:48:56.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T03:48:56.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T03:48:56.799 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps": 0, 2024-06-06T03:48:56.799 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps_dirty": 0, 2024-06-06T03:48:56.799 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_cap_release": 0, 2024-06-06T03:48:56.799 INFO:teuthology.orchestra.run.smithi040.stdout: "process_request_cap_release": 0 2024-06-06T03:48:56.799 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.799 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache": { 2024-06-06T03:48:56.800 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update": 0, 2024-06-06T03:48:56.800 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update_receipt": 0, 2024-06-06T03:48:56.800 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_try_discover": 0, 2024-06-06T03:48:56.800 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_send_discover": 0, 2024-06-06T03:48:56.800 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_handle_discover": 0, 2024-06-06T03:48:56.800 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays": 0, 2024-06-06T03:48:56.800 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_delayed": 0, 2024-06-06T03:48:56.800 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_enqueuing": 0, 2024-06-06T03:48:56.801 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_created": 0, 2024-06-06T03:48:56.801 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_enqueued": 0, 2024-06-06T03:48:56.801 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_reintegrated": 0, 2024-06-06T03:48:56.801 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_migrated": 0, 2024-06-06T03:48:56.801 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_processing": 0, 2024-06-06T03:48:56.801 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_enqueued": 0, 2024-06-06T03:48:56.801 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_prioritized": 0, 2024-06-06T03:48:56.801 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_started": 0, 2024-06-06T03:48:56.802 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_completed": 0, 2024-06-06T03:48:56.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_path": 0, 2024-06-06T03:48:56.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_inode": 0, 2024-06-06T03:48:56.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T03:48:56.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_exportdir": 0, 2024-06-06T03:48:56.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_flush": 0, 2024-06-06T03:48:56.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragmentdir": 0, 2024-06-06T03:48:56.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragstats": 0, 2024-06-06T03:48:56.803 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_inodestats": 0 2024-06-06T03:48:56.803 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.803 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log": { 2024-06-06T03:48:56.803 INFO:teuthology.orchestra.run.smithi040.stdout: "evlrg": 0, 2024-06-06T03:48:56.803 INFO:teuthology.orchestra.run.smithi040.stdout: "evadd": 0, 2024-06-06T03:48:56.803 INFO:teuthology.orchestra.run.smithi040.stdout: "evex": 0, 2024-06-06T03:48:56.803 INFO:teuthology.orchestra.run.smithi040.stdout: "evtrm": 0, 2024-06-06T03:48:56.803 INFO:teuthology.orchestra.run.smithi040.stdout: "ev": 26305, 2024-06-06T03:48:56.803 INFO:teuthology.orchestra.run.smithi040.stdout: "evexg": 0, 2024-06-06T03:48:56.804 INFO:teuthology.orchestra.run.smithi040.stdout: "evexd": 0, 2024-06-06T03:48:56.804 INFO:teuthology.orchestra.run.smithi040.stdout: "segadd": 0, 2024-06-06T03:48:56.804 INFO:teuthology.orchestra.run.smithi040.stdout: "segex": 0, 2024-06-06T03:48:56.804 INFO:teuthology.orchestra.run.smithi040.stdout: "segtrm": 0, 2024-06-06T03:48:56.804 INFO:teuthology.orchestra.run.smithi040.stdout: "seg": 34, 2024-06-06T03:48:56.804 INFO:teuthology.orchestra.run.smithi040.stdout: "segmjr": 4, 2024-06-06T03:48:56.804 INFO:teuthology.orchestra.run.smithi040.stdout: "segexg": 0, 2024-06-06T03:48:56.804 INFO:teuthology.orchestra.run.smithi040.stdout: "segexd": 0, 2024-06-06T03:48:56.804 INFO:teuthology.orchestra.run.smithi040.stdout: "expos": 4194304, 2024-06-06T03:48:56.805 INFO:teuthology.orchestra.run.smithi040.stdout: "wrpos": 81072491, 2024-06-06T03:48:56.805 INFO:teuthology.orchestra.run.smithi040.stdout: "rdpos": 81071342, 2024-06-06T03:48:56.805 INFO:teuthology.orchestra.run.smithi040.stdout: "jlat": { 2024-06-06T03:48:56.805 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.805 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.805 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.805 INFO:teuthology.orchestra.run.smithi040.stdout: "replayed": 26305 2024-06-06T03:48:56.806 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.806 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mem": { 2024-06-06T03:48:56.806 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 18107, 2024-06-06T03:48:56.806 INFO:teuthology.orchestra.run.smithi040.stdout: "ino+": 34780, 2024-06-06T03:48:56.806 INFO:teuthology.orchestra.run.smithi040.stdout: "ino-": 16673, 2024-06-06T03:48:56.806 INFO:teuthology.orchestra.run.smithi040.stdout: "dir": 1439, 2024-06-06T03:48:56.806 INFO:teuthology.orchestra.run.smithi040.stdout: "dir+": 2596, 2024-06-06T03:48:56.806 INFO:teuthology.orchestra.run.smithi040.stdout: "dir-": 1157, 2024-06-06T03:48:56.807 INFO:teuthology.orchestra.run.smithi040.stdout: "dn": 18543, 2024-06-06T03:48:56.807 INFO:teuthology.orchestra.run.smithi040.stdout: "dn+": 35087, 2024-06-06T03:48:56.807 INFO:teuthology.orchestra.run.smithi040.stdout: "dn-": 16544, 2024-06-06T03:48:56.807 INFO:teuthology.orchestra.run.smithi040.stdout: "cap": 0, 2024-06-06T03:48:56.807 INFO:teuthology.orchestra.run.smithi040.stdout: "cap+": 0, 2024-06-06T03:48:56.807 INFO:teuthology.orchestra.run.smithi040.stdout: "cap-": 0, 2024-06-06T03:48:56.807 INFO:teuthology.orchestra.run.smithi040.stdout: "rss": 118432, 2024-06-06T03:48:56.807 INFO:teuthology.orchestra.run.smithi040.stdout: "heap": 215420 2024-06-06T03:48:56.807 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.808 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_server": { 2024-06-06T03:48:56.808 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_client_request": 0, 2024-06-06T03:48:56.808 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_server_request": 0, 2024-06-06T03:48:56.808 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_request": 0, 2024-06-06T03:48:56.808 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_session": 0, 2024-06-06T03:48:56.808 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_peer_request": 0, 2024-06-06T03:48:56.808 INFO:teuthology.orchestra.run.smithi040.stdout: "req_create_latency": { 2024-06-06T03:48:56.808 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.808 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.809 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.809 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getattr_latency": { 2024-06-06T03:48:56.809 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.809 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.809 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.809 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getfilelock_latency": { 2024-06-06T03:48:56.809 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.810 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.810 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.810 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.810 INFO:teuthology.orchestra.run.smithi040.stdout: "req_link_latency": { 2024-06-06T03:48:56.810 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.810 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.810 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.810 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.810 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookup_latency": { 2024-06-06T03:48:56.811 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.811 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.811 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.811 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.811 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookuphash_latency": { 2024-06-06T03:48:56.811 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.811 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.811 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.812 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupino_latency": { 2024-06-06T03:48:56.812 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.812 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.812 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.812 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupname_latency": { 2024-06-06T03:48:56.812 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.812 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.813 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.813 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupparent_latency": { 2024-06-06T03:48:56.813 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.813 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.813 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.813 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupsnap_latency": { 2024-06-06T03:48:56.814 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.814 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.814 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.814 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lssnap_latency": { 2024-06-06T03:48:56.814 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.814 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.814 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.815 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mkdir_latency": { 2024-06-06T03:48:56.815 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.815 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.816 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.816 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mknod_latency": { 2024-06-06T03:48:56.816 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.816 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.816 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.816 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mksnap_latency": { 2024-06-06T03:48:56.817 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.817 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.817 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.817 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.817 INFO:teuthology.orchestra.run.smithi040.stdout: "req_open_latency": { 2024-06-06T03:48:56.817 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.817 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.817 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.818 INFO:teuthology.orchestra.run.smithi040.stdout: "req_readdir_latency": { 2024-06-06T03:48:56.818 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.818 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.818 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.818 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rename_latency": { 2024-06-06T03:48:56.818 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.819 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.819 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.819 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.819 INFO:teuthology.orchestra.run.smithi040.stdout: "req_renamesnap_latency": { 2024-06-06T03:48:56.819 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.819 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.819 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.819 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.819 INFO:teuthology.orchestra.run.smithi040.stdout: "req_snapdiff_latency": { 2024-06-06T03:48:56.820 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.820 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.820 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.820 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmdir_latency": { 2024-06-06T03:48:56.820 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.820 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.820 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.821 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmsnap_latency": { 2024-06-06T03:48:56.821 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.821 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.821 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.821 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmxattr_latency": { 2024-06-06T03:48:56.821 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.822 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.822 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.822 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.822 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setattr_latency": { 2024-06-06T03:48:56.822 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.822 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.822 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.822 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.823 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setdirlayout_latency": { 2024-06-06T03:48:56.823 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.823 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.823 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.823 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setfilelock_latency": { 2024-06-06T03:48:56.823 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.823 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.823 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.824 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setlayout_latency": { 2024-06-06T03:48:56.824 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.824 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.824 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.824 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setxattr_latency": { 2024-06-06T03:48:56.824 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.825 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.825 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.825 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.825 INFO:teuthology.orchestra.run.smithi040.stdout: "req_symlink_latency": { 2024-06-06T03:48:56.825 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.825 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.825 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.825 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.826 INFO:teuthology.orchestra.run.smithi040.stdout: "req_unlink_latency": { 2024-06-06T03:48:56.826 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.826 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.826 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.826 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.826 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_revoke_eviction": 0, 2024-06-06T03:48:56.826 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_acquisition_throttle": 0, 2024-06-06T03:48:56.826 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getvxattr_latency": { 2024-06-06T03:48:56.827 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.827 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.827 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.827 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:56.827 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.827 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessions": { 2024-06-06T03:48:56.827 INFO:teuthology.orchestra.run.smithi040.stdout: "session_count": 1, 2024-06-06T03:48:56.827 INFO:teuthology.orchestra.run.smithi040.stdout: "session_add": 3, 2024-06-06T03:48:56.827 INFO:teuthology.orchestra.run.smithi040.stdout: "session_remove": 2, 2024-06-06T03:48:56.828 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_open": 1, 2024-06-06T03:48:56.828 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_stale": 0, 2024-06-06T03:48:56.828 INFO:teuthology.orchestra.run.smithi040.stdout: "total_load": 0, 2024-06-06T03:48:56.828 INFO:teuthology.orchestra.run.smithi040.stdout: "average_load": 0, 2024-06-06T03:48:56.828 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_session_uptime": 1820, 2024-06-06T03:48:56.828 INFO:teuthology.orchestra.run.smithi040.stdout: "mdthresh_evicted": 0 2024-06-06T03:48:56.828 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.828 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2024-06-06T03:48:56.829 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2024-06-06T03:48:56.829 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2024-06-06T03:48:56.829 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T03:48:56.829 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2024-06-06T03:48:56.829 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T03:48:56.829 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2024-06-06T03:48:56.829 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T03:48:56.829 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T03:48:56.829 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T03:48:56.830 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T03:48:56.830 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T03:48:56.830 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2024-06-06T03:48:56.830 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T03:48:56.830 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T03:48:56.830 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2024-06-06T03:48:56.830 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2024-06-06T03:48:56.830 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2024-06-06T03:48:56.831 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2024-06-06T03:48:56.831 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T03:48:56.831 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T03:48:56.831 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T03:48:56.831 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T03:48:56.831 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T03:48:56.831 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2024-06-06T03:48:56.831 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2024-06-06T03:48:56.832 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2024-06-06T03:48:56.832 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T03:48:56.832 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T03:48:56.832 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2024-06-06T03:48:56.832 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2024-06-06T03:48:56.832 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2024-06-06T03:48:56.832 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2024-06-06T03:48:56.832 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T03:48:56.832 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2024-06-06T03:48:56.833 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T03:48:56.833 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2024-06-06T03:48:56.833 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 906730, 2024-06-06T03:48:56.833 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 18524, 2024-06-06T03:48:56.833 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2024-06-06T03:48:56.833 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2024-06-06T03:48:56.833 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2024-06-06T03:48:56.833 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2024-06-06T03:48:56.834 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2024-06-06T03:48:56.834 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2024-06-06T03:48:56.834 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2024-06-06T03:48:56.834 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2024-06-06T03:48:56.834 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 7744, 2024-06-06T03:48:56.834 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 161, 2024-06-06T03:48:56.834 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T03:48:56.834 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2024-06-06T03:48:56.834 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2024-06-06T03:48:56.835 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2024-06-06T03:48:56.835 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 56633395, 2024-06-06T03:48:56.835 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 589336, 2024-06-06T03:48:56.835 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2024-06-06T03:48:56.835 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2024-06-06T03:48:56.835 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2024-06-06T03:48:56.835 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2024-06-06T03:48:56.835 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.836 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter": { 2024-06-06T03:48:56.836 INFO:teuthology.orchestra.run.smithi040.stdout: "op_active": 0, 2024-06-06T03:48:56.836 INFO:teuthology.orchestra.run.smithi040.stdout: "op_laggy": 0, 2024-06-06T03:48:56.836 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send": 2433, 2024-06-06T03:48:56.836 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send_bytes": 21, 2024-06-06T03:48:56.836 INFO:teuthology.orchestra.run.smithi040.stdout: "op_resend": 3, 2024-06-06T03:48:56.836 INFO:teuthology.orchestra.run.smithi040.stdout: "op_reply": 2433, 2024-06-06T03:48:56.836 INFO:teuthology.orchestra.run.smithi040.stdout: "op_latency": { 2024-06-06T03:48:56.837 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2433, 2024-06-06T03:48:56.837 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 5.300938237, 2024-06-06T03:48:56.837 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.002178766 2024-06-06T03:48:56.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.837 INFO:teuthology.orchestra.run.smithi040.stdout: "op_inflight": 0, 2024-06-06T03:48:56.837 INFO:teuthology.orchestra.run.smithi040.stdout: "oplen_avg": { 2024-06-06T03:48:56.837 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2433, 2024-06-06T03:48:56.837 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 2434 2024-06-06T03:48:56.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.838 INFO:teuthology.orchestra.run.smithi040.stdout: "op": 2433, 2024-06-06T03:48:56.838 INFO:teuthology.orchestra.run.smithi040.stdout: "op_r": 2433, 2024-06-06T03:48:56.838 INFO:teuthology.orchestra.run.smithi040.stdout: "op_w": 0, 2024-06-06T03:48:56.838 INFO:teuthology.orchestra.run.smithi040.stdout: "op_rmw": 0, 2024-06-06T03:48:56.838 INFO:teuthology.orchestra.run.smithi040.stdout: "op_pg": 0, 2024-06-06T03:48:56.838 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_stat": 1460, 2024-06-06T03:48:56.838 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_create": 0, 2024-06-06T03:48:56.838 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_read": 971, 2024-06-06T03:48:56.839 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_write": 0, 2024-06-06T03:48:56.839 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writefull": 0, 2024-06-06T03:48:56.839 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writesame": 0, 2024-06-06T03:48:56.839 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_append": 0, 2024-06-06T03:48:56.839 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_zero": 0, 2024-06-06T03:48:56.839 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_truncate": 0, 2024-06-06T03:48:56.839 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_delete": 0, 2024-06-06T03:48:56.839 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_mapext": 0, 2024-06-06T03:48:56.839 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_sparse_read": 0, 2024-06-06T03:48:56.840 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_clonerange": 0, 2024-06-06T03:48:56.840 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_getxattr": 1, 2024-06-06T03:48:56.840 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_setxattr": 0, 2024-06-06T03:48:56.840 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_cmpxattr": 0, 2024-06-06T03:48:56.840 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_rmxattr": 0, 2024-06-06T03:48:56.840 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_resetxattrs": 0, 2024-06-06T03:48:56.840 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_call": 0, 2024-06-06T03:48:56.840 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_watch": 0, 2024-06-06T03:48:56.841 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_notify": 0, 2024-06-06T03:48:56.841 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T03:48:56.841 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls": 0, 2024-06-06T03:48:56.841 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls_filter": 0, 2024-06-06T03:48:56.841 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_other": 0, 2024-06-06T03:48:56.841 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_active": 0, 2024-06-06T03:48:56.841 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_send": 0, 2024-06-06T03:48:56.841 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_resend": 0, 2024-06-06T03:48:56.842 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_ping": 0, 2024-06-06T03:48:56.842 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_active": 0, 2024-06-06T03:48:56.842 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_send": 0, 2024-06-06T03:48:56.842 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_resend": 0, 2024-06-06T03:48:56.842 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_active": 0, 2024-06-06T03:48:56.842 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_send": 0, 2024-06-06T03:48:56.842 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_resend": 0, 2024-06-06T03:48:56.842 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_active": 0, 2024-06-06T03:48:56.842 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_send": 0, 2024-06-06T03:48:56.843 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_resend": 0, 2024-06-06T03:48:56.843 INFO:teuthology.orchestra.run.smithi040.stdout: "command_active": 0, 2024-06-06T03:48:56.843 INFO:teuthology.orchestra.run.smithi040.stdout: "command_send": 0, 2024-06-06T03:48:56.843 INFO:teuthology.orchestra.run.smithi040.stdout: "command_resend": 0, 2024-06-06T03:48:56.843 INFO:teuthology.orchestra.run.smithi040.stdout: "map_epoch": 99, 2024-06-06T03:48:56.843 INFO:teuthology.orchestra.run.smithi040.stdout: "map_full": 0, 2024-06-06T03:48:56.843 INFO:teuthology.orchestra.run.smithi040.stdout: "map_inc": 11, 2024-06-06T03:48:56.843 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_sessions": 9, 2024-06-06T03:48:56.844 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_open": 9, 2024-06-06T03:48:56.844 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_close": 0, 2024-06-06T03:48:56.844 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_laggy": 0, 2024-06-06T03:48:56.844 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_wr": 0, 2024-06-06T03:48:56.844 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_rd": 2, 2024-06-06T03:48:56.844 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_del": 0 2024-06-06T03:48:56.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.844 INFO:teuthology.orchestra.run.smithi040.stdout: "oft": { 2024-06-06T03:48:56.845 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_objs": 0, 2024-06-06T03:48:56.845 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_kv_pairs": 0, 2024-06-06T03:48:56.845 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_updates": 0, 2024-06-06T03:48:56.845 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_removes": 0 2024-06-06T03:48:56.845 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.845 INFO:teuthology.orchestra.run.smithi040.stdout: "osd-slow-ops": { 2024-06-06T03:48:56.845 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_ops_count": 0 2024-06-06T03:48:56.845 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.845 INFO:teuthology.orchestra.run.smithi040.stdout: "purge_queue": { 2024-06-06T03:48:56.846 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops": 0, 2024-06-06T03:48:56.846 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T03:48:56.846 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing": 0, 2024-06-06T03:48:56.846 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_high_water": 0, 2024-06-06T03:48:56.846 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed_ops": 0, 2024-06-06T03:48:56.846 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed": 0, 2024-06-06T03:48:56.846 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_item_in_journal": 0 2024-06-06T03:48:56.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.847 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T03:48:56.847 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-06-06T03:48:56.847 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-06-06T03:48:56.847 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-06-06T03:48:56.847 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 2704, 2024-06-06T03:48:56.847 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 79034831, 2024-06-06T03:48:56.847 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-06-06T03:48:56.847 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 2704, 2024-06-06T03:48:56.847 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-06-06T03:48:56.848 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-06-06T03:48:56.848 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2704, 2024-06-06T03:48:56.848 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 79034831, 2024-06-06T03:48:56.848 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-06-06T03:48:56.848 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.848 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.848 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.848 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:56.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.849 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_bytes": { 2024-06-06T03:48:56.849 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-06-06T03:48:56.849 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-06-06T03:48:56.849 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-06-06T03:48:56.849 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-06-06T03:48:56.849 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-06-06T03:48:56.850 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-06-06T03:48:56.850 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-06-06T03:48:56.850 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2433, 2024-06-06T03:48:56.850 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 76889745, 2024-06-06T03:48:56.850 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2433, 2024-06-06T03:48:56.851 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 76889745, 2024-06-06T03:48:56.851 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-06-06T03:48:56.851 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.851 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.851 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.851 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:56.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.851 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_ops": { 2024-06-06T03:48:56.852 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-06-06T03:48:56.852 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 1024, 2024-06-06T03:48:56.852 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-06-06T03:48:56.852 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-06-06T03:48:56.852 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-06-06T03:48:56.852 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-06-06T03:48:56.852 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-06-06T03:48:56.852 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2433, 2024-06-06T03:48:56.852 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 2433, 2024-06-06T03:48:56.853 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2433, 2024-06-06T03:48:56.853 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 2433, 2024-06-06T03:48:56.853 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-06-06T03:48:56.853 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.853 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.853 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.853 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:56.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.854 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle": { 2024-06-06T03:48:56.854 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-06-06T03:48:56.854 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-06-06T03:48:56.854 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-06-06T03:48:56.854 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-06-06T03:48:56.854 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-06-06T03:48:56.854 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-06-06T03:48:56.854 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-06-06T03:48:56.855 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-06-06T03:48:56.855 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-06-06T03:48:56.855 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-06-06T03:48:56.855 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-06-06T03:48:56.855 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-06-06T03:48:56.855 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.855 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.855 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.856 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:56.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:56.856 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle-0x55ed6d5a26e0": { 2024-06-06T03:48:56.856 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-06-06T03:48:56.856 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-06-06T03:48:56.856 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-06-06T03:48:56.856 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-06-06T03:48:56.856 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-06-06T03:48:56.856 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-06-06T03:48:56.857 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-06-06T03:48:56.857 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-06-06T03:48:56.857 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-06-06T03:48:56.857 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-06-06T03:48:56.857 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-06-06T03:48:56.857 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-06-06T03:48:56.857 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:56.857 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:56.858 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:56.858 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:56.858 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:56.858 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:48:57.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:56 smithi131 ceph-mon[36687]: pgmap v320: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 91 KiB/s wr, 38 op/s 2024-06-06T03:48:57.285 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:56 smithi040 ceph-mon[32220]: pgmap v320: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 91 KiB/s wr, 38 op/s 2024-06-06T03:48:57.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:56 smithi104 ceph-mon[36569]: pgmap v320: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 91 KiB/s wr, 38 op/s 2024-06-06T03:48:57.635 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-06-06T03:48:57.636 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-06-06T03:48:57.636 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-06-06T03:48:57.636 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-06-06T03:48:57.636 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T03:48:57.636 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-06-06T03:48:57.636 DEBUG:tasks.check_counter:Getting stats from d 2024-06-06T03:48:57.637 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-06T03:48:57.897 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:48:59.071 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:58 smithi040 ceph-mon[32220]: pgmap v321: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 184 KiB/s wr, 25 op/s 2024-06-06T03:48:59.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:58 smithi131 ceph-mon[36687]: pgmap v321: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 184 KiB/s wr, 25 op/s 2024-06-06T03:48:59.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:58 smithi104 ceph-mon[36569]: pgmap v321: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 184 KiB/s wr, 25 op/s 2024-06-06T03:48:59.719 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:48:59.719 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T03:48:59.719 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 4, 2024-06-06T03:48:59.719 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 4, 2024-06-06T03:48:59.719 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 2992, 2024-06-06T03:48:59.720 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 448, 2024-06-06T03:48:59.720 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 4, 2024-06-06T03:48:59.720 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 0, 2024-06-06T03:48:59.720 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.001770623, 2024-06-06T03:48:59.720 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.000032813, 2024-06-06T03:48:59.720 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.000727126, 2024-06-06T03:48:59.720 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T03:48:59.720 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:48:59.720 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 3, 2024-06-06T03:48:59.721 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000771657, 2024-06-06T03:48:59.721 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000257219 2024-06-06T03:48:59.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:59.721 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:48:59.721 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:59.721 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:59.721 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:59.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:59.722 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-06-06T03:48:59.722 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T03:48:59.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:59.722 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T03:48:59.722 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 243, 2024-06-06T03:48:59.722 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 232, 2024-06-06T03:48:59.722 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 1586528, 2024-06-06T03:48:59.723 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 142784, 2024-06-06T03:48:59.723 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 2, 2024-06-06T03:48:59.723 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 1, 2024-06-06T03:48:59.723 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.045020306, 2024-06-06T03:48:59.723 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.014244288, 2024-06-06T03:48:59.723 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.034809801, 2024-06-06T03:48:59.723 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.004484472, 2024-06-06T03:48:59.723 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:48:59.724 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 231, 2024-06-06T03:48:59.724 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.010410766, 2024-06-06T03:48:59.724 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000045068 2024-06-06T03:48:59.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:59.724 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:48:59.724 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:59.724 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:59.724 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:59.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:59.725 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 1586528, 2024-06-06T03:48:59.725 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 142784 2024-06-06T03:48:59.725 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:59.725 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T03:48:59.725 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 2, 2024-06-06T03:48:59.725 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 178, 2024-06-06T03:48:59.725 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 320, 2024-06-06T03:48:59.725 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 187024, 2024-06-06T03:48:59.725 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 5, 2024-06-06T03:48:59.725 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 3, 2024-06-06T03:48:59.726 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.014647717, 2024-06-06T03:48:59.726 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.010248556, 2024-06-06T03:48:59.726 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.001691869, 2024-06-06T03:48:59.726 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T03:48:59.726 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:48:59.726 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 175, 2024-06-06T03:48:59.726 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.018394904, 2024-06-06T03:48:59.726 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000105113 2024-06-06T03:48:59.727 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:59.727 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:48:59.727 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:59.727 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:59.727 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:59.727 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:59.727 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-06T03:48:59.727 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 187024 2024-06-06T03:48:59.727 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:59.728 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2024-06-06T03:48:59.728 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 0, 2024-06-06T03:48:59.728 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2024-06-06T03:48:59.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:59.728 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2024-06-06T03:48:59.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2024-06-06T03:48:59.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2024-06-06T03:48:59.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T03:48:59.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2024-06-06T03:48:59.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T03:48:59.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2024-06-06T03:48:59.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T03:48:59.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T03:48:59.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T03:48:59.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T03:48:59.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T03:48:59.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2024-06-06T03:48:59.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T03:48:59.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T03:48:59.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2024-06-06T03:48:59.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2024-06-06T03:48:59.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2024-06-06T03:48:59.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2024-06-06T03:48:59.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T03:48:59.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T03:48:59.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T03:48:59.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T03:48:59.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T03:48:59.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2024-06-06T03:48:59.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2024-06-06T03:48:59.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2024-06-06T03:48:59.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T03:48:59.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T03:48:59.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2024-06-06T03:48:59.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2024-06-06T03:48:59.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2024-06-06T03:48:59.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2024-06-06T03:48:59.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T03:48:59.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2024-06-06T03:48:59.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T03:48:59.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2024-06-06T03:48:59.732 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 148771, 2024-06-06T03:48:59.733 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 32, 2024-06-06T03:48:59.733 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2024-06-06T03:48:59.733 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2024-06-06T03:48:59.733 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2024-06-06T03:48:59.733 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2024-06-06T03:48:59.733 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2024-06-06T03:48:59.733 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2024-06-06T03:48:59.733 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2024-06-06T03:48:59.733 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2024-06-06T03:48:59.734 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 0, 2024-06-06T03:48:59.734 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 0, 2024-06-06T03:48:59.734 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T03:48:59.734 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2024-06-06T03:48:59.734 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2024-06-06T03:48:59.734 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2024-06-06T03:48:59.734 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 928, 2024-06-06T03:48:59.735 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 2, 2024-06-06T03:48:59.735 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2024-06-06T03:48:59.735 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2024-06-06T03:48:59.735 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2024-06-06T03:48:59.735 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2024-06-06T03:48:59.735 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:48:59.735 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T03:48:59.735 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-06-06T03:48:59.735 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-06-06T03:48:59.736 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-06-06T03:48:59.736 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 245, 2024-06-06T03:48:59.736 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 1552250, 2024-06-06T03:48:59.736 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-06-06T03:48:59.736 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 245, 2024-06-06T03:48:59.736 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-06-06T03:48:59.736 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-06-06T03:48:59.736 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 245, 2024-06-06T03:48:59.737 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 1552250, 2024-06-06T03:48:59.737 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-06-06T03:48:59.737 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:48:59.737 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:48:59.737 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:48:59.737 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:59.737 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:48:59.737 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:49:00.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:59 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:00.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:48:59 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:49:00.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:59 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:00.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:48:59 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:49:00.340 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:59 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:00.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:48:59 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T03:49:00.558 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-06-06T03:49:00.558 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-06-06T03:49:00.558 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-06-06T03:49:00.559 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-06-06T03:49:00.559 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-06-06T03:49:00.559 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-06-06T03:49:00.559 DEBUG:tasks.check_counter:Getting stats from g 2024-06-06T03:49:00.559 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-06T03:49:00.834 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:49:01.505 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:01 smithi040 ceph-mon[32220]: pgmap v322: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 642 KiB/s wr, 47 op/s 2024-06-06T03:49:01.506 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:01 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.506 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:01 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.506 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:01 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.506 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:01 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.506 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:01 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.506 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:01 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:01 smithi131 ceph-mon[36687]: pgmap v322: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 642 KiB/s wr, 47 op/s 2024-06-06T03:49:01.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:01 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:01 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:01 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:01 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:01 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:01 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:01 smithi104 ceph-mon[36569]: pgmap v322: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 642 KiB/s wr, 47 op/s 2024-06-06T03:49:01.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:01 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:01 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:01 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:01 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:01 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:01.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:01 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:02.585 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:49:02.585 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T03:49:02.586 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1313, 2024-06-06T03:49:02.586 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1283, 2024-06-06T03:49:02.586 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 23027901, 2024-06-06T03:49:02.586 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 473206, 2024-06-06T03:49:02.586 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 7, 2024-06-06T03:49:02.586 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 3, 2024-06-06T03:49:02.586 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.161195133, 2024-06-06T03:49:02.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.056225310, 2024-06-06T03:49:02.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.071821063, 2024-06-06T03:49:02.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.034245849, 2024-06-06T03:49:02.587 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:02.587 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1283, 2024-06-06T03:49:02.587 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.050881062, 2024-06-06T03:49:02.587 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000039657 2024-06-06T03:49:02.587 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.588 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:02.588 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.588 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.588 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.588 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 1725696, 2024-06-06T03:49:02.588 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 143440 2024-06-06T03:49:02.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.589 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T03:49:02.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 997, 2024-06-06T03:49:02.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1169, 2024-06-06T03:49:02.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 29642797, 2024-06-06T03:49:02.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 779907, 2024-06-06T03:49:02.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 8, 2024-06-06T03:49:02.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 6, 2024-06-06T03:49:02.589 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.091296680, 2024-06-06T03:49:02.590 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.026386236, 2024-06-06T03:49:02.590 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.036247004, 2024-06-06T03:49:02.590 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.021993048, 2024-06-06T03:49:02.590 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:02.590 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1166, 2024-06-06T03:49:02.590 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.036227773, 2024-06-06T03:49:02.590 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000031070 2024-06-06T03:49:02.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.591 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:02.591 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.591 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.591 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.591 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-06T03:49:02.591 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 463440 2024-06-06T03:49:02.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.592 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T03:49:02.592 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 483, 2024-06-06T03:49:02.592 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 485, 2024-06-06T03:49:02.592 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 26463105, 2024-06-06T03:49:02.592 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 153606, 2024-06-06T03:49:02.592 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 6, 2024-06-06T03:49:02.592 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 5, 2024-06-06T03:49:02.592 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.057464569, 2024-06-06T03:49:02.593 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.009406945, 2024-06-06T03:49:02.593 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.032127769, 2024-06-06T03:49:02.593 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.013200587, 2024-06-06T03:49:02.593 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:02.593 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 483, 2024-06-06T03:49:02.593 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.016866590, 2024-06-06T03:49:02.593 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000034920 2024-06-06T03:49:02.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.593 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:02.594 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.594 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.594 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.594 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T03:49:02.594 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T03:49:02.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.595 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2024-06-06T03:49:02.595 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 1, 2024-06-06T03:49:02.595 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2024-06-06T03:49:02.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.595 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-MDSRank": { 2024-06-06T03:49:02.595 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-06-06T03:49:02.595 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-06-06T03:49:02.595 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 4207, 2024-06-06T03:49:02.595 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.101925380, 2024-06-06T03:49:02.596 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000024227 2024-06-06T03:49:02.596 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:49:02.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.596 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-PurgeQueue": { 2024-06-06T03:49:02.596 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-06-06T03:49:02.596 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-06-06T03:49:02.596 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.596 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.597 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.597 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:49:02.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.597 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-06-06T03:49:02.597 INFO:teuthology.orchestra.run.smithi040.stdout: "request": 0, 2024-06-06T03:49:02.597 INFO:teuthology.orchestra.run.smithi040.stdout: "reply": 0, 2024-06-06T03:49:02.597 INFO:teuthology.orchestra.run.smithi040.stdout: "reply_latency": { 2024-06-06T03:49:02.597 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.598 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.598 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.598 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_reply": 0, 2024-06-06T03:49:02.598 INFO:teuthology.orchestra.run.smithi040.stdout: "forward": 0, 2024-06-06T03:49:02.598 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_complete": 0, 2024-06-06T03:49:02.598 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_keys": 0, 2024-06-06T03:49:02.599 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_commit": 0, 2024-06-06T03:49:02.599 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_split": 0, 2024-06-06T03:49:02.599 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_merge": 0, 2024-06-06T03:49:02.599 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes": 17384, 2024-06-06T03:49:02.599 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_top": 47, 2024-06-06T03:49:02.599 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_bottom": 17337, 2024-06-06T03:49:02.599 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pin_tail": 0, 2024-06-06T03:49:02.599 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pinned": 17317, 2024-06-06T03:49:02.600 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_expired": 0, 2024-06-06T03:49:02.600 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_with_caps": 0, 2024-06-06T03:49:02.600 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": 0, 2024-06-06T03:49:02.600 INFO:teuthology.orchestra.run.smithi040.stdout: "subtrees": 91, 2024-06-06T03:49:02.600 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse": 0, 2024-06-06T03:49:02.600 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_hit": 0, 2024-06-06T03:49:02.600 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_forward": 0, 2024-06-06T03:49:02.600 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_discover": 0, 2024-06-06T03:49:02.601 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_dir_fetch": 0, 2024-06-06T03:49:02.601 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_remote_ino": 0, 2024-06-06T03:49:02.601 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_lock": 0, 2024-06-06T03:49:02.601 INFO:teuthology.orchestra.run.smithi040.stdout: "load_cent": 0, 2024-06-06T03:49:02.601 INFO:teuthology.orchestra.run.smithi040.stdout: "q": 0, 2024-06-06T03:49:02.601 INFO:teuthology.orchestra.run.smithi040.stdout: "exported": 0, 2024-06-06T03:49:02.601 INFO:teuthology.orchestra.run.smithi040.stdout: "exported_inodes": 0, 2024-06-06T03:49:02.601 INFO:teuthology.orchestra.run.smithi040.stdout: "imported": 0, 2024-06-06T03:49:02.601 INFO:teuthology.orchestra.run.smithi040.stdout: "imported_inodes": 0, 2024-06-06T03:49:02.602 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_dir_fetch": 0, 2024-06-06T03:49:02.602 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_backtrace_fetch": 0, 2024-06-06T03:49:02.602 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_peer_discover": 0, 2024-06-06T03:49:02.602 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rfiles": 0, 2024-06-06T03:49:02.602 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rbytes": 0, 2024-06-06T03:49:02.602 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rsnaps": 0, 2024-06-06T03:49:02.602 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T03:49:02.602 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_set_tag": 0, 2024-06-06T03:49:02.603 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T03:49:02.603 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_inotable_repaired": 0, 2024-06-06T03:49:02.603 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_inodes": 0, 2024-06-06T03:49:02.603 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T03:49:02.603 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T03:49:02.603 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_file_inodes": 0, 2024-06-06T03:49:02.603 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_inode_file_caps": 0, 2024-06-06T03:49:02.603 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T03:49:02.604 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_grant": 0, 2024-06-06T03:49:02.604 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T03:49:02.604 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T03:49:02.604 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T03:49:02.604 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps": 0, 2024-06-06T03:49:02.604 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps_dirty": 0, 2024-06-06T03:49:02.604 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_cap_release": 0, 2024-06-06T03:49:02.605 INFO:teuthology.orchestra.run.smithi040.stdout: "process_request_cap_release": 0 2024-06-06T03:49:02.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.605 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache": { 2024-06-06T03:49:02.605 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update": 0, 2024-06-06T03:49:02.605 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update_receipt": 0, 2024-06-06T03:49:02.605 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_try_discover": 0, 2024-06-06T03:49:02.605 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_send_discover": 0, 2024-06-06T03:49:02.605 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_handle_discover": 0, 2024-06-06T03:49:02.606 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays": 0, 2024-06-06T03:49:02.606 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_delayed": 0, 2024-06-06T03:49:02.606 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_enqueuing": 0, 2024-06-06T03:49:02.606 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_created": 0, 2024-06-06T03:49:02.606 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_enqueued": 0, 2024-06-06T03:49:02.606 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_reintegrated": 0, 2024-06-06T03:49:02.606 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_migrated": 0, 2024-06-06T03:49:02.606 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_processing": 0, 2024-06-06T03:49:02.607 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_enqueued": 0, 2024-06-06T03:49:02.607 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_prioritized": 0, 2024-06-06T03:49:02.607 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_started": 0, 2024-06-06T03:49:02.607 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_completed": 0, 2024-06-06T03:49:02.607 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_path": 0, 2024-06-06T03:49:02.607 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_inode": 0, 2024-06-06T03:49:02.607 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T03:49:02.607 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_exportdir": 0, 2024-06-06T03:49:02.608 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_flush": 0, 2024-06-06T03:49:02.608 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragmentdir": 0, 2024-06-06T03:49:02.608 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragstats": 0, 2024-06-06T03:49:02.608 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_inodestats": 0 2024-06-06T03:49:02.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.608 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log": { 2024-06-06T03:49:02.608 INFO:teuthology.orchestra.run.smithi040.stdout: "evlrg": 0, 2024-06-06T03:49:02.608 INFO:teuthology.orchestra.run.smithi040.stdout: "evadd": 0, 2024-06-06T03:49:02.609 INFO:teuthology.orchestra.run.smithi040.stdout: "evex": 0, 2024-06-06T03:49:02.609 INFO:teuthology.orchestra.run.smithi040.stdout: "evtrm": 0, 2024-06-06T03:49:02.609 INFO:teuthology.orchestra.run.smithi040.stdout: "ev": 26622, 2024-06-06T03:49:02.609 INFO:teuthology.orchestra.run.smithi040.stdout: "evexg": 0, 2024-06-06T03:49:02.609 INFO:teuthology.orchestra.run.smithi040.stdout: "evexd": 0, 2024-06-06T03:49:02.609 INFO:teuthology.orchestra.run.smithi040.stdout: "segadd": 0, 2024-06-06T03:49:02.609 INFO:teuthology.orchestra.run.smithi040.stdout: "segex": 0, 2024-06-06T03:49:02.609 INFO:teuthology.orchestra.run.smithi040.stdout: "segtrm": 0, 2024-06-06T03:49:02.610 INFO:teuthology.orchestra.run.smithi040.stdout: "seg": 36, 2024-06-06T03:49:02.610 INFO:teuthology.orchestra.run.smithi040.stdout: "segmjr": 4, 2024-06-06T03:49:02.610 INFO:teuthology.orchestra.run.smithi040.stdout: "segexg": 0, 2024-06-06T03:49:02.610 INFO:teuthology.orchestra.run.smithi040.stdout: "segexd": 0, 2024-06-06T03:49:02.610 INFO:teuthology.orchestra.run.smithi040.stdout: "expos": 4194304, 2024-06-06T03:49:02.610 INFO:teuthology.orchestra.run.smithi040.stdout: "wrpos": 80860570, 2024-06-06T03:49:02.610 INFO:teuthology.orchestra.run.smithi040.stdout: "rdpos": 80859971, 2024-06-06T03:49:02.610 INFO:teuthology.orchestra.run.smithi040.stdout: "jlat": { 2024-06-06T03:49:02.610 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.611 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.611 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.611 INFO:teuthology.orchestra.run.smithi040.stdout: "replayed": 26622 2024-06-06T03:49:02.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.611 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mem": { 2024-06-06T03:49:02.611 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 16933, 2024-06-06T03:49:02.611 INFO:teuthology.orchestra.run.smithi040.stdout: "ino+": 29035, 2024-06-06T03:49:02.612 INFO:teuthology.orchestra.run.smithi040.stdout: "ino-": 12102, 2024-06-06T03:49:02.612 INFO:teuthology.orchestra.run.smithi040.stdout: "dir": 1020, 2024-06-06T03:49:02.612 INFO:teuthology.orchestra.run.smithi040.stdout: "dir+": 1933, 2024-06-06T03:49:02.612 INFO:teuthology.orchestra.run.smithi040.stdout: "dir-": 913, 2024-06-06T03:49:02.612 INFO:teuthology.orchestra.run.smithi040.stdout: "dn": 17384, 2024-06-06T03:49:02.612 INFO:teuthology.orchestra.run.smithi040.stdout: "dn+": 29314, 2024-06-06T03:49:02.612 INFO:teuthology.orchestra.run.smithi040.stdout: "dn-": 11930, 2024-06-06T03:49:02.612 INFO:teuthology.orchestra.run.smithi040.stdout: "cap": 0, 2024-06-06T03:49:02.613 INFO:teuthology.orchestra.run.smithi040.stdout: "cap+": 0, 2024-06-06T03:49:02.613 INFO:teuthology.orchestra.run.smithi040.stdout: "cap-": 0, 2024-06-06T03:49:02.613 INFO:teuthology.orchestra.run.smithi040.stdout: "rss": 121960, 2024-06-06T03:49:02.613 INFO:teuthology.orchestra.run.smithi040.stdout: "heap": 215420 2024-06-06T03:49:02.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.613 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_server": { 2024-06-06T03:49:02.613 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_client_request": 0, 2024-06-06T03:49:02.613 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_server_request": 0, 2024-06-06T03:49:02.614 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_request": 0, 2024-06-06T03:49:02.614 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_session": 0, 2024-06-06T03:49:02.614 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_peer_request": 0, 2024-06-06T03:49:02.614 INFO:teuthology.orchestra.run.smithi040.stdout: "req_create_latency": { 2024-06-06T03:49:02.614 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.614 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.614 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.615 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getattr_latency": { 2024-06-06T03:49:02.615 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.615 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.615 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.615 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getfilelock_latency": { 2024-06-06T03:49:02.615 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.615 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.616 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.616 INFO:teuthology.orchestra.run.smithi040.stdout: "req_link_latency": { 2024-06-06T03:49:02.616 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.616 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.616 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.616 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookup_latency": { 2024-06-06T03:49:02.617 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.617 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.617 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.617 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookuphash_latency": { 2024-06-06T03:49:02.617 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.617 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.617 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.618 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupino_latency": { 2024-06-06T03:49:02.618 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.618 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.618 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.618 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupname_latency": { 2024-06-06T03:49:02.619 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.619 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.619 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.619 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupparent_latency": { 2024-06-06T03:49:02.619 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.619 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.619 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.620 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupsnap_latency": { 2024-06-06T03:49:02.620 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.620 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.620 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.620 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lssnap_latency": { 2024-06-06T03:49:02.620 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.620 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.621 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.621 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mkdir_latency": { 2024-06-06T03:49:02.622 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.622 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.622 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.622 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mknod_latency": { 2024-06-06T03:49:02.622 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.622 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.623 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.623 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mksnap_latency": { 2024-06-06T03:49:02.623 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.623 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.623 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.623 INFO:teuthology.orchestra.run.smithi040.stdout: "req_open_latency": { 2024-06-06T03:49:02.623 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.624 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.624 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.624 INFO:teuthology.orchestra.run.smithi040.stdout: "req_readdir_latency": { 2024-06-06T03:49:02.624 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.624 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.624 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.625 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rename_latency": { 2024-06-06T03:49:02.625 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.625 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.625 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.625 INFO:teuthology.orchestra.run.smithi040.stdout: "req_renamesnap_latency": { 2024-06-06T03:49:02.625 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.625 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.625 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.626 INFO:teuthology.orchestra.run.smithi040.stdout: "req_snapdiff_latency": { 2024-06-06T03:49:02.626 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.626 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.626 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.626 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmdir_latency": { 2024-06-06T03:49:02.626 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.627 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.627 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.627 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmsnap_latency": { 2024-06-06T03:49:02.627 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.627 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.627 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.628 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmxattr_latency": { 2024-06-06T03:49:02.628 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.628 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.628 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.628 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setattr_latency": { 2024-06-06T03:49:02.628 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.628 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.628 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.629 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setdirlayout_latency": { 2024-06-06T03:49:02.629 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.629 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.629 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.629 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setfilelock_latency": { 2024-06-06T03:49:02.629 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.630 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.630 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.630 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setlayout_latency": { 2024-06-06T03:49:02.630 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.630 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.630 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.631 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setxattr_latency": { 2024-06-06T03:49:02.631 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.631 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.631 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.631 INFO:teuthology.orchestra.run.smithi040.stdout: "req_symlink_latency": { 2024-06-06T03:49:02.631 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.631 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "req_unlink_latency": { 2024-06-06T03:49:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_revoke_eviction": 0, 2024-06-06T03:49:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_acquisition_throttle": 0, 2024-06-06T03:49:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getvxattr_latency": { 2024-06-06T03:49:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:49:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessions": { 2024-06-06T03:49:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "session_count": 1, 2024-06-06T03:49:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "session_add": 2, 2024-06-06T03:49:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "session_remove": 1, 2024-06-06T03:49:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_open": 1, 2024-06-06T03:49:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_stale": 0, 2024-06-06T03:49:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "total_load": 0, 2024-06-06T03:49:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "average_load": 0, 2024-06-06T03:49:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_session_uptime": 1821, 2024-06-06T03:49:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "mdthresh_evicted": 0 2024-06-06T03:49:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2024-06-06T03:49:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2024-06-06T03:49:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2024-06-06T03:49:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T03:49:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2024-06-06T03:49:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T03:49:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2024-06-06T03:49:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T03:49:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T03:49:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T03:49:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T03:49:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T03:49:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2024-06-06T03:49:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T03:49:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T03:49:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2024-06-06T03:49:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2024-06-06T03:49:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2024-06-06T03:49:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2024-06-06T03:49:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T03:49:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T03:49:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T03:49:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T03:49:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T03:49:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2024-06-06T03:49:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2024-06-06T03:49:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2024-06-06T03:49:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T03:49:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T03:49:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2024-06-06T03:49:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2024-06-06T03:49:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2024-06-06T03:49:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2024-06-06T03:49:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T03:49:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2024-06-06T03:49:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T03:49:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2024-06-06T03:49:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 881553, 2024-06-06T03:49:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 17628, 2024-06-06T03:49:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2024-06-06T03:49:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2024-06-06T03:49:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2024-06-06T03:49:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2024-06-06T03:49:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2024-06-06T03:49:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2024-06-06T03:49:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2024-06-06T03:49:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2024-06-06T03:49:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 7744, 2024-06-06T03:49:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 161, 2024-06-06T03:49:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T03:49:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2024-06-06T03:49:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2024-06-06T03:49:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2024-06-06T03:49:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 52849106, 2024-06-06T03:49:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 575362, 2024-06-06T03:49:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2024-06-06T03:49:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2024-06-06T03:49:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2024-06-06T03:49:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2024-06-06T03:49:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter": { 2024-06-06T03:49:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "op_active": 0, 2024-06-06T03:49:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "op_laggy": 0, 2024-06-06T03:49:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send": 2515, 2024-06-06T03:49:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send_bytes": 21, 2024-06-06T03:49:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "op_resend": 3, 2024-06-06T03:49:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "op_reply": 2515, 2024-06-06T03:49:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "op_latency": { 2024-06-06T03:49:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2515, 2024-06-06T03:49:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 6.180927651, 2024-06-06T03:49:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.002457625 2024-06-06T03:49:02.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.644 INFO:teuthology.orchestra.run.smithi040.stdout: "op_inflight": 0, 2024-06-06T03:49:02.644 INFO:teuthology.orchestra.run.smithi040.stdout: "oplen_avg": { 2024-06-06T03:49:02.644 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2515, 2024-06-06T03:49:02.644 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 2516 2024-06-06T03:49:02.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.644 INFO:teuthology.orchestra.run.smithi040.stdout: "op": 2515, 2024-06-06T03:49:02.644 INFO:teuthology.orchestra.run.smithi040.stdout: "op_r": 2515, 2024-06-06T03:49:02.645 INFO:teuthology.orchestra.run.smithi040.stdout: "op_w": 0, 2024-06-06T03:49:02.645 INFO:teuthology.orchestra.run.smithi040.stdout: "op_rmw": 0, 2024-06-06T03:49:02.645 INFO:teuthology.orchestra.run.smithi040.stdout: "op_pg": 0, 2024-06-06T03:49:02.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_stat": 1513, 2024-06-06T03:49:02.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_create": 0, 2024-06-06T03:49:02.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_read": 1000, 2024-06-06T03:49:02.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_write": 0, 2024-06-06T03:49:02.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writefull": 0, 2024-06-06T03:49:02.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writesame": 0, 2024-06-06T03:49:02.646 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_append": 0, 2024-06-06T03:49:02.646 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_zero": 0, 2024-06-06T03:49:02.646 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_truncate": 0, 2024-06-06T03:49:02.646 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_delete": 0, 2024-06-06T03:49:02.646 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_mapext": 0, 2024-06-06T03:49:02.646 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_sparse_read": 0, 2024-06-06T03:49:02.646 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_clonerange": 0, 2024-06-06T03:49:02.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_getxattr": 1, 2024-06-06T03:49:02.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_setxattr": 0, 2024-06-06T03:49:02.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_cmpxattr": 0, 2024-06-06T03:49:02.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_rmxattr": 0, 2024-06-06T03:49:02.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_resetxattrs": 0, 2024-06-06T03:49:02.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_call": 0, 2024-06-06T03:49:02.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_watch": 0, 2024-06-06T03:49:02.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_notify": 0, 2024-06-06T03:49:02.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T03:49:02.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls": 0, 2024-06-06T03:49:02.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls_filter": 0, 2024-06-06T03:49:02.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_other": 0, 2024-06-06T03:49:02.648 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_active": 0, 2024-06-06T03:49:02.648 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_send": 0, 2024-06-06T03:49:02.648 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_resend": 0, 2024-06-06T03:49:02.648 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_ping": 0, 2024-06-06T03:49:02.649 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_active": 0, 2024-06-06T03:49:02.649 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_send": 0, 2024-06-06T03:49:02.649 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_resend": 0, 2024-06-06T03:49:02.649 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_active": 0, 2024-06-06T03:49:02.649 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_send": 0, 2024-06-06T03:49:02.649 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_resend": 0, 2024-06-06T03:49:02.649 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_active": 0, 2024-06-06T03:49:02.649 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_send": 0, 2024-06-06T03:49:02.649 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_resend": 0, 2024-06-06T03:49:02.650 INFO:teuthology.orchestra.run.smithi040.stdout: "command_active": 0, 2024-06-06T03:49:02.650 INFO:teuthology.orchestra.run.smithi040.stdout: "command_send": 0, 2024-06-06T03:49:02.650 INFO:teuthology.orchestra.run.smithi040.stdout: "command_resend": 0, 2024-06-06T03:49:02.650 INFO:teuthology.orchestra.run.smithi040.stdout: "map_epoch": 99, 2024-06-06T03:49:02.650 INFO:teuthology.orchestra.run.smithi040.stdout: "map_full": 0, 2024-06-06T03:49:02.650 INFO:teuthology.orchestra.run.smithi040.stdout: "map_inc": 11, 2024-06-06T03:49:02.650 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_sessions": 10, 2024-06-06T03:49:02.651 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_open": 10, 2024-06-06T03:49:02.651 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_close": 0, 2024-06-06T03:49:02.651 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_laggy": 0, 2024-06-06T03:49:02.651 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_wr": 0, 2024-06-06T03:49:02.651 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_rd": 2, 2024-06-06T03:49:02.651 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_del": 0 2024-06-06T03:49:02.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.651 INFO:teuthology.orchestra.run.smithi040.stdout: "oft": { 2024-06-06T03:49:02.651 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_objs": 0, 2024-06-06T03:49:02.652 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_kv_pairs": 0, 2024-06-06T03:49:02.652 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_updates": 0, 2024-06-06T03:49:02.652 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_removes": 0 2024-06-06T03:49:02.652 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.652 INFO:teuthology.orchestra.run.smithi040.stdout: "osd-slow-ops": { 2024-06-06T03:49:02.652 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_ops_count": 0 2024-06-06T03:49:02.652 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.652 INFO:teuthology.orchestra.run.smithi040.stdout: "purge_queue": { 2024-06-06T03:49:02.653 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops": 0, 2024-06-06T03:49:02.653 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T03:49:02.653 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing": 0, 2024-06-06T03:49:02.653 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_high_water": 0, 2024-06-06T03:49:02.653 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed_ops": 0, 2024-06-06T03:49:02.653 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed": 0, 2024-06-06T03:49:02.653 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_item_in_journal": 0 2024-06-06T03:49:02.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.654 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T03:49:02.654 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-06-06T03:49:02.654 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-06-06T03:49:02.654 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-06-06T03:49:02.654 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 2789, 2024-06-06T03:49:02.654 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 78865915, 2024-06-06T03:49:02.654 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:02.654 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 2789, 2024-06-06T03:49:02.655 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-06-06T03:49:02.655 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-06-06T03:49:02.655 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2789, 2024-06-06T03:49:02.655 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 78865915, 2024-06-06T03:49:02.655 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-06-06T03:49:02.655 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.655 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.655 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.656 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:49:02.656 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.656 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_bytes": { 2024-06-06T03:49:02.656 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-06-06T03:49:02.656 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-06-06T03:49:02.656 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-06-06T03:49:02.656 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-06-06T03:49:02.656 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-06-06T03:49:02.657 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:02.657 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:02.657 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2515, 2024-06-06T03:49:02.657 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 76701865, 2024-06-06T03:49:02.657 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2515, 2024-06-06T03:49:02.657 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 76701865, 2024-06-06T03:49:02.657 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-06-06T03:49:02.657 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.658 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.658 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.658 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:49:02.658 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.658 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_ops": { 2024-06-06T03:49:02.658 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-06-06T03:49:02.658 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 1024, 2024-06-06T03:49:02.659 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-06-06T03:49:02.659 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-06-06T03:49:02.659 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-06-06T03:49:02.659 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:02.659 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:02.659 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2515, 2024-06-06T03:49:02.659 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 2515, 2024-06-06T03:49:02.659 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2515, 2024-06-06T03:49:02.660 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 2515, 2024-06-06T03:49:02.660 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-06-06T03:49:02.660 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.660 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.660 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.660 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:49:02.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.660 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle": { 2024-06-06T03:49:02.660 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-06-06T03:49:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-06-06T03:49:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-06-06T03:49:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-06-06T03:49:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-06-06T03:49:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-06-06T03:49:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-06-06T03:49:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-06-06T03:49:02.662 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-06-06T03:49:02.662 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-06-06T03:49:02.662 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle-0x5579aae3c6e0": { 2024-06-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-06-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-06-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-06-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-06-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-06-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-06-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-06-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-06-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-06-06T03:49:02.668 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-06-06T03:49:02.668 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:02.668 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:02.668 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:02.668 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:49:02.668 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:49:02.668 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:49:03.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:02 smithi131 ceph-mon[36687]: pgmap v323: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 635 KiB/s wr, 46 op/s 2024-06-06T03:49:03.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:02 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:03.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:02 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:03.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:02 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:03.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:02 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:03.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:02 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:03.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:02 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:03.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:02 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:03.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:02 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:03.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:02 smithi040 ceph-mon[32220]: pgmap v323: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 635 KiB/s wr, 46 op/s 2024-06-06T03:49:03.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:02 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:03.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:02 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:03.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:02 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:03.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:02 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:03.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:02 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:03.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:02 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:03.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:02 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:03.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:02 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:03.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:02 smithi104 ceph-mon[36569]: pgmap v323: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 635 KiB/s wr, 46 op/s 2024-06-06T03:49:03.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:02 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:03.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:02 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:03.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:02 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:03.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:02 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:03.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:02 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:03.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:02 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:03.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:02 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:03.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:02 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:03.399 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-06-06T03:49:03.399 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-06-06T03:49:03.399 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-06-06T03:49:03.399 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-06-06T03:49:03.399 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T03:49:03.399 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-06-06T03:49:03.399 DEBUG:tasks.check_counter:Getting stats from j 2024-06-06T03:49:03.400 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-06T03:49:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:49:04.246 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:03 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi131 to 176.2M 2024-06-06T03:49:04.247 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:03 smithi040 ceph-mon[32220]: Unable to set osd_memory_target on smithi131 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-06T03:49:04.247 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:03 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:04.247 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:03 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:04.247 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:03 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:04.247 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:03 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:04.247 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:03 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-06-06T03:49:04.247 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:03 smithi040 ceph-mon[32220]: Unable to set osd_memory_target on smithi104 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-06-06T03:49:04.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:03 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi131 to 176.2M 2024-06-06T03:49:04.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:03 smithi131 ceph-mon[36687]: Unable to set osd_memory_target on smithi131 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-06T03:49:04.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:03 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:04.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:03 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:04.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:03 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:04.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:03 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:04.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:03 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-06-06T03:49:04.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:03 smithi131 ceph-mon[36687]: Unable to set osd_memory_target on smithi104 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-06-06T03:49:04.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:03 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi131 to 176.2M 2024-06-06T03:49:04.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:03 smithi104 ceph-mon[36569]: Unable to set osd_memory_target on smithi131 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-06T03:49:04.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:03 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:04.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:03 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:04.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:03 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:04.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:03 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:04.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:03 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-06-06T03:49:04.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:03 smithi104 ceph-mon[36569]: Unable to set osd_memory_target on smithi104 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-06-06T03:49:05.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:04 smithi131 ceph-mon[36687]: pgmap v324: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 634 KiB/s wr, 44 op/s 2024-06-06T03:49:05.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:04 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:04 smithi040 ceph-mon[32220]: pgmap v324: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 634 KiB/s wr, 44 op/s 2024-06-06T03:49:05.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:04 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:05.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:04 smithi104 ceph-mon[36569]: pgmap v324: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 634 KiB/s wr, 44 op/s 2024-06-06T03:49:05.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:04 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:06.009 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-06-06T03:49:06.010 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T03:49:06.010 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 4, 2024-06-06T03:49:06.010 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 4, 2024-06-06T03:49:06.010 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 2992, 2024-06-06T03:49:06.010 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 448, 2024-06-06T03:49:06.010 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 4, 2024-06-06T03:49:06.010 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 0, 2024-06-06T03:49:06.010 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.001738034, 2024-06-06T03:49:06.011 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.000039872, 2024-06-06T03:49:06.011 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.000679122, 2024-06-06T03:49:06.011 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T03:49:06.011 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:06.011 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 3, 2024-06-06T03:49:06.011 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000766246, 2024-06-06T03:49:06.011 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000255415 2024-06-06T03:49:06.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:06.012 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:06.012 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:06.012 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:06.012 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:06.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:06.012 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-06-06T03:49:06.012 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T03:49:06.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T03:49:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 2, 2024-06-06T03:49:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 179, 2024-06-06T03:49:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 320, 2024-06-06T03:49:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 187200, 2024-06-06T03:49:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 5, 2024-06-06T03:49:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 3, 2024-06-06T03:49:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.016297714, 2024-06-06T03:49:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.011210538, 2024-06-06T03:49:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.002142077, 2024-06-06T03:49:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T03:49:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 176, 2024-06-06T03:49:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.011368896, 2024-06-06T03:49:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000064596 2024-06-06T03:49:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-06T03:49:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 187200 2024-06-06T03:49:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T03:49:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 245, 2024-06-06T03:49:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 234, 2024-06-06T03:49:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 1587072, 2024-06-06T03:49:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 144064, 2024-06-06T03:49:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 2, 2024-06-06T03:49:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 1, 2024-06-06T03:49:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.046526540, 2024-06-06T03:49:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.013532197, 2024-06-06T03:49:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.037569665, 2024-06-06T03:49:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.004697625, 2024-06-06T03:49:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 233, 2024-06-06T03:49:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.010802601, 2024-06-06T03:49:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000046363 2024-06-06T03:49:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 1587072, 2024-06-06T03:49:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 144064 2024-06-06T03:49:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2024-06-06T03:49:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 0, 2024-06-06T03:49:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2024-06-06T03:49:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2024-06-06T03:49:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2024-06-06T03:49:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2024-06-06T03:49:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T03:49:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2024-06-06T03:49:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T03:49:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2024-06-06T03:49:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T03:49:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T03:49:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T03:49:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T03:49:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T03:49:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2024-06-06T03:49:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T03:49:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T03:49:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2024-06-06T03:49:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2024-06-06T03:49:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2024-06-06T03:49:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2024-06-06T03:49:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T03:49:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T03:49:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T03:49:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T03:49:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T03:49:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2024-06-06T03:49:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2024-06-06T03:49:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2024-06-06T03:49:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T03:49:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T03:49:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2024-06-06T03:49:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2024-06-06T03:49:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2024-06-06T03:49:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2024-06-06T03:49:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T03:49:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2024-06-06T03:49:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T03:49:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2024-06-06T03:49:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 148714, 2024-06-06T03:49:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 30, 2024-06-06T03:49:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2024-06-06T03:49:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2024-06-06T03:49:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2024-06-06T03:49:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2024-06-06T03:49:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2024-06-06T03:49:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2024-06-06T03:49:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2024-06-06T03:49:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2024-06-06T03:49:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 0, 2024-06-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 0, 2024-06-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2024-06-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2024-06-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2024-06-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 928, 2024-06-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 2, 2024-06-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2024-06-06T03:49:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2024-06-06T03:49:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2024-06-06T03:49:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2024-06-06T03:49:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-06-06T03:49:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T03:49:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-06-06T03:49:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-06-06T03:49:06.027 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-06-06T03:49:06.027 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 247, 2024-06-06T03:49:06.027 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 1552510, 2024-06-06T03:49:06.027 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:06.027 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 247, 2024-06-06T03:49:06.027 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-06-06T03:49:06.027 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-06-06T03:49:06.027 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 247, 2024-06-06T03:49:06.028 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 1552510, 2024-06-06T03:49:06.028 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-06-06T03:49:06.028 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-06-06T03:49:06.028 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-06-06T03:49:06.028 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-06-06T03:49:06.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:49:06.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-06-06T03:49:06.028 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-06-06T03:49:06.756 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-06-06T03:49:06.757 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-06-06T03:49:06.757 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-06-06T03:49:06.757 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-06-06T03:49:06.757 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-06-06T03:49:06.757 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-06-06T03:49:06.757 DEBUG:tasks.check_counter:Getting stats from b 2024-06-06T03:49:06.758 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-06T03:49:07.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:06 smithi040 ceph-mon[32220]: pgmap v325: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 693 KiB/s wr, 65 op/s 2024-06-06T03:49:07.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:06 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:07.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:06 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:07.170 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T03:49:07.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:06 smithi131 ceph-mon[36687]: pgmap v325: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 693 KiB/s wr, 65 op/s 2024-06-06T03:49:07.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:06 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:07.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:06 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:07.299 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:06 smithi104 ceph-mon[36569]: pgmap v325: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 693 KiB/s wr, 65 op/s 2024-06-06T03:49:07.299 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:06 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:07.299 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:06 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:08.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:07 smithi131 ceph-mon[36687]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T03:49:08.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:07 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:08.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:07 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:08.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:07 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:08.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:07 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:08.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:07 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:49:08.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:07 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:49:08.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:07 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:07 smithi040 ceph-mon[32220]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T03:49:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:07 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:07 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:07 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:07 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:07 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:49:08.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:07 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:49:08.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:07 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:08.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:07 smithi104 ceph-mon[36569]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T03:49:08.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:07 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:08.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:07 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:08.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:07 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:08.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:07 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T03:49:08.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:07 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T03:49:08.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:07 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T03:49:08.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:07 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' 2024-06-06T03:49:09.233 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:08 smithi104 ceph-mon[36569]: Adjusting osd_memory_target on smithi040 to 177.2M 2024-06-06T03:49:09.234 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:08 smithi104 ceph-mon[36569]: Unable to set osd_memory_target on smithi040 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-06T03:49:09.234 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:08 smithi104 ceph-mon[36569]: pgmap v327: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 627 KiB/s wr, 64 op/s 2024-06-06T03:49:09.234 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-06-06T03:49:09.234 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T03:49:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 175708, 2024-06-06T03:49:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 404560, 2024-06-06T03:49:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 61743678, 2024-06-06T03:49:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 335346469, 2024-06-06T03:49:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 17, 2024-06-06T03:49:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 4, 2024-06-06T03:49:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 11.443533709, 2024-06-06T03:49:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 5.484815420, 2024-06-06T03:49:09.235 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 28.107499675, 2024-06-06T03:49:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.328815522, 2024-06-06T03:49:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 404561, 2024-06-06T03:49:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 18.514684746, 2024-06-06T03:49:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000045764 2024-06-06T03:49:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:09.236 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 518311, 2024-06-06T03:49:09.237 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.093621685, 2024-06-06T03:49:09.237 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000180 2024-06-06T03:49:09.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.237 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 60992, 2024-06-06T03:49:09.237 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 347488 2024-06-06T03:49:09.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.237 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T03:49:09.237 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 25509, 2024-06-06T03:49:09.237 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 15522, 2024-06-06T03:49:09.238 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 9180432, 2024-06-06T03:49:09.238 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 168419550, 2024-06-06T03:49:09.238 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 9, 2024-06-06T03:49:09.238 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 7, 2024-06-06T03:49:09.238 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 1.699306304, 2024-06-06T03:49:09.238 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.864776633, 2024-06-06T03:49:09.238 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.724517902, 2024-06-06T03:49:09.238 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.255543409, 2024-06-06T03:49:09.239 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:09.239 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 15525, 2024-06-06T03:49:09.239 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.976090102, 2024-06-06T03:49:09.239 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000062872 2024-06-06T03:49:09.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.239 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:09.239 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 25552, 2024-06-06T03:49:09.239 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.004940236, 2024-06-06T03:49:09.240 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000193 2024-06-06T03:49:09.240 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.240 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 2099744, 2024-06-06T03:49:09.240 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 252848 2024-06-06T03:49:09.240 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.240 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T03:49:09.240 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 21360, 2024-06-06T03:49:09.240 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 15098, 2024-06-06T03:49:09.241 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 21490361, 2024-06-06T03:49:09.241 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 122476704, 2024-06-06T03:49:09.241 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 45, 2024-06-06T03:49:09.241 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 7, 2024-06-06T03:49:09.241 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 1.532753871, 2024-06-06T03:49:09.241 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.745614821, 2024-06-06T03:49:09.241 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.566432080, 2024-06-06T03:49:09.241 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.300579982, 2024-06-06T03:49:09.241 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:09.242 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 15089, 2024-06-06T03:49:09.242 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.947420793, 2024-06-06T03:49:09.242 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000062788 2024-06-06T03:49:09.242 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.242 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:09.242 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 22578, 2024-06-06T03:49:09.242 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.003517090, 2024-06-06T03:49:09.242 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000155 2024-06-06T03:49:09.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.243 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 86320, 2024-06-06T03:49:09.243 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 164192 2024-06-06T03:49:09.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.243 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-06-06T03:49:09.243 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-06-06T03:49:09.243 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-06-06T03:49:09.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.243 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-06-06T03:49:09.244 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-06-06T03:49:09.244 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-06-06T03:49:09.244 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 7168, 2024-06-06T03:49:09.244 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 126.031547118, 2024-06-06T03:49:09.244 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.017582526 2024-06-06T03:49:09.244 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:09.244 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.244 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-06-06T03:49:09.245 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-06-06T03:49:09.245 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-06-06T03:49:09.245 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1216, 2024-06-06T03:49:09.245 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.059070688, 2024-06-06T03:49:09.245 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000048577 2024-06-06T03:49:09.245 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:09.245 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.245 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-06-06T03:49:09.245 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 78611, 2024-06-06T03:49:09.246 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 77921, 2024-06-06T03:49:09.246 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-06-06T03:49:09.246 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 77921, 2024-06-06T03:49:09.246 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 311.629445765, 2024-06-06T03:49:09.246 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003999299 2024-06-06T03:49:09.246 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.246 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-06-06T03:49:09.247 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 690, 2024-06-06T03:49:09.247 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 2, 2024-06-06T03:49:09.247 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-06-06T03:49:09.247 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 1014, 2024-06-06T03:49:09.247 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 111, 2024-06-06T03:49:09.247 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-06-06T03:49:09.247 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 19567, 2024-06-06T03:49:09.247 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 1481, 2024-06-06T03:49:09.247 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 18052, 2024-06-06T03:49:09.248 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 34, 2024-06-06T03:49:09.248 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 17443, 2024-06-06T03:49:09.248 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 74068, 2024-06-06T03:49:09.248 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 735, 2024-06-06T03:49:09.248 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 736, 2024-06-06T03:49:09.248 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 106, 2024-06-06T03:49:09.248 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 81208, 2024-06-06T03:49:09.248 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 77489, 2024-06-06T03:49:09.248 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 628, 2024-06-06T03:49:09.249 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 8, 2024-06-06T03:49:09.249 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 1, 2024-06-06T03:49:09.249 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-06-06T03:49:09.249 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 1, 2024-06-06T03:49:09.249 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 3141, 2024-06-06T03:49:09.249 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-06-06T03:49:09.249 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 323, 2024-06-06T03:49:09.249 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 75668, 2024-06-06T03:49:09.250 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 28, 2024-06-06T03:49:09.250 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 13927, 2024-06-06T03:49:09.250 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-06-06T03:49:09.250 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-06-06T03:49:09.250 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-06-06T03:49:09.250 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 84951, 2024-06-06T03:49:09.250 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 1552868865, 2024-06-06T03:49:09.251 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 6, 2024-06-06T03:49:09.251 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T03:49:09.251 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-06-06T03:49:09.251 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T03:49:09.251 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-06-06T03:49:09.251 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-06-06T03:49:09.251 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T03:49:09.251 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T03:49:09.252 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-06-06T03:49:09.252 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 87, 2024-06-06T03:49:09.252 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 44573, 2024-06-06T03:49:09.252 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 76047, 2024-06-06T03:49:09.252 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 21, 2024-06-06T03:49:09.252 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T03:49:09.252 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T03:49:09.252 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 80500, 2024-06-06T03:49:09.253 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 49000, 2024-06-06T03:49:09.253 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 1157, 2024-06-06T03:49:09.253 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 5968 2024-06-06T03:49:09.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.253 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-06-06T03:49:09.253 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-06-06T03:49:09.253 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-06-06T03:49:09.253 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-06-06T03:49:09.254 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 30, 2024-06-06T03:49:09.254 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 408, 2024-06-06T03:49:09.254 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-06-06T03:49:09.254 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-06-06T03:49:09.254 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-06-06T03:49:09.254 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 578, 2024-06-06T03:49:09.254 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 578, 2024-06-06T03:49:09.254 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-06-06T03:49:09.255 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-06-06T03:49:09.255 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-06-06T03:49:09.255 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-06-06T03:49:09.255 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-06-06T03:49:09.255 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-06-06T03:49:09.255 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-06-06T03:49:09.255 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 7, 2024-06-06T03:49:09.255 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 53442, 2024-06-06T03:49:09.255 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T03:49:09.256 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 324, 2024-06-06T03:49:09.256 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-06-06T03:49:09.256 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 116, 2024-06-06T03:49:09.256 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-06-06T03:49:09.256 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-06-06T03:49:09.256 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.256 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-06-06T03:49:09.256 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 2, 2024-06-06T03:49:09.257 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 146167, 2024-06-06T03:49:09.257 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 39912, 2024-06-06T03:49:09.257 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 36871, 2024-06-06T03:49:09.257 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 109296, 2024-06-06T03:49:09.257 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-06-06T03:49:09.257 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 3041, 2024-06-06T03:49:09.257 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 174, 2024-06-06T03:49:09.257 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 46, 2024-06-06T03:49:09.257 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 42, 2024-06-06T03:49:09.258 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 132, 2024-06-06T03:49:09.258 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 9, 2024-06-06T03:49:09.258 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-06-06T03:49:09.258 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 4, 2024-06-06T03:49:09.258 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 85886036, 2024-06-06T03:49:09.258 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 358837704, 2024-06-06T03:49:09.258 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-06-06T03:49:09.258 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-06-06T03:49:09.259 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 18720, 2024-06-06T03:49:09.259 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 2369.200922441, 2024-06-06T03:49:09.259 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.126559878 2024-06-06T03:49:09.259 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.259 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-06-06T03:49:09.259 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.259 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-06-06T03:49:09.259 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 19112, 2024-06-06T03:49:09.259 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 93379, 2024-06-06T03:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 74267, 2024-06-06T03:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 1435, 2024-06-06T03:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 6533, 2024-06-06T03:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 5098, 2024-06-06T03:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 19567, 2024-06-06T03:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 94745, 2024-06-06T03:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 75178, 2024-06-06T03:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 736, 2024-06-06T03:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 91302, 2024-06-06T03:49:09.260 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 90566, 2024-06-06T03:49:09.261 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 197736, 2024-06-06T03:49:09.261 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 231804 2024-06-06T03:49:09.261 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.261 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-06-06T03:49:09.261 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 79714, 2024-06-06T03:49:09.261 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 833, 2024-06-06T03:49:09.261 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 78611, 2024-06-06T03:49:09.261 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 340, 2024-06-06T03:49:09.261 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 1997, 2024-06-06T03:49:09.262 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-06-06T03:49:09.262 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 69687, 2024-06-06T03:49:09.262 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 254.118610954, 2024-06-06T03:49:09.262 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003646571 2024-06-06T03:49:09.262 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.262 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-06-06T03:49:09.262 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 176, 2024-06-06T03:49:09.262 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.747359695, 2024-06-06T03:49:09.263 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004246361 2024-06-06T03:49:09.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.263 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-06-06T03:49:09.263 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.263 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.263 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.263 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-06-06T03:49:09.263 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.264 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.264 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.264 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.264 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-06-06T03:49:09.264 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1973, 2024-06-06T03:49:09.264 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 17.853676561, 2024-06-06T03:49:09.264 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.009048999 2024-06-06T03:49:09.264 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.265 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-06-06T03:49:09.265 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.265 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.265 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.265 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-06-06T03:49:09.265 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-06-06T03:49:09.265 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000519295, 2024-06-06T03:49:09.265 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000259647 2024-06-06T03:49:09.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.266 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-06-06T03:49:09.266 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.266 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.266 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.266 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-06-06T03:49:09.267 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.267 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.267 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.267 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.267 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-06-06T03:49:09.267 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.268 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.268 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.268 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-06-06T03:49:09.268 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11, 2024-06-06T03:49:09.268 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.014435572, 2024-06-06T03:49:09.269 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001312324 2024-06-06T03:49:09.269 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.269 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-06-06T03:49:09.269 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4485, 2024-06-06T03:49:09.269 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 31.790877060, 2024-06-06T03:49:09.269 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.007088266 2024-06-06T03:49:09.270 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.270 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-06-06T03:49:09.270 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.270 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.270 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.270 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.270 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-06-06T03:49:09.271 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11, 2024-06-06T03:49:09.271 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 2.425423089, 2024-06-06T03:49:09.271 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.220493008 2024-06-06T03:49:09.271 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.271 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-06-06T03:49:09.271 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 342, 2024-06-06T03:49:09.271 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.810654243, 2024-06-06T03:49:09.271 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002370334 2024-06-06T03:49:09.272 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.272 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-06-06T03:49:09.272 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 564, 2024-06-06T03:49:09.272 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.827242330, 2024-06-06T03:49:09.272 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001466741 2024-06-06T03:49:09.272 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.272 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-06-06T03:49:09.272 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5, 2024-06-06T03:49:09.273 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.005603810, 2024-06-06T03:49:09.273 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001120762 2024-06-06T03:49:09.273 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.273 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-06-06T03:49:09.273 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.273 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.273 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.273 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.273 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-06-06T03:49:09.274 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.274 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.274 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.274 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.274 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-06-06T03:49:09.274 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 109, 2024-06-06T03:49:09.274 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.469851687, 2024-06-06T03:49:09.274 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004310565 2024-06-06T03:49:09.275 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.275 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-06-06T03:49:09.275 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5, 2024-06-06T03:49:09.275 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.532991347, 2024-06-06T03:49:09.275 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.106598269 2024-06-06T03:49:09.275 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.275 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-06-06T03:49:09.275 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.275 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.276 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.276 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-06-06T03:49:09.276 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 53, 2024-06-06T03:49:09.276 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.086913304, 2024-06-06T03:49:09.276 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.020507798 2024-06-06T03:49:09.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.276 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-06-06T03:49:09.277 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.277 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.277 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.277 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.277 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-06-06T03:49:09.277 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.277 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.277 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.278 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.278 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-06-06T03:49:09.278 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.278 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.278 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.278 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.278 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-06-06T03:49:09.278 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 7, 2024-06-06T03:49:09.278 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.020500284, 2024-06-06T03:49:09.279 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002928612 2024-06-06T03:49:09.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.279 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-06-06T03:49:09.279 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 22, 2024-06-06T03:49:09.279 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.016826190, 2024-06-06T03:49:09.279 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000764826 2024-06-06T03:49:09.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.279 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-06-06T03:49:09.280 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 469, 2024-06-06T03:49:09.280 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.907960344, 2024-06-06T03:49:09.280 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001935949 2024-06-06T03:49:09.280 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.280 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-06-06T03:49:09.280 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-06-06T03:49:09.280 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-06-06T03:49:09.280 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.280 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.281 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.281 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:09.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.281 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-06-06T03:49:09.281 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 2, 2024-06-06T03:49:09.281 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 5, 2024-06-06T03:49:09.281 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 3, 2024-06-06T03:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 2, 2024-06-06T03:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-06-06T03:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 3301, 2024-06-06T03:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 1650, 2024-06-06T03:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 618, 2024-06-06T03:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-06-06T03:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-06-06T03:49:09.282 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 480, 2024-06-06T03:49:09.283 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 480, 2024-06-06T03:49:09.283 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T03:49:09.283 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-06-06T03:49:09.283 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T03:49:09.283 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-06-06T03:49:09.283 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T03:49:09.283 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T03:49:09.283 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T03:49:09.284 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T03:49:09.284 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T03:49:09.284 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-06-06T03:49:09.284 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T03:49:09.284 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T03:49:09.284 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-06-06T03:49:09.284 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-06-06T03:49:09.284 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-06-06T03:49:09.285 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-06-06T03:49:09.285 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T03:49:09.285 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T03:49:09.285 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T03:49:09.285 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T03:49:09.285 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T03:49:09.285 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-06-06T03:49:09.285 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-06-06T03:49:09.286 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-06-06T03:49:09.286 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T03:49:09.286 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T03:49:09.286 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-06-06T03:49:09.286 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-06-06T03:49:09.286 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-06-06T03:49:09.286 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-06-06T03:49:09.286 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T03:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-06-06T03:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T03:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-06-06T03:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 930819, 2024-06-06T03:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 17259, 2024-06-06T03:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 264, 2024-06-06T03:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 3, 2024-06-06T03:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-06-06T03:49:09.287 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-06-06T03:49:09.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-06-06T03:49:09.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-06-06T03:49:09.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-06-06T03:49:09.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-06-06T03:49:09.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 7744, 2024-06-06T03:49:09.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 161, 2024-06-06T03:49:09.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T03:49:09.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-06-06T03:49:09.289 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-06-06T03:49:09.289 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-06-06T03:49:09.289 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 61556731, 2024-06-06T03:49:09.289 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 733747, 2024-06-06T03:49:09.289 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-06-06T03:49:09.289 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-06-06T03:49:09.289 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-06-06T03:49:09.289 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-06-06T03:49:09.290 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.290 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-06-06T03:49:09.290 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-06-06T03:49:09.290 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-06-06T03:49:09.290 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 21723, 2024-06-06T03:49:09.290 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 376977243, 2024-06-06T03:49:09.290 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 31, 2024-06-06T03:49:09.290 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 21723, 2024-06-06T03:49:09.290 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-06-06T03:49:09.291 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 21723, 2024-06-06T03:49:09.291 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 139.310525622, 2024-06-06T03:49:09.291 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006413042 2024-06-06T03:49:09.291 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.291 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-06-06T03:49:09.291 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-06-06T03:49:09.291 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 21723, 2024-06-06T03:49:09.291 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 23404 2024-06-06T03:49:09.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.292 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 21723, 2024-06-06T03:49:09.292 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 60, 2024-06-06T03:49:09.292 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 21663, 2024-06-06T03:49:09.292 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-06-06T03:49:09.292 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-06-06T03:49:09.292 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 0, 2024-06-06T03:49:09.292 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 48, 2024-06-06T03:49:09.293 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 56, 2024-06-06T03:49:09.293 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 18777, 2024-06-06T03:49:09.293 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 89, 2024-06-06T03:49:09.293 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-06-06T03:49:09.293 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-06-06T03:49:09.293 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-06-06T03:49:09.293 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 102, 2024-06-06T03:49:09.293 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 1202, 2024-06-06T03:49:09.294 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-06-06T03:49:09.294 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-06-06T03:49:09.294 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-06-06T03:49:09.294 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 2, 2024-06-06T03:49:09.294 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 96, 2024-06-06T03:49:09.294 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-06-06T03:49:09.294 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-06-06T03:49:09.294 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-06-06T03:49:09.295 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-06-06T03:49:09.295 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-06-06T03:49:09.295 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-06-06T03:49:09.295 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T03:49:09.295 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-06-06T03:49:09.295 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-06-06T03:49:09.295 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 21, 2024-06-06T03:49:09.295 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-06-06T03:49:09.296 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-06-06T03:49:09.296 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-06-06T03:49:09.296 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-06-06T03:49:09.296 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-06-06T03:49:09.296 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-06-06T03:49:09.296 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-06-06T03:49:09.296 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-06-06T03:49:09.296 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-06-06T03:49:09.296 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-06-06T03:49:09.297 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-06-06T03:49:09.297 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-06-06T03:49:09.297 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-06-06T03:49:09.297 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-06-06T03:49:09.297 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-06-06T03:49:09.297 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-06-06T03:49:09.297 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 100, 2024-06-06T03:49:09.297 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-06-06T03:49:09.297 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 12, 2024-06-06T03:49:09.298 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2024-06-06T03:49:09.298 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2024-06-06T03:49:09.298 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-06-06T03:49:09.298 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-06-06T03:49:09.298 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 2715, 2024-06-06T03:49:09.298 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 6, 2024-06-06T03:49:09.298 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 290 2024-06-06T03:49:09.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.299 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-06-06T03:49:09.299 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-06-06T03:49:09.299 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-06-06T03:49:09.299 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-06-06T03:49:09.299 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-06-06T03:49:09.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.299 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-06-06T03:49:09.299 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 51889 2024-06-06T03:49:09.300 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.300 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-06-06T03:49:09.300 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-06-06T03:49:09.300 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 6, 2024-06-06T03:49:09.300 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-06-06T03:49:09.300 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 6, 2024-06-06T03:49:09.300 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 578, 2024-06-06T03:49:09.300 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 578, 2024-06-06T03:49:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-06-06T03:49:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T03:49:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-06-06T03:49:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 222573, 2024-06-06T03:49:09.301 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 72314141, 2024-06-06T03:49:09.302 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:09.302 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 222573, 2024-06-06T03:49:09.302 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-06-06T03:49:09.302 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-06-06T03:49:09.302 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 222573, 2024-06-06T03:49:09.302 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 72314141, 2024-06-06T03:49:09.303 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:09.303 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.303 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.303 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.303 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:09.303 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.303 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-06-06T03:49:09.303 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-06-06T03:49:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-06-06T03:49:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-06-06T03:49:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:09.304 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 21723, 2024-06-06T03:49:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 377035624, 2024-06-06T03:49:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 21723, 2024-06-06T03:49:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 377035624, 2024-06-06T03:49:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.305 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-06-06T03:49:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-06-06T03:49:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:09.306 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-06-06T03:49:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-06-06T03:49:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 21723, 2024-06-06T03:49:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 21723, 2024-06-06T03:49:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 21723, 2024-06-06T03:49:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 21723, 2024-06-06T03:49:09.307 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.308 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-06-06T03:49:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-06-06T03:49:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 578, 2024-06-06T03:49:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 58474, 2024-06-06T03:49:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 578, 2024-06-06T03:49:09.309 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-06-06T03:49:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-06-06T03:49:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 56, 2024-06-06T03:49:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 58474, 2024-06-06T03:49:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.310 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.311 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:09.311 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:09.311 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55a51d6d6500": { 2024-06-06T03:49:09.311 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 112, 2024-06-06T03:49:09.311 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-06-06T03:49:09.311 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 146167, 2024-06-06T03:49:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 354643400, 2024-06-06T03:49:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 146167, 2024-06-06T03:49:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-06-06T03:49:09.312 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-06-06T03:49:09.313 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 18720, 2024-06-06T03:49:09.313 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 354643288, 2024-06-06T03:49:09.313 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:09.313 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:09.313 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:09.313 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:09.314 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:09.314 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:09.314 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-06-06T03:49:09.314 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:08 smithi131 ceph-mon[36687]: Adjusting osd_memory_target on smithi040 to 177.2M 2024-06-06T03:49:09.314 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:08 smithi131 ceph-mon[36687]: Unable to set osd_memory_target on smithi040 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-06T03:49:09.315 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:08 smithi131 ceph-mon[36687]: pgmap v327: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 627 KiB/s wr, 64 op/s 2024-06-06T03:49:09.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:08 smithi040 ceph-mon[32220]: Adjusting osd_memory_target on smithi040 to 177.2M 2024-06-06T03:49:09.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:08 smithi040 ceph-mon[32220]: Unable to set osd_memory_target on smithi040 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-06T03:49:09.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:08 smithi040 ceph-mon[32220]: pgmap v327: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 627 KiB/s wr, 64 op/s 2024-06-06T03:49:10.246 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:09 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:10.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:09 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:10.289 INFO:tasks.check_counter:Daemon mds.b mds.exported=323 2024-06-06T03:49:10.289 INFO:tasks.check_counter:Daemon mds.b mds.imported=28 2024-06-06T03:49:10.289 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-06-06T03:49:10.289 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-06-06T03:49:10.290 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-06-06T03:49:10.290 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=111 2024-06-06T03:49:10.290 DEBUG:tasks.check_counter:Getting stats from e 2024-06-06T03:49:10.290 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-06T03:49:10.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:09 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:10.533 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T03:49:11.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:10 smithi131 ceph-mon[36687]: pgmap v328: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 75 KiB/s wr, 38 op/s 2024-06-06T03:49:11.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:10 smithi040 ceph-mon[32220]: pgmap v328: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 75 KiB/s wr, 38 op/s 2024-06-06T03:49:11.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:10 smithi104 ceph-mon[36569]: pgmap v328: 97 pgs: 97 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 75 KiB/s wr, 38 op/s 2024-06-06T03:49:12.698 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-06-06T03:49:12.698 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T03:49:12.699 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 56813, 2024-06-06T03:49:12.699 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 82184, 2024-06-06T03:49:12.699 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 29036776, 2024-06-06T03:49:12.699 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 55122694, 2024-06-06T03:49:12.699 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 16, 2024-06-06T03:49:12.699 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 4, 2024-06-06T03:49:12.699 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 2.611404124, 2024-06-06T03:49:12.700 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 1.263002208, 2024-06-06T03:49:12.700 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 3.753895868, 2024-06-06T03:49:12.700 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.021261858, 2024-06-06T03:49:12.700 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:12.700 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 82184, 2024-06-06T03:49:12.700 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 8.691465665, 2024-06-06T03:49:12.700 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000105756 2024-06-06T03:49:12.700 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.701 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:12.701 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 94437, 2024-06-06T03:49:12.701 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.020517675, 2024-06-06T03:49:12.701 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000217 2024-06-06T03:49:12.701 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.701 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1766992, 2024-06-06T03:49:12.701 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 154096 2024-06-06T03:49:12.701 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.702 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T03:49:12.702 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1768, 2024-06-06T03:49:12.702 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 1938, 2024-06-06T03:49:12.702 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 628844, 2024-06-06T03:49:12.702 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 19061486, 2024-06-06T03:49:12.702 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 13, 2024-06-06T03:49:12.702 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 7, 2024-06-06T03:49:12.702 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.215309909, 2024-06-06T03:49:12.703 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.108819658, 2024-06-06T03:49:12.703 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.049817641, 2024-06-06T03:49:12.703 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.046427924, 2024-06-06T03:49:12.703 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:12.703 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1935, 2024-06-06T03:49:12.703 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.312061319, 2024-06-06T03:49:12.703 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000161271 2024-06-06T03:49:12.703 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.703 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:12.704 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 915, 2024-06-06T03:49:12.704 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000670565, 2024-06-06T03:49:12.704 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000732 2024-06-06T03:49:12.704 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.704 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 2976, 2024-06-06T03:49:12.704 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 473328 2024-06-06T03:49:12.704 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.704 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T03:49:12.704 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 8638, 2024-06-06T03:49:12.705 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 17966, 2024-06-06T03:49:12.705 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 19815570, 2024-06-06T03:49:12.705 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 21295271, 2024-06-06T03:49:12.705 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 11, 2024-06-06T03:49:12.705 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 7, 2024-06-06T03:49:12.705 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 1.030431106, 2024-06-06T03:49:12.705 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.456321289, 2024-06-06T03:49:12.705 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.462400181, 2024-06-06T03:49:12.705 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.177108983, 2024-06-06T03:49:12.706 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:12.706 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 17964, 2024-06-06T03:49:12.706 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.085371172, 2024-06-06T03:49:12.706 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000060419 2024-06-06T03:49:12.706 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.706 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:12.706 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 24659, 2024-06-06T03:49:12.706 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.005906336, 2024-06-06T03:49:12.706 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000239 2024-06-06T03:49:12.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.707 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T03:49:12.707 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T03:49:12.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.707 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-06-06T03:49:12.707 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-06-06T03:49:12.707 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-06-06T03:49:12.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.707 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-06-06T03:49:12.708 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-06-06T03:49:12.708 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-06-06T03:49:12.708 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2348, 2024-06-06T03:49:12.708 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 11.672300721, 2024-06-06T03:49:12.708 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004971167 2024-06-06T03:49:12.708 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:12.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.708 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-06-06T03:49:12.708 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-06-06T03:49:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-06-06T03:49:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 263, 2024-06-06T03:49:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.009810273, 2024-06-06T03:49:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000037301 2024-06-06T03:49:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-06-06T03:49:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 4610, 2024-06-06T03:49:12.709 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 4379, 2024-06-06T03:49:12.710 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-06-06T03:49:12.710 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4379, 2024-06-06T03:49:12.710 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 35.514520515, 2024-06-06T03:49:12.710 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.008110189 2024-06-06T03:49:12.710 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.710 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-06-06T03:49:12.710 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 231, 2024-06-06T03:49:12.710 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-06-06T03:49:12.711 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-06-06T03:49:12.711 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 101, 2024-06-06T03:49:12.711 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 10, 2024-06-06T03:49:12.711 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-06-06T03:49:12.711 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 29019, 2024-06-06T03:49:12.711 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 403, 2024-06-06T03:49:12.711 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 28616, 2024-06-06T03:49:12.711 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-06-06T03:49:12.712 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 28433, 2024-06-06T03:49:12.712 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 1, 2024-06-06T03:49:12.712 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 953, 2024-06-06T03:49:12.712 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 953, 2024-06-06T03:49:12.712 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 84, 2024-06-06T03:49:12.712 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 5907, 2024-06-06T03:49:12.712 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 4925, 2024-06-06T03:49:12.712 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 219, 2024-06-06T03:49:12.712 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 53, 2024-06-06T03:49:12.713 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-06-06T03:49:12.713 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-06-06T03:49:12.713 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-06-06T03:49:12.713 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 2695, 2024-06-06T03:49:12.713 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-06-06T03:49:12.713 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-06-06T03:49:12.713 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-06-06T03:49:12.713 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 65, 2024-06-06T03:49:12.713 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 25073, 2024-06-06T03:49:12.714 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-06-06T03:49:12.714 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-06-06T03:49:12.714 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-06-06T03:49:12.714 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 84812, 2024-06-06T03:49:12.714 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 1539542236, 2024-06-06T03:49:12.714 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 6, 2024-06-06T03:49:12.714 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T03:49:12.714 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-06-06T03:49:12.714 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T03:49:12.715 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-06-06T03:49:12.715 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-06-06T03:49:12.715 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T03:49:12.715 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T03:49:12.715 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-06-06T03:49:12.715 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 256, 2024-06-06T03:49:12.715 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 38315, 2024-06-06T03:49:12.715 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 6012, 2024-06-06T03:49:12.716 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 7, 2024-06-06T03:49:12.716 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T03:49:12.716 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T03:49:12.716 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 49547, 2024-06-06T03:49:12.716 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 12120, 2024-06-06T03:49:12.716 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 1459, 2024-06-06T03:49:12.716 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 1682 2024-06-06T03:49:12.716 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.717 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-06-06T03:49:12.717 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-06-06T03:49:12.717 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-06-06T03:49:12.717 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-06-06T03:49:12.717 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 121, 2024-06-06T03:49:12.717 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 42, 2024-06-06T03:49:12.717 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-06-06T03:49:12.717 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-06-06T03:49:12.718 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-06-06T03:49:12.718 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 107, 2024-06-06T03:49:12.719 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 107, 2024-06-06T03:49:12.719 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-06-06T03:49:12.719 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-06-06T03:49:12.719 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-06-06T03:49:12.719 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-06-06T03:49:12.719 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-06-06T03:49:12.719 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-06-06T03:49:12.719 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-06-06T03:49:12.720 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 7, 2024-06-06T03:49:12.720 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 78743, 2024-06-06T03:49:12.720 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T03:49:12.720 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-06-06T03:49:12.720 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-06-06T03:49:12.720 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 10, 2024-06-06T03:49:12.720 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-06-06T03:49:12.720 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-06-06T03:49:12.721 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.721 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-06-06T03:49:12.721 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 5, 2024-06-06T03:49:12.721 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 15071, 2024-06-06T03:49:12.721 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-06-06T03:49:12.721 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-06-06T03:49:12.721 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 15071, 2024-06-06T03:49:12.721 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-06-06T03:49:12.722 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-06-06T03:49:12.722 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 24, 2024-06-06T03:49:12.722 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-06-06T03:49:12.722 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-06-06T03:49:12.722 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 24, 2024-06-06T03:49:12.722 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 3, 2024-06-06T03:49:12.722 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-06-06T03:49:12.722 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-06-06T03:49:12.723 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-06-06T03:49:12.723 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 55168420, 2024-06-06T03:49:12.723 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-06-06T03:49:12.723 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-06-06T03:49:12.723 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1311, 2024-06-06T03:49:12.723 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 17.452913554, 2024-06-06T03:49:12.723 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.013312672 2024-06-06T03:49:12.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.724 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-06-06T03:49:12.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.724 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-06-06T03:49:12.724 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 28629, 2024-06-06T03:49:12.724 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 28736, 2024-06-06T03:49:12.724 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 107, 2024-06-06T03:49:12.724 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 1743, 2024-06-06T03:49:12.724 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 1756, 2024-06-06T03:49:12.725 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 13, 2024-06-06T03:49:12.725 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 29019, 2024-06-06T03:49:12.725 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 29178, 2024-06-06T03:49:12.725 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 159, 2024-06-06T03:49:12.725 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 953, 2024-06-06T03:49:12.725 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 38961, 2024-06-06T03:49:12.725 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 38008, 2024-06-06T03:49:12.725 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 209680, 2024-06-06T03:49:12.725 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 223612 2024-06-06T03:49:12.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.726 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-06-06T03:49:12.726 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 4936, 2024-06-06T03:49:12.726 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 12, 2024-06-06T03:49:12.726 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 4610, 2024-06-06T03:49:12.726 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 16, 2024-06-06T03:49:12.726 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 170, 2024-06-06T03:49:12.726 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-06-06T03:49:12.727 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1130, 2024-06-06T03:49:12.727 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 15.995429855, 2024-06-06T03:49:12.727 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.014155247 2024-06-06T03:49:12.727 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.727 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-06-06T03:49:12.727 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1, 2024-06-06T03:49:12.727 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.001549192, 2024-06-06T03:49:12.727 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001549192 2024-06-06T03:49:12.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.728 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-06-06T03:49:12.728 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.728 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.728 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.728 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-06-06T03:49:12.728 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.728 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.729 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.729 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.729 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-06-06T03:49:12.729 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1073, 2024-06-06T03:49:12.729 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 16.453088149, 2024-06-06T03:49:12.729 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.015333726 2024-06-06T03:49:12.729 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.729 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-06-06T03:49:12.730 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.730 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.730 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.730 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.730 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-06-06T03:49:12.730 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.730 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.730 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.731 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-06-06T03:49:12.731 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.731 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.731 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.731 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-06-06T03:49:12.732 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.732 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.732 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.732 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.732 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-06-06T03:49:12.732 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.732 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.732 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.732 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.733 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-06-06T03:49:12.733 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.733 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.734 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.734 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.734 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-06-06T03:49:12.734 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 40, 2024-06-06T03:49:12.734 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.200868070, 2024-06-06T03:49:12.734 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.005021701 2024-06-06T03:49:12.734 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.735 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-06-06T03:49:12.735 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.735 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.735 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.735 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.735 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-06-06T03:49:12.735 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.735 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.736 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.736 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-06-06T03:49:12.736 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 572, 2024-06-06T03:49:12.736 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.118373691, 2024-06-06T03:49:12.736 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001955198 2024-06-06T03:49:12.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.736 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-06-06T03:49:12.736 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1449, 2024-06-06T03:49:12.737 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.452580573, 2024-06-06T03:49:12.737 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001002471 2024-06-06T03:49:12.737 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.737 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-06-06T03:49:12.737 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.737 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.737 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.737 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.738 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-06-06T03:49:12.738 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.738 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.738 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.738 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-06-06T03:49:12.738 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.738 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.739 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.739 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-06-06T03:49:12.739 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.739 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.739 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.739 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-06-06T03:49:12.739 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.740 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.740 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.740 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-06-06T03:49:12.740 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.740 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.740 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.741 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-06-06T03:49:12.741 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 7, 2024-06-06T03:49:12.741 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.091754132, 2024-06-06T03:49:12.741 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.013107733 2024-06-06T03:49:12.741 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.741 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-06-06T03:49:12.741 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.741 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.742 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.742 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.742 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-06-06T03:49:12.742 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.742 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.742 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.742 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.742 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-06-06T03:49:12.743 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.743 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.743 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.743 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-06-06T03:49:12.743 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.743 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.743 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.744 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.744 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-06-06T03:49:12.744 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.744 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.744 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.744 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.744 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-06-06T03:49:12.744 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 107, 2024-06-06T03:49:12.744 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.200876853, 2024-06-06T03:49:12.745 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001877353 2024-06-06T03:49:12.745 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.745 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-06-06T03:49:12.745 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-06-06T03:49:12.745 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-06-06T03:49:12.745 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.745 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.745 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.745 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:12.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.746 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-06-06T03:49:12.746 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 1, 2024-06-06T03:49:12.746 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 3, 2024-06-06T03:49:12.746 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 2, 2024-06-06T03:49:12.746 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 1, 2024-06-06T03:49:12.746 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-06-06T03:49:12.746 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 1172, 2024-06-06T03:49:12.747 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 1172, 2024-06-06T03:49:12.747 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 2041, 2024-06-06T03:49:12.747 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-06-06T03:49:12.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.747 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-06-06T03:49:12.747 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-06-06T03:49:12.747 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-06-06T03:49:12.747 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T03:49:12.748 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-06-06T03:49:12.748 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T03:49:12.748 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-06-06T03:49:12.748 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T03:49:12.748 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T03:49:12.748 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T03:49:12.748 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T03:49:12.748 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T03:49:12.748 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-06-06T03:49:12.749 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T03:49:12.749 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T03:49:12.749 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-06-06T03:49:12.749 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-06-06T03:49:12.749 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-06-06T03:49:12.749 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-06-06T03:49:12.749 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T03:49:12.749 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T03:49:12.750 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T03:49:12.750 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T03:49:12.750 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T03:49:12.750 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-06-06T03:49:12.750 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-06-06T03:49:12.750 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-06-06T03:49:12.750 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T03:49:12.750 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T03:49:12.751 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-06-06T03:49:12.751 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-06-06T03:49:12.751 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-06-06T03:49:12.751 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-06-06T03:49:12.751 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T03:49:12.751 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-06-06T03:49:12.751 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T03:49:12.751 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-06-06T03:49:12.751 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 1303121, 2024-06-06T03:49:12.752 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 26663, 2024-06-06T03:49:12.752 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 88, 2024-06-06T03:49:12.752 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 1, 2024-06-06T03:49:12.752 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-06-06T03:49:12.752 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-06-06T03:49:12.752 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-06-06T03:49:12.752 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-06-06T03:49:12.752 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-06-06T03:49:12.753 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-06-06T03:49:12.753 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 7864, 2024-06-06T03:49:12.753 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 164, 2024-06-06T03:49:12.753 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T03:49:12.753 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-06-06T03:49:12.753 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-06-06T03:49:12.753 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-06-06T03:49:12.753 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 91122742, 2024-06-06T03:49:12.753 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 1184057, 2024-06-06T03:49:12.754 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-06-06T03:49:12.754 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-06-06T03:49:12.754 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-06-06T03:49:12.754 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-06-06T03:49:12.754 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.754 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-06-06T03:49:12.754 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-06-06T03:49:12.755 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-06-06T03:49:12.755 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 1852, 2024-06-06T03:49:12.755 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 53456391, 2024-06-06T03:49:12.755 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 33, 2024-06-06T03:49:12.755 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 1852, 2024-06-06T03:49:12.755 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-06-06T03:49:12.755 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1852, 2024-06-06T03:49:12.755 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 11.793875452, 2024-06-06T03:49:12.755 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006368183 2024-06-06T03:49:12.756 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.756 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-06-06T03:49:12.756 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-06-06T03:49:12.756 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1852, 2024-06-06T03:49:12.756 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 2190 2024-06-06T03:49:12.756 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.756 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 1852, 2024-06-06T03:49:12.756 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 17, 2024-06-06T03:49:12.757 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 1835, 2024-06-06T03:49:12.757 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-06-06T03:49:12.757 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-06-06T03:49:12.757 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 0, 2024-06-06T03:49:12.757 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 10, 2024-06-06T03:49:12.757 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 17, 2024-06-06T03:49:12.757 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 1328, 2024-06-06T03:49:12.757 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 54, 2024-06-06T03:49:12.757 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-06-06T03:49:12.758 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-06-06T03:49:12.758 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-06-06T03:49:12.758 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 10, 2024-06-06T03:49:12.758 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 131, 2024-06-06T03:49:12.758 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-06-06T03:49:12.758 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-06-06T03:49:12.758 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-06-06T03:49:12.758 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 0, 2024-06-06T03:49:12.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 20, 2024-06-06T03:49:12.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-06-06T03:49:12.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-06-06T03:49:12.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-06-06T03:49:12.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-06-06T03:49:12.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-06-06T03:49:12.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-06-06T03:49:12.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T03:49:12.760 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-06-06T03:49:12.760 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-06-06T03:49:12.760 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 7, 2024-06-06T03:49:12.760 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-06-06T03:49:12.760 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-06-06T03:49:12.760 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-06-06T03:49:12.760 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-06-06T03:49:12.760 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-06-06T03:49:12.760 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-06-06T03:49:12.761 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-06-06T03:49:12.761 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-06-06T03:49:12.761 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-06-06T03:49:12.761 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-06-06T03:49:12.761 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-06-06T03:49:12.761 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-06-06T03:49:12.761 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-06-06T03:49:12.761 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-06-06T03:49:12.762 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-06-06T03:49:12.762 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-06-06T03:49:12.762 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 100, 2024-06-06T03:49:12.762 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-06-06T03:49:12.762 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 12, 2024-06-06T03:49:12.762 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2024-06-06T03:49:12.762 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2024-06-06T03:49:12.762 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-06-06T03:49:12.763 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-06-06T03:49:12.763 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 517, 2024-06-06T03:49:12.763 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 0, 2024-06-06T03:49:12.763 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 96 2024-06-06T03:49:12.763 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.763 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-06-06T03:49:12.763 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-06-06T03:49:12.763 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-06-06T03:49:12.763 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-06-06T03:49:12.764 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-06-06T03:49:12.764 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.764 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-06-06T03:49:12.764 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 78751 2024-06-06T03:49:12.764 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.764 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-06-06T03:49:12.764 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-06-06T03:49:12.764 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 6, 2024-06-06T03:49:12.765 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-06-06T03:49:12.765 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 6, 2024-06-06T03:49:12.765 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 107, 2024-06-06T03:49:12.765 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 107, 2024-06-06T03:49:12.765 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-06-06T03:49:12.765 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.765 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T03:49:12.765 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:12.766 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-06-06T03:49:12.766 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:12.766 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 67215, 2024-06-06T03:49:12.766 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 43412227, 2024-06-06T03:49:12.766 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:12.766 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 67215, 2024-06-06T03:49:12.766 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-06-06T03:49:12.766 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-06-06T03:49:12.767 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 67215, 2024-06-06T03:49:12.767 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 43412227, 2024-06-06T03:49:12.767 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:12.767 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.767 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.767 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.767 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:12.767 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.768 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-06-06T03:49:12.768 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:12.768 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-06-06T03:49:12.768 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:12.768 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-06-06T03:49:12.768 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-06-06T03:49:12.768 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:12.768 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:12.769 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 1852, 2024-06-06T03:49:12.769 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 53467198, 2024-06-06T03:49:12.769 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 1852, 2024-06-06T03:49:12.769 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 53467198, 2024-06-06T03:49:12.769 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:12.769 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.769 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.769 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.770 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:12.770 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.770 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-06-06T03:49:12.770 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:12.770 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-06-06T03:49:12.770 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:12.770 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-06-06T03:49:12.770 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-06-06T03:49:12.771 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:12.771 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:12.771 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 1852, 2024-06-06T03:49:12.771 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 1852, 2024-06-06T03:49:12.771 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 1852, 2024-06-06T03:49:12.771 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1852, 2024-06-06T03:49:12.771 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:12.771 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.771 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.772 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.772 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:12.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.772 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-06-06T03:49:12.772 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:12.772 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-06-06T03:49:12.772 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:12.772 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 107, 2024-06-06T03:49:12.773 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 10807, 2024-06-06T03:49:12.773 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:12.773 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 107, 2024-06-06T03:49:12.773 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-06-06T03:49:12.773 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-06-06T03:49:12.773 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 17, 2024-06-06T03:49:12.773 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 10807, 2024-06-06T03:49:12.773 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:12.773 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.774 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.774 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.774 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:12.774 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:12.774 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55575f6fc500": { 2024-06-06T03:49:12.774 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:12.774 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-06-06T03:49:12.774 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:12.775 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 15071, 2024-06-06T03:49:12.775 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 50974116, 2024-06-06T03:49:12.775 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:12.775 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 15071, 2024-06-06T03:49:12.775 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-06-06T03:49:12.775 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-06-06T03:49:12.775 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 1311, 2024-06-06T03:49:12.775 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 50974116, 2024-06-06T03:49:12.776 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:12.776 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:12.776 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:12.776 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:12.776 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:12.776 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:12.776 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-06-06T03:49:13.261 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:12 smithi104 ceph-mon[36569]: pgmap v329: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 195 KiB/s wr, 35 op/s 2024-06-06T03:49:13.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:12 smithi131 ceph-mon[36687]: pgmap v329: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 195 KiB/s wr, 35 op/s 2024-06-06T03:49:13.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:12 smithi040 ceph-mon[32220]: pgmap v329: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 195 KiB/s wr, 35 op/s 2024-06-06T03:49:13.437 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-06-06T03:49:13.437 INFO:tasks.check_counter:Daemon mds.e mds.imported=65 2024-06-06T03:49:13.438 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-06-06T03:49:13.438 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-06-06T03:49:13.438 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T03:49:13.438 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=10 2024-06-06T03:49:13.438 DEBUG:tasks.check_counter:Getting stats from h 2024-06-06T03:49:13.438 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-06T03:49:13.679 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T03:49:15.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:14 smithi040 ceph-mon[32220]: pgmap v330: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 195 KiB/s wr, 35 op/s 2024-06-06T03:49:15.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:14 smithi040 ceph-mon[32220]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T03:49:15.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:14 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:15.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:14 smithi104 ceph-mon[36569]: pgmap v330: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 195 KiB/s wr, 35 op/s 2024-06-06T03:49:15.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:14 smithi104 ceph-mon[36569]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T03:49:15.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:14 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:15.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:14 smithi131 ceph-mon[36687]: pgmap v330: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 195 KiB/s wr, 35 op/s 2024-06-06T03:49:15.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:14 smithi131 ceph-mon[36687]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T03:49:15.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:14 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:15.439 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-06-06T03:49:15.439 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T03:49:15.439 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 91214, 2024-06-06T03:49:15.439 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 145009, 2024-06-06T03:49:15.439 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 59266337, 2024-06-06T03:49:15.440 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 106646518, 2024-06-06T03:49:15.440 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 13, 2024-06-06T03:49:15.440 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-06-06T03:49:15.440 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 4.801146659, 2024-06-06T03:49:15.440 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 2.219958188, 2024-06-06T03:49:15.440 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 9.253725452, 2024-06-06T03:49:15.440 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.216974226, 2024-06-06T03:49:15.440 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:15.441 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 145006, 2024-06-06T03:49:15.441 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 8.344223484, 2024-06-06T03:49:15.441 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000057543 2024-06-06T03:49:15.441 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.441 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:15.441 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 175972, 2024-06-06T03:49:15.441 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.035120960, 2024-06-06T03:49:15.441 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000199 2024-06-06T03:49:15.441 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.442 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1775792, 2024-06-06T03:49:15.442 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 154048 2024-06-06T03:49:15.442 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.442 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T03:49:15.442 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 3163, 2024-06-06T03:49:15.442 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 3182, 2024-06-06T03:49:15.442 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 5142960, 2024-06-06T03:49:15.442 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 43720674, 2024-06-06T03:49:15.443 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 18, 2024-06-06T03:49:15.443 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-06-06T03:49:15.443 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.330615292, 2024-06-06T03:49:15.443 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.194228176, 2024-06-06T03:49:15.443 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.095818104, 2024-06-06T03:49:15.443 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.046407871, 2024-06-06T03:49:15.443 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:15.443 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3181, 2024-06-06T03:49:15.443 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.319492680, 2024-06-06T03:49:15.444 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000100437 2024-06-06T03:49:15.444 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.444 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:15.444 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3819, 2024-06-06T03:49:15.444 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000611202, 2024-06-06T03:49:15.444 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000160 2024-06-06T03:49:15.444 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.444 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 2176, 2024-06-06T03:49:15.444 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 472560 2024-06-06T03:49:15.445 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.445 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T03:49:15.445 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 2676, 2024-06-06T03:49:15.445 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 2717, 2024-06-06T03:49:15.445 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 804064, 2024-06-06T03:49:15.445 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 33699590, 2024-06-06T03:49:15.445 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 9, 2024-06-06T03:49:15.445 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-06-06T03:49:15.445 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.351813241, 2024-06-06T03:49:15.446 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.190736655, 2024-06-06T03:49:15.446 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.059519189, 2024-06-06T03:49:15.446 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.088160933, 2024-06-06T03:49:15.446 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:15.446 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2716, 2024-06-06T03:49:15.446 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.280627674, 2024-06-06T03:49:15.446 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000103323 2024-06-06T03:49:15.446 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.447 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:15.447 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 434, 2024-06-06T03:49:15.447 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000105239, 2024-06-06T03:49:15.447 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000242 2024-06-06T03:49:15.447 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.447 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-06-06T03:49:15.447 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T03:49:15.447 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.447 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-06-06T03:49:15.448 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-06-06T03:49:15.448 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-06-06T03:49:15.448 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.448 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-06-06T03:49:15.448 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-06-06T03:49:15.448 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-06-06T03:49:15.448 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4818, 2024-06-06T03:49:15.448 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 35.728580059, 2024-06-06T03:49:15.448 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.007415645 2024-06-06T03:49:15.449 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:15.449 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.449 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-06-06T03:49:15.449 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-06-06T03:49:15.449 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-06-06T03:49:15.449 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 452, 2024-06-06T03:49:15.449 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.048565139, 2024-06-06T03:49:15.449 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000107444 2024-06-06T03:49:15.450 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:15.450 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.450 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-06-06T03:49:15.450 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 11793, 2024-06-06T03:49:15.450 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 11291, 2024-06-06T03:49:15.450 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-06-06T03:49:15.450 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11291, 2024-06-06T03:49:15.450 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 207.198549944, 2024-06-06T03:49:15.450 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.018350770 2024-06-06T03:49:15.451 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.451 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-06-06T03:49:15.451 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 502, 2024-06-06T03:49:15.451 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 8, 2024-06-06T03:49:15.451 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-06-06T03:49:15.451 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 380, 2024-06-06T03:49:15.451 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 43, 2024-06-06T03:49:15.451 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-06-06T03:49:15.451 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 28730, 2024-06-06T03:49:15.452 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 1198, 2024-06-06T03:49:15.452 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 27461, 2024-06-06T03:49:15.452 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 71, 2024-06-06T03:49:15.452 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 27006, 2024-06-06T03:49:15.452 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 13873, 2024-06-06T03:49:15.452 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 798, 2024-06-06T03:49:15.452 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 798, 2024-06-06T03:49:15.452 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 70, 2024-06-06T03:49:15.453 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 14296, 2024-06-06T03:49:15.453 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 11703, 2024-06-06T03:49:15.453 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 476, 2024-06-06T03:49:15.453 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 81, 2024-06-06T03:49:15.453 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-06-06T03:49:15.453 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-06-06T03:49:15.453 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-06-06T03:49:15.453 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 1788, 2024-06-06T03:49:15.453 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-06-06T03:49:15.454 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 11, 2024-06-06T03:49:15.454 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 13657, 2024-06-06T03:49:15.454 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 59, 2024-06-06T03:49:15.454 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 29555, 2024-06-06T03:49:15.454 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-06-06T03:49:15.454 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-06-06T03:49:15.454 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-06-06T03:49:15.454 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 84812, 2024-06-06T03:49:15.454 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 1539542236, 2024-06-06T03:49:15.455 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 6, 2024-06-06T03:49:15.455 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T03:49:15.455 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-06-06T03:49:15.455 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T03:49:15.455 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-06-06T03:49:15.455 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-06-06T03:49:15.455 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T03:49:15.455 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T03:49:15.456 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-06-06T03:49:15.456 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 42, 2024-06-06T03:49:15.456 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 53717, 2024-06-06T03:49:15.456 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 11808, 2024-06-06T03:49:15.456 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 19, 2024-06-06T03:49:15.456 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T03:49:15.456 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T03:49:15.456 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 72326, 2024-06-06T03:49:15.456 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 22978, 2024-06-06T03:49:15.457 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 1740, 2024-06-06T03:49:15.457 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 2601 2024-06-06T03:49:15.457 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.457 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-06-06T03:49:15.457 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-06-06T03:49:15.457 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-06-06T03:49:15.457 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-06-06T03:49:15.457 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 163, 2024-06-06T03:49:15.457 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 27, 2024-06-06T03:49:15.458 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-06-06T03:49:15.458 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-06-06T03:49:15.458 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-06-06T03:49:15.458 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 187, 2024-06-06T03:49:15.458 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 187, 2024-06-06T03:49:15.458 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-06-06T03:49:15.458 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-06-06T03:49:15.458 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-06-06T03:49:15.459 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-06-06T03:49:15.459 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-06-06T03:49:15.459 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-06-06T03:49:15.459 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-06-06T03:49:15.459 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 7, 2024-06-06T03:49:15.459 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 112412, 2024-06-06T03:49:15.459 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T03:49:15.459 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 11, 2024-06-06T03:49:15.459 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-06-06T03:49:15.460 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 43, 2024-06-06T03:49:15.460 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-06-06T03:49:15.460 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-06-06T03:49:15.460 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.460 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-06-06T03:49:15.460 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 7, 2024-06-06T03:49:15.460 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 34426, 2024-06-06T03:49:15.460 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-06-06T03:49:15.460 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-06-06T03:49:15.461 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 34426, 2024-06-06T03:49:15.461 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-06-06T03:49:15.461 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-06-06T03:49:15.461 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 49, 2024-06-06T03:49:15.461 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-06-06T03:49:15.461 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-06-06T03:49:15.461 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 49, 2024-06-06T03:49:15.461 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 4, 2024-06-06T03:49:15.462 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-06-06T03:49:15.462 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-06-06T03:49:15.462 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-06-06T03:49:15.462 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 109974615, 2024-06-06T03:49:15.462 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-06-06T03:49:15.462 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-06-06T03:49:15.462 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4527, 2024-06-06T03:49:15.462 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 212.946901167, 2024-06-06T03:49:15.462 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.047039297 2024-06-06T03:49:15.463 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.463 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-06-06T03:49:15.463 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.463 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-06-06T03:49:15.463 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 28374, 2024-06-06T03:49:15.463 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 42218, 2024-06-06T03:49:15.463 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 13844, 2024-06-06T03:49:15.463 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 1727, 2024-06-06T03:49:15.464 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 2523, 2024-06-06T03:49:15.464 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 796, 2024-06-06T03:49:15.464 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 28730, 2024-06-06T03:49:15.464 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 43087, 2024-06-06T03:49:15.464 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 14357, 2024-06-06T03:49:15.464 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 798, 2024-06-06T03:49:15.464 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 53409, 2024-06-06T03:49:15.464 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 52611, 2024-06-06T03:49:15.464 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 222384, 2024-06-06T03:49:15.465 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 223612 2024-06-06T03:49:15.465 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.465 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-06-06T03:49:15.465 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 12329, 2024-06-06T03:49:15.465 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 28, 2024-06-06T03:49:15.465 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 11793, 2024-06-06T03:49:15.465 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 26, 2024-06-06T03:49:15.465 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 280, 2024-06-06T03:49:15.465 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-06-06T03:49:15.466 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 6024, 2024-06-06T03:49:15.466 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 45.859189715, 2024-06-06T03:49:15.466 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.007612747 2024-06-06T03:49:15.466 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.466 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-06-06T03:49:15.466 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4, 2024-06-06T03:49:15.466 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 3.815863314, 2024-06-06T03:49:15.466 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.953965828 2024-06-06T03:49:15.467 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.467 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-06-06T03:49:15.467 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.467 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.467 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.467 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.467 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-06-06T03:49:15.467 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.467 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.468 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.468 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.468 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-06-06T03:49:15.468 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2163, 2024-06-06T03:49:15.468 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 152.736685662, 2024-06-06T03:49:15.468 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.070613354 2024-06-06T03:49:15.468 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.468 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-06-06T03:49:15.469 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.469 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.469 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.469 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.469 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-06-06T03:49:15.469 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.469 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.469 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.469 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.470 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-06-06T03:49:15.470 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.470 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.470 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.470 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.470 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-06-06T03:49:15.470 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.470 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.471 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.471 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.471 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-06-06T03:49:15.471 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.471 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.472 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.472 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.472 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-06-06T03:49:15.472 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.472 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.472 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.472 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.473 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-06-06T03:49:15.473 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 220, 2024-06-06T03:49:15.473 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.197453855, 2024-06-06T03:49:15.473 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000897517 2024-06-06T03:49:15.473 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.473 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-06-06T03:49:15.473 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.473 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.473 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.474 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.474 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-06-06T03:49:15.474 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.474 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.474 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.474 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.474 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-06-06T03:49:15.474 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 689, 2024-06-06T03:49:15.475 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.957240250, 2024-06-06T03:49:15.475 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002840697 2024-06-06T03:49:15.475 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.475 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-06-06T03:49:15.475 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1981, 2024-06-06T03:49:15.475 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.678475168, 2024-06-06T03:49:15.475 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000847286 2024-06-06T03:49:15.475 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.475 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-06-06T03:49:15.476 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.476 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.476 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.476 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.476 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-06-06T03:49:15.476 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.476 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.476 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.476 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.477 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-06-06T03:49:15.477 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.477 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.477 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.477 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.477 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-06-06T03:49:15.477 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.477 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.477 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.478 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.478 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-06-06T03:49:15.478 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.478 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.478 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.478 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.478 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-06-06T03:49:15.478 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.479 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.479 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.479 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.479 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-06-06T03:49:15.479 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 23, 2024-06-06T03:49:15.479 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.444028510, 2024-06-06T03:49:15.479 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.019305587 2024-06-06T03:49:15.479 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.479 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-06-06T03:49:15.480 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.480 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.480 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.480 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.480 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-06-06T03:49:15.480 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.480 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.480 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.480 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.481 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-06-06T03:49:15.481 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.481 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.481 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.481 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.481 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-06-06T03:49:15.481 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.481 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.482 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.482 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.482 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-06-06T03:49:15.482 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.482 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.482 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.482 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.482 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-06-06T03:49:15.482 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 187, 2024-06-06T03:49:15.483 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.509613470, 2024-06-06T03:49:15.483 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002725205 2024-06-06T03:49:15.483 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.483 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-06-06T03:49:15.483 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-06-06T03:49:15.483 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-06-06T03:49:15.483 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.483 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.484 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.484 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:15.484 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.484 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-06-06T03:49:15.484 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 1, 2024-06-06T03:49:15.484 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 3, 2024-06-06T03:49:15.484 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 2, 2024-06-06T03:49:15.484 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 1, 2024-06-06T03:49:15.484 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-06-06T03:49:15.485 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 1550, 2024-06-06T03:49:15.485 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 1550, 2024-06-06T03:49:15.485 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 2046, 2024-06-06T03:49:15.485 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-06-06T03:49:15.485 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.485 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-06-06T03:49:15.485 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-06-06T03:49:15.485 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-06-06T03:49:15.486 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T03:49:15.486 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-06-06T03:49:15.486 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T03:49:15.486 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-06-06T03:49:15.486 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T03:49:15.486 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T03:49:15.486 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T03:49:15.486 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T03:49:15.486 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T03:49:15.487 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-06-06T03:49:15.487 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T03:49:15.487 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T03:49:15.487 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-06-06T03:49:15.487 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-06-06T03:49:15.487 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-06-06T03:49:15.487 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-06-06T03:49:15.487 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T03:49:15.487 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T03:49:15.488 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T03:49:15.488 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T03:49:15.488 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T03:49:15.488 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-06-06T03:49:15.488 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-06-06T03:49:15.488 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-06-06T03:49:15.488 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T03:49:15.488 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T03:49:15.489 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-06-06T03:49:15.489 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-06-06T03:49:15.489 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-06-06T03:49:15.489 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-06-06T03:49:15.489 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T03:49:15.489 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-06-06T03:49:15.489 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T03:49:15.489 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-06-06T03:49:15.489 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 1175464, 2024-06-06T03:49:15.490 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 23641, 2024-06-06T03:49:15.490 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-06-06T03:49:15.490 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-06-06T03:49:15.490 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-06-06T03:49:15.490 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-06-06T03:49:15.490 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-06-06T03:49:15.490 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-06-06T03:49:15.490 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-06-06T03:49:15.491 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-06-06T03:49:15.491 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 7864, 2024-06-06T03:49:15.491 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 164, 2024-06-06T03:49:15.491 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T03:49:15.491 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-06-06T03:49:15.491 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-06-06T03:49:15.491 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-06-06T03:49:15.491 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 89225698, 2024-06-06T03:49:15.491 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 1110632, 2024-06-06T03:49:15.492 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-06-06T03:49:15.492 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-06-06T03:49:15.492 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-06-06T03:49:15.492 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-06-06T03:49:15.492 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.492 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-06-06T03:49:15.492 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-06-06T03:49:15.492 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-06-06T03:49:15.493 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 5587, 2024-06-06T03:49:15.493 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 112800430, 2024-06-06T03:49:15.493 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 33, 2024-06-06T03:49:15.493 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 5587, 2024-06-06T03:49:15.493 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-06-06T03:49:15.493 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5587, 2024-06-06T03:49:15.493 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 72.293234068, 2024-06-06T03:49:15.493 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.012939544 2024-06-06T03:49:15.493 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.494 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-06-06T03:49:15.494 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-06-06T03:49:15.494 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5587, 2024-06-06T03:49:15.494 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 6289 2024-06-06T03:49:15.494 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.494 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 5587, 2024-06-06T03:49:15.494 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 49, 2024-06-06T03:49:15.494 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 5538, 2024-06-06T03:49:15.495 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-06-06T03:49:15.495 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-06-06T03:49:15.495 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 0, 2024-06-06T03:49:15.495 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 10, 2024-06-06T03:49:15.495 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 31, 2024-06-06T03:49:15.495 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 4558, 2024-06-06T03:49:15.495 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 65, 2024-06-06T03:49:15.495 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-06-06T03:49:15.495 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-06-06T03:49:15.496 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-06-06T03:49:15.496 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 27, 2024-06-06T03:49:15.496 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 240, 2024-06-06T03:49:15.496 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-06-06T03:49:15.496 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-06-06T03:49:15.496 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-06-06T03:49:15.496 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 0, 2024-06-06T03:49:15.496 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 20, 2024-06-06T03:49:15.496 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-06-06T03:49:15.497 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-06-06T03:49:15.497 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-06-06T03:49:15.497 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-06-06T03:49:15.497 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-06-06T03:49:15.497 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-06-06T03:49:15.497 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T03:49:15.497 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-06-06T03:49:15.497 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-06-06T03:49:15.497 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 19, 2024-06-06T03:49:15.498 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-06-06T03:49:15.498 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-06-06T03:49:15.498 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-06-06T03:49:15.498 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-06-06T03:49:15.498 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-06-06T03:49:15.498 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-06-06T03:49:15.498 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-06-06T03:49:15.498 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-06-06T03:49:15.499 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-06-06T03:49:15.499 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-06-06T03:49:15.499 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-06-06T03:49:15.499 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-06-06T03:49:15.499 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-06-06T03:49:15.499 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-06-06T03:49:15.499 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-06-06T03:49:15.499 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-06-06T03:49:15.499 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 101, 2024-06-06T03:49:15.500 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-06-06T03:49:15.500 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 13, 2024-06-06T03:49:15.500 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2024-06-06T03:49:15.500 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2024-06-06T03:49:15.500 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-06-06T03:49:15.500 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-06-06T03:49:15.500 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 1158, 2024-06-06T03:49:15.500 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 26, 2024-06-06T03:49:15.501 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 135 2024-06-06T03:49:15.501 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.501 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-06-06T03:49:15.501 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-06-06T03:49:15.501 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-06-06T03:49:15.501 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-06-06T03:49:15.501 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-06-06T03:49:15.501 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.501 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-06-06T03:49:15.502 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 106644 2024-06-06T03:49:15.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.502 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-06-06T03:49:15.502 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-06-06T03:49:15.502 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 6, 2024-06-06T03:49:15.502 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-06-06T03:49:15.502 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 6, 2024-06-06T03:49:15.502 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 187, 2024-06-06T03:49:15.503 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 187, 2024-06-06T03:49:15.503 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-06-06T03:49:15.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.503 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T03:49:15.503 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:15.503 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-06-06T03:49:15.503 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:15.503 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 97049, 2024-06-06T03:49:15.503 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 56459373, 2024-06-06T03:49:15.504 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:15.504 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 97049, 2024-06-06T03:49:15.504 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-06-06T03:49:15.504 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-06-06T03:49:15.504 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 97049, 2024-06-06T03:49:15.504 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 56459373, 2024-06-06T03:49:15.504 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:15.504 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.504 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.505 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.505 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:15.505 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.505 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-06-06T03:49:15.505 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:15.505 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-06-06T03:49:15.505 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:15.505 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-06-06T03:49:15.506 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-06-06T03:49:15.506 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:15.506 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:15.506 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 5587, 2024-06-06T03:49:15.506 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 112818834, 2024-06-06T03:49:15.506 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 5587, 2024-06-06T03:49:15.506 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 112818834, 2024-06-06T03:49:15.506 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:15.506 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.507 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.507 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.507 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:15.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.507 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-06-06T03:49:15.507 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:15.507 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-06-06T03:49:15.507 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:15.507 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-06-06T03:49:15.508 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-06-06T03:49:15.508 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:15.508 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:15.508 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 5587, 2024-06-06T03:49:15.508 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 5587, 2024-06-06T03:49:15.508 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 5587, 2024-06-06T03:49:15.508 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 5587, 2024-06-06T03:49:15.508 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:15.508 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.509 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.509 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.509 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:15.509 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.509 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-06-06T03:49:15.509 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:15.509 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-06-06T03:49:15.509 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:15.510 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 187, 2024-06-06T03:49:15.510 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 18895, 2024-06-06T03:49:15.510 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:15.510 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 187, 2024-06-06T03:49:15.510 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-06-06T03:49:15.510 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-06-06T03:49:15.510 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 31, 2024-06-06T03:49:15.510 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 18895, 2024-06-06T03:49:15.511 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:15.511 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.511 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.511 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.511 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:15.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:15.511 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x558e4c022500": { 2024-06-06T03:49:15.511 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:15.511 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-06-06T03:49:15.512 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:15.512 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 34426, 2024-06-06T03:49:15.512 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 105780311, 2024-06-06T03:49:15.512 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:15.512 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 34426, 2024-06-06T03:49:15.512 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-06-06T03:49:15.512 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-06-06T03:49:15.512 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 4527, 2024-06-06T03:49:15.512 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 105780311, 2024-06-06T03:49:15.513 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:15.513 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:15.513 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:15.513 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:15.513 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:15.513 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:15.513 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-06-06T03:49:16.312 INFO:tasks.check_counter:Daemon mds.h mds.exported=11 2024-06-06T03:49:16.312 INFO:tasks.check_counter:Daemon mds.h mds.imported=59 2024-06-06T03:49:16.312 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-06-06T03:49:16.312 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-06-06T03:49:16.312 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T03:49:16.312 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=43 2024-06-06T03:49:16.312 DEBUG:tasks.check_counter:Getting stats from k 2024-06-06T03:49:16.313 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-06T03:49:16.556 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T03:49:16.981 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:16 smithi104 ceph-mon[36569]: pgmap v332: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 150 KiB/s wr, 13 op/s 2024-06-06T03:49:17.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:16 smithi040 ceph-mon[32220]: pgmap v332: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 150 KiB/s wr, 13 op/s 2024-06-06T03:49:17.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:16 smithi131 ceph-mon[36687]: pgmap v332: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 150 KiB/s wr, 13 op/s 2024-06-06T03:49:18.366 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-06-06T03:49:18.366 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T03:49:18.367 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1400, 2024-06-06T03:49:18.367 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 2105, 2024-06-06T03:49:18.367 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 4467707, 2024-06-06T03:49:18.367 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 28316407, 2024-06-06T03:49:18.367 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 18, 2024-06-06T03:49:18.367 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-06-06T03:49:18.367 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.240103463, 2024-06-06T03:49:18.367 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.147268438, 2024-06-06T03:49:18.368 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.052274460, 2024-06-06T03:49:18.368 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.039134580, 2024-06-06T03:49:18.368 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:18.368 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2100, 2024-06-06T03:49:18.368 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.176087391, 2024-06-06T03:49:18.368 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000083851 2024-06-06T03:49:18.368 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.368 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:18.369 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1631, 2024-06-06T03:49:18.369 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000380600, 2024-06-06T03:49:18.369 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000233 2024-06-06T03:49:18.369 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.369 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-06-06T03:49:18.369 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 317680 2024-06-06T03:49:18.369 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.369 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T03:49:18.370 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 74303, 2024-06-06T03:49:18.370 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 123526, 2024-06-06T03:49:18.370 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 42608964, 2024-06-06T03:49:18.370 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 81030455, 2024-06-06T03:49:18.370 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 11, 2024-06-06T03:49:18.370 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 7, 2024-06-06T03:49:18.370 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 3.972783745, 2024-06-06T03:49:18.371 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 1.826280132, 2024-06-06T03:49:18.371 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 4.636952794, 2024-06-06T03:49:18.371 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.174379849, 2024-06-06T03:49:18.371 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:18.371 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 123526, 2024-06-06T03:49:18.371 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 8.998607071, 2024-06-06T03:49:18.371 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000072847 2024-06-06T03:49:18.371 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.371 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:18.372 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 148626, 2024-06-06T03:49:18.372 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.030582586, 2024-06-06T03:49:18.372 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000205 2024-06-06T03:49:18.372 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.372 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T03:49:18.372 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 152752 2024-06-06T03:49:18.372 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.372 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T03:49:18.373 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 3379, 2024-06-06T03:49:18.373 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 3316, 2024-06-06T03:49:18.373 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 2641871, 2024-06-06T03:49:18.373 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 32301453, 2024-06-06T03:49:18.373 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 10, 2024-06-06T03:49:18.373 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 9, 2024-06-06T03:49:18.373 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.429270500, 2024-06-06T03:49:18.373 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.187290898, 2024-06-06T03:49:18.374 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.106373914, 2024-06-06T03:49:18.374 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.133041693, 2024-06-06T03:49:18.374 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:18.374 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3316, 2024-06-06T03:49:18.374 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.280885784, 2024-06-06T03:49:18.374 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000084706 2024-06-06T03:49:18.374 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.374 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:18.375 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 950, 2024-06-06T03:49:18.375 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000194221, 2024-06-06T03:49:18.375 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000204 2024-06-06T03:49:18.375 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.375 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1762768, 2024-06-06T03:49:18.375 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 156512 2024-06-06T03:49:18.375 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.375 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-06-06T03:49:18.376 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-06-06T03:49:18.376 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-06-06T03:49:18.376 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.376 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-06-06T03:49:18.376 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-06-06T03:49:18.376 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-06-06T03:49:18.376 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5641, 2024-06-06T03:49:18.376 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 21.768632171, 2024-06-06T03:49:18.377 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003859002 2024-06-06T03:49:18.377 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:18.377 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.377 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-06-06T03:49:18.377 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-06-06T03:49:18.377 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-06-06T03:49:18.377 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 463, 2024-06-06T03:49:18.377 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.035938233, 2024-06-06T03:49:18.377 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000077620 2024-06-06T03:49:18.378 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:18.378 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.378 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-06-06T03:49:18.378 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 10996, 2024-06-06T03:49:18.378 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 10693, 2024-06-06T03:49:18.378 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-06-06T03:49:18.378 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 10693, 2024-06-06T03:49:18.379 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 144.764165394, 2024-06-06T03:49:18.379 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.013538218 2024-06-06T03:49:18.379 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.379 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-06-06T03:49:18.379 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 303, 2024-06-06T03:49:18.379 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-06-06T03:49:18.379 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-06-06T03:49:18.379 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 183, 2024-06-06T03:49:18.379 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 20, 2024-06-06T03:49:18.380 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-06-06T03:49:18.380 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 15253, 2024-06-06T03:49:18.380 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 263, 2024-06-06T03:49:18.380 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 14039, 2024-06-06T03:49:18.380 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 951, 2024-06-06T03:49:18.380 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 14862, 2024-06-06T03:49:18.380 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 14128, 2024-06-06T03:49:18.380 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 1326, 2024-06-06T03:49:18.381 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 1326, 2024-06-06T03:49:18.381 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 100, 2024-06-06T03:49:18.381 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 14009, 2024-06-06T03:49:18.381 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 11912, 2024-06-06T03:49:18.381 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 266, 2024-06-06T03:49:18.381 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 138, 2024-06-06T03:49:18.381 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-06-06T03:49:18.381 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-06-06T03:49:18.382 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-06-06T03:49:18.382 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 4764, 2024-06-06T03:49:18.382 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-06-06T03:49:18.382 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 24, 2024-06-06T03:49:18.382 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 14002, 2024-06-06T03:49:18.382 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 182, 2024-06-06T03:49:18.382 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 20180, 2024-06-06T03:49:18.382 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-06-06T03:49:18.383 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-06-06T03:49:18.383 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-06-06T03:49:18.383 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 84812, 2024-06-06T03:49:18.383 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 1539542236, 2024-06-06T03:49:18.383 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 6, 2024-06-06T03:49:18.383 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T03:49:18.383 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-06-06T03:49:18.383 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T03:49:18.384 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-06-06T03:49:18.384 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-06-06T03:49:18.384 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T03:49:18.384 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T03:49:18.384 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-06-06T03:49:18.384 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 85, 2024-06-06T03:49:18.384 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 39859, 2024-06-06T03:49:18.384 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 10658, 2024-06-06T03:49:18.385 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 19, 2024-06-06T03:49:18.385 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T03:49:18.385 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T03:49:18.385 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 53899, 2024-06-06T03:49:18.385 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 16123, 2024-06-06T03:49:18.385 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 2778, 2024-06-06T03:49:18.385 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 2370 2024-06-06T03:49:18.385 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.386 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-06-06T03:49:18.386 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-06-06T03:49:18.386 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-06-06T03:49:18.386 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-06-06T03:49:18.386 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 247, 2024-06-06T03:49:18.386 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 116, 2024-06-06T03:49:18.386 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-06-06T03:49:18.386 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-06-06T03:49:18.387 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-06-06T03:49:18.387 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 193, 2024-06-06T03:49:18.387 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 193, 2024-06-06T03:49:18.387 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-06-06T03:49:18.388 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-06-06T03:49:18.388 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-06-06T03:49:18.388 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-06-06T03:49:18.388 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-06-06T03:49:18.388 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-06-06T03:49:18.388 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-06-06T03:49:18.388 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 7, 2024-06-06T03:49:18.388 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 67846, 2024-06-06T03:49:18.389 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T03:49:18.389 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 24, 2024-06-06T03:49:18.389 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-06-06T03:49:18.389 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 20, 2024-06-06T03:49:18.389 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-06-06T03:49:18.389 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-06-06T03:49:18.389 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.389 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-06-06T03:49:18.389 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 6, 2024-06-06T03:49:18.390 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 26634, 2024-06-06T03:49:18.390 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-06-06T03:49:18.390 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-06-06T03:49:18.390 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 26634, 2024-06-06T03:49:18.390 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-06-06T03:49:18.390 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-06-06T03:49:18.390 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 36, 2024-06-06T03:49:18.390 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-06-06T03:49:18.391 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-06-06T03:49:18.391 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 36, 2024-06-06T03:49:18.391 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 4, 2024-06-06T03:49:18.391 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-06-06T03:49:18.391 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-06-06T03:49:18.391 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-06-06T03:49:18.391 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 80927188, 2024-06-06T03:49:18.391 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-06-06T03:49:18.392 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-06-06T03:49:18.392 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3984, 2024-06-06T03:49:18.392 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 79.026584878, 2024-06-06T03:49:18.392 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.019835990 2024-06-06T03:49:18.392 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.392 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-06-06T03:49:18.392 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.392 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-06-06T03:49:18.393 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 14843, 2024-06-06T03:49:18.393 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 29038, 2024-06-06T03:49:18.393 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 14195, 2024-06-06T03:49:18.393 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 948, 2024-06-06T03:49:18.393 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 1936, 2024-06-06T03:49:18.393 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 988, 2024-06-06T03:49:18.393 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 15253, 2024-06-06T03:49:18.393 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 29797, 2024-06-06T03:49:18.393 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 14544, 2024-06-06T03:49:18.394 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 1252, 2024-06-06T03:49:18.394 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 43074, 2024-06-06T03:49:18.394 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 41822, 2024-06-06T03:49:18.394 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 176000, 2024-06-06T03:49:18.394 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 223612 2024-06-06T03:49:18.394 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.394 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-06-06T03:49:18.394 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 11379, 2024-06-06T03:49:18.395 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 25, 2024-06-06T03:49:18.395 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 10996, 2024-06-06T03:49:18.395 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 33, 2024-06-06T03:49:18.395 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 268, 2024-06-06T03:49:18.395 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-06-06T03:49:18.395 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4900, 2024-06-06T03:49:18.395 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 64.889273576, 2024-06-06T03:49:18.395 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.013242708 2024-06-06T03:49:18.396 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.396 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-06-06T03:49:18.396 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 26, 2024-06-06T03:49:18.396 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 51.167701762, 2024-06-06T03:49:18.396 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 1.967988529 2024-06-06T03:49:18.396 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.396 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-06-06T03:49:18.396 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.396 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.397 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.397 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.397 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-06-06T03:49:18.397 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.397 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.397 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.397 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.397 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-06-06T03:49:18.398 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2562, 2024-06-06T03:49:18.398 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 25.898663742, 2024-06-06T03:49:18.398 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.010108768 2024-06-06T03:49:18.398 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.398 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-06-06T03:49:18.398 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.398 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.398 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.399 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.399 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-06-06T03:49:18.399 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.399 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.399 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.399 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.399 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-06-06T03:49:18.399 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.400 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.400 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.400 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.400 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-06-06T03:49:18.400 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.400 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.400 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.400 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.400 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-06-06T03:49:18.401 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.401 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.401 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.402 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.402 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-06-06T03:49:18.402 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.402 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.402 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.402 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.402 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-06-06T03:49:18.403 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 312, 2024-06-06T03:49:18.403 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.316062733, 2024-06-06T03:49:18.403 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001013021 2024-06-06T03:49:18.403 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.403 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-06-06T03:49:18.403 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.403 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.403 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.404 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.404 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-06-06T03:49:18.404 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.404 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.404 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.404 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.404 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-06-06T03:49:18.404 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 989, 2024-06-06T03:49:18.404 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.474865502, 2024-06-06T03:49:18.405 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000480147 2024-06-06T03:49:18.405 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.405 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-06-06T03:49:18.405 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1678, 2024-06-06T03:49:18.405 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.296995910, 2024-06-06T03:49:18.405 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000772941 2024-06-06T03:49:18.405 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.405 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-06-06T03:49:18.406 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11, 2024-06-06T03:49:18.406 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.014655873, 2024-06-06T03:49:18.406 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001332352 2024-06-06T03:49:18.406 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.406 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-06-06T03:49:18.406 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.406 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.406 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.407 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.407 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-06-06T03:49:18.407 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.407 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.407 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.407 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.407 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-06-06T03:49:18.407 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.408 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.408 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.408 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.408 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-06-06T03:49:18.408 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.408 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.408 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.408 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.409 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-06-06T03:49:18.409 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.409 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.409 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.409 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.409 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-06-06T03:49:18.409 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 21, 2024-06-06T03:49:18.409 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.314152104, 2024-06-06T03:49:18.409 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.014959624 2024-06-06T03:49:18.410 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.410 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-06-06T03:49:18.410 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.410 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.410 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.410 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.410 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-06-06T03:49:18.410 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.411 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.411 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.411 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.411 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-06-06T03:49:18.411 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.411 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.411 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.411 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.412 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-06-06T03:49:18.412 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.412 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.412 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.412 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.412 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-06-06T03:49:18.412 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1, 2024-06-06T03:49:18.412 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000858730, 2024-06-06T03:49:18.413 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000858730 2024-06-06T03:49:18.413 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.413 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-06-06T03:49:18.413 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 193, 2024-06-06T03:49:18.413 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.390935462, 2024-06-06T03:49:18.413 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002025572 2024-06-06T03:49:18.413 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.413 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-06-06T03:49:18.414 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-06-06T03:49:18.414 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-06-06T03:49:18.414 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.414 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.414 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.414 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:18.414 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.414 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-06-06T03:49:18.414 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 1, 2024-06-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 2, 2024-06-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 1, 2024-06-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 1, 2024-06-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-06-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 2345, 2024-06-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 2345, 2024-06-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 639, 2024-06-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-06-06T03:49:18.416 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.416 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-06-06T03:49:18.416 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-06-06T03:49:18.416 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-06-06T03:49:18.416 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T03:49:18.416 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-06-06T03:49:18.416 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T03:49:18.416 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-06-06T03:49:18.416 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T03:49:18.417 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T03:49:18.417 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T03:49:18.417 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T03:49:18.417 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T03:49:18.417 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-06-06T03:49:18.417 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T03:49:18.417 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T03:49:18.417 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-06-06T03:49:18.418 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-06-06T03:49:18.418 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-06-06T03:49:18.418 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-06-06T03:49:18.418 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T03:49:18.418 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T03:49:18.418 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T03:49:18.418 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T03:49:18.418 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-06-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-06-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-06-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-06-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-06-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-06-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-06-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T03:49:18.420 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-06-06T03:49:18.420 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T03:49:18.420 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-06-06T03:49:18.420 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 732317, 2024-06-06T03:49:18.420 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 12084, 2024-06-06T03:49:18.420 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 176, 2024-06-06T03:49:18.421 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 2, 2024-06-06T03:49:18.421 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-06-06T03:49:18.421 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-06-06T03:49:18.421 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-06-06T03:49:18.421 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-06-06T03:49:18.421 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-06-06T03:49:18.421 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-06-06T03:49:18.421 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 7744, 2024-06-06T03:49:18.421 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 161, 2024-06-06T03:49:18.422 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T03:49:18.422 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-06-06T03:49:18.422 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-06-06T03:49:18.422 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-06-06T03:49:18.422 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 47347075, 2024-06-06T03:49:18.422 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 576603, 2024-06-06T03:49:18.422 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-06-06T03:49:18.423 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-06-06T03:49:18.423 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-06-06T03:49:18.423 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-06-06T03:49:18.423 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.423 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-06-06T03:49:18.423 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-06-06T03:49:18.423 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-06-06T03:49:18.423 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 4804, 2024-06-06T03:49:18.424 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 80735107, 2024-06-06T03:49:18.424 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 33, 2024-06-06T03:49:18.424 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 4804, 2024-06-06T03:49:18.424 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-06-06T03:49:18.424 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4804, 2024-06-06T03:49:18.424 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 54.789418872, 2024-06-06T03:49:18.424 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.011404958 2024-06-06T03:49:18.424 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.424 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-06-06T03:49:18.425 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-06-06T03:49:18.425 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4804, 2024-06-06T03:49:18.425 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 5287 2024-06-06T03:49:18.425 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.425 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 4804, 2024-06-06T03:49:18.425 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 32, 2024-06-06T03:49:18.425 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 4772, 2024-06-06T03:49:18.425 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-06-06T03:49:18.426 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-06-06T03:49:18.426 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 0, 2024-06-06T03:49:18.426 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 10, 2024-06-06T03:49:18.426 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 32, 2024-06-06T03:49:18.426 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 4016, 2024-06-06T03:49:18.426 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 61, 2024-06-06T03:49:18.426 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-06-06T03:49:18.426 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-06-06T03:49:18.427 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-06-06T03:49:18.427 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 12, 2024-06-06T03:49:18.427 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 231, 2024-06-06T03:49:18.427 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-06-06T03:49:18.427 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-06-06T03:49:18.427 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-06-06T03:49:18.427 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 0, 2024-06-06T03:49:18.428 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 20, 2024-06-06T03:49:18.428 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-06-06T03:49:18.428 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-06-06T03:49:18.428 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-06-06T03:49:18.428 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-06-06T03:49:18.428 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-06-06T03:49:18.428 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-06-06T03:49:18.428 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T03:49:18.429 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-06-06T03:49:18.429 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-06-06T03:49:18.429 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 19, 2024-06-06T03:49:18.429 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-06-06T03:49:18.429 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-06-06T03:49:18.429 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-06-06T03:49:18.429 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-06-06T03:49:18.429 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-06-06T03:49:18.429 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-06-06T03:49:18.430 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-06-06T03:49:18.430 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-06-06T03:49:18.430 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-06-06T03:49:18.430 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-06-06T03:49:18.430 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-06-06T03:49:18.430 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-06-06T03:49:18.430 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-06-06T03:49:18.430 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-06-06T03:49:18.430 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-06-06T03:49:18.431 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-06-06T03:49:18.431 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 101, 2024-06-06T03:49:18.431 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-06-06T03:49:18.431 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 13, 2024-06-06T03:49:18.431 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2024-06-06T03:49:18.431 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2024-06-06T03:49:18.431 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-06-06T03:49:18.431 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-06-06T03:49:18.432 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 770, 2024-06-06T03:49:18.432 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 0, 2024-06-06T03:49:18.432 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 116 2024-06-06T03:49:18.432 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.432 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-06-06T03:49:18.432 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-06-06T03:49:18.432 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-06-06T03:49:18.432 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-06-06T03:49:18.433 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-06-06T03:49:18.433 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.433 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-06-06T03:49:18.433 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 67559 2024-06-06T03:49:18.433 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.433 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-06-06T03:49:18.433 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-06-06T03:49:18.433 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 6, 2024-06-06T03:49:18.434 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-06-06T03:49:18.434 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 6, 2024-06-06T03:49:18.434 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 193, 2024-06-06T03:49:18.434 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 193, 2024-06-06T03:49:18.434 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-06-06T03:49:18.434 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.434 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T03:49:18.434 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:18.435 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-06-06T03:49:18.435 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:18.435 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 79078, 2024-06-06T03:49:18.435 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 42582207, 2024-06-06T03:49:18.435 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:18.435 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 79078, 2024-06-06T03:49:18.435 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-06-06T03:49:18.435 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-06-06T03:49:18.435 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 79078, 2024-06-06T03:49:18.436 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 42582207, 2024-06-06T03:49:18.436 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:18.436 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.436 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.436 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.436 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:18.436 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.436 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-06-06T03:49:18.437 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:18.437 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-06-06T03:49:18.437 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:18.437 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-06-06T03:49:18.437 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-06-06T03:49:18.437 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:18.437 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:18.437 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 4804, 2024-06-06T03:49:18.438 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 80754632, 2024-06-06T03:49:18.438 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 4804, 2024-06-06T03:49:18.438 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 80754632, 2024-06-06T03:49:18.438 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:18.438 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.438 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.438 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.438 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:18.439 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.439 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-06-06T03:49:18.439 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:18.439 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-06-06T03:49:18.439 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:18.439 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-06-06T03:49:18.439 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-06-06T03:49:18.439 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:18.439 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:18.440 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 4804, 2024-06-06T03:49:18.440 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 4804, 2024-06-06T03:49:18.440 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 4804, 2024-06-06T03:49:18.440 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 4804, 2024-06-06T03:49:18.440 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:18.440 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.440 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.440 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.440 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:18.441 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.441 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-06-06T03:49:18.441 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:18.441 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-06-06T03:49:18.441 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:18.441 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 193, 2024-06-06T03:49:18.441 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 19525, 2024-06-06T03:49:18.441 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:18.442 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 193, 2024-06-06T03:49:18.442 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-06-06T03:49:18.442 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-06-06T03:49:18.442 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 32, 2024-06-06T03:49:18.442 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 19525, 2024-06-06T03:49:18.442 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:18.442 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.442 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.443 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.443 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:18.443 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-06-06T03:49:18.443 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x5650389b4500": { 2024-06-06T03:49:18.443 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-06-06T03:49:18.443 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-06-06T03:49:18.443 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-06-06T03:49:18.443 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 26634, 2024-06-06T03:49:18.443 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 76732884, 2024-06-06T03:49:18.444 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:18.444 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 26634, 2024-06-06T03:49:18.444 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-06-06T03:49:18.444 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-06-06T03:49:18.444 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 3984, 2024-06-06T03:49:18.444 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 76732884, 2024-06-06T03:49:18.444 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-06-06T03:49:18.444 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-06-06T03:49:18.444 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-06-06T03:49:18.445 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-06-06T03:49:18.445 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:18.445 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-06-06T03:49:18.445 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-06-06T03:49:19.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:18 smithi040 ceph-mon[32220]: pgmap v333: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 133 KiB/s wr, 13 op/s 2024-06-06T03:49:19.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:18 smithi104 ceph-mon[36569]: pgmap v333: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 133 KiB/s wr, 13 op/s 2024-06-06T03:49:19.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:18 smithi131 ceph-mon[36687]: pgmap v333: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 133 KiB/s wr, 13 op/s 2024-06-06T03:49:19.367 INFO:tasks.check_counter:Daemon mds.k mds.exported=24 2024-06-06T03:49:19.367 INFO:tasks.check_counter:Daemon mds.k mds.imported=182 2024-06-06T03:49:19.367 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-06-06T03:49:19.367 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-06-06T03:49:19.367 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T03:49:19.367 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=20 2024-06-06T03:49:19.367 DEBUG:tasks.check_counter:Getting stats from c 2024-06-06T03:49:19.368 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-06T03:49:19.618 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:49:19.999 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:19 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:19 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:20.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:19 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:21.167 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:20 smithi131 ceph-mon[36687]: pgmap v334: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 13 op/s 2024-06-06T03:49:21.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:20 smithi040 ceph-mon[32220]: pgmap v334: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 13 op/s 2024-06-06T03:49:21.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:20 smithi104 ceph-mon[36569]: pgmap v334: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 13 op/s 2024-06-06T03:49:21.422 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-06-06T03:49:21.423 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T03:49:21.423 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1923, 2024-06-06T03:49:21.423 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1889, 2024-06-06T03:49:21.423 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 119692539, 2024-06-06T03:49:21.423 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 667730, 2024-06-06T03:49:21.423 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 7, 2024-06-06T03:49:21.423 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-06-06T03:49:21.424 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.264771560, 2024-06-06T03:49:21.424 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.059637111, 2024-06-06T03:49:21.424 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.144775302, 2024-06-06T03:49:21.424 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.056284618, 2024-06-06T03:49:21.424 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:21.424 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1889, 2024-06-06T03:49:21.424 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.071559089, 2024-06-06T03:49:21.424 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000037881 2024-06-06T03:49:21.424 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.425 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:21.425 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.425 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.425 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.425 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.425 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1756160, 2024-06-06T03:49:21.425 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 148432 2024-06-06T03:49:21.425 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.426 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T03:49:21.426 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 545, 2024-06-06T03:49:21.426 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 723, 2024-06-06T03:49:21.426 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 99446228, 2024-06-06T03:49:21.426 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 643700, 2024-06-06T03:49:21.426 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 9, 2024-06-06T03:49:21.426 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 7, 2024-06-06T03:49:21.426 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.109004068, 2024-06-06T03:49:21.427 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.021133465, 2024-06-06T03:49:21.427 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.068812146, 2024-06-06T03:49:21.427 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.013441011, 2024-06-06T03:49:21.427 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:21.427 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 720, 2024-06-06T03:49:21.427 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.032606922, 2024-06-06T03:49:21.427 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000045287 2024-06-06T03:49:21.427 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.428 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:21.428 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.428 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.428 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.428 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.428 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-06T03:49:21.428 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 470960 2024-06-06T03:49:21.428 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.428 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T03:49:21.429 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1059, 2024-06-06T03:49:21.429 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1060, 2024-06-06T03:49:21.429 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 138705794, 2024-06-06T03:49:21.429 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 336451, 2024-06-06T03:49:21.429 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 6, 2024-06-06T03:49:21.429 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 5, 2024-06-06T03:49:21.429 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.153262558, 2024-06-06T03:49:21.429 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.028057055, 2024-06-06T03:49:21.430 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.089969443, 2024-06-06T03:49:21.430 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.027663027, 2024-06-06T03:49:21.430 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:21.430 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1058, 2024-06-06T03:49:21.430 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.035650303, 2024-06-06T03:49:21.430 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000033695 2024-06-06T03:49:21.430 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.430 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:21.431 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.431 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.431 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.431 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.431 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T03:49:21.431 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T03:49:21.431 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.431 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-06-06T03:49:21.432 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-06-06T03:49:21.432 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-06-06T03:49:21.432 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.432 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-06-06T03:49:21.432 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-06-06T03:49:21.432 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-06-06T03:49:21.432 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 5150, 2024-06-06T03:49:21.432 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.134416452, 2024-06-06T03:49:21.433 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000026100 2024-06-06T03:49:21.433 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:21.433 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.433 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-06-06T03:49:21.433 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-06-06T03:49:21.433 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-06-06T03:49:21.433 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.433 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.433 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.434 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:21.434 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.434 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-06-06T03:49:21.434 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-06-06T03:49:21.434 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-06-06T03:49:21.434 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-06-06T03:49:21.434 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.434 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.435 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.435 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.435 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-06-06T03:49:21.435 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-06-06T03:49:21.435 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-06-06T03:49:21.435 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-06-06T03:49:21.435 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 0, 2024-06-06T03:49:21.435 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-06-06T03:49:21.435 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-06-06T03:49:21.436 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 18324, 2024-06-06T03:49:21.436 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 154, 2024-06-06T03:49:21.436 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 18170, 2024-06-06T03:49:21.436 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-06-06T03:49:21.436 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 18105, 2024-06-06T03:49:21.436 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-06-06T03:49:21.436 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-06-06T03:49:21.436 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-06-06T03:49:21.436 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 105, 2024-06-06T03:49:21.437 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-06-06T03:49:21.437 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-06-06T03:49:21.437 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-06-06T03:49:21.437 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-06-06T03:49:21.437 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-06-06T03:49:21.437 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-06-06T03:49:21.437 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-06-06T03:49:21.437 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-06-06T03:49:21.437 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-06-06T03:49:21.438 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-06-06T03:49:21.438 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-06-06T03:49:21.438 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-06-06T03:49:21.438 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-06-06T03:49:21.438 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-06-06T03:49:21.438 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-06-06T03:49:21.438 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-06-06T03:49:21.438 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 84951, 2024-06-06T03:49:21.439 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 1552868865, 2024-06-06T03:49:21.439 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 6, 2024-06-06T03:49:21.439 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T03:49:21.439 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-06-06T03:49:21.439 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T03:49:21.439 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-06-06T03:49:21.439 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-06-06T03:49:21.439 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T03:49:21.440 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T03:49:21.440 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-06-06T03:49:21.440 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-06-06T03:49:21.440 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T03:49:21.440 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-06-06T03:49:21.440 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T03:49:21.441 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T03:49:21.441 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T03:49:21.441 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-06-06T03:49:21.441 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-06-06T03:49:21.442 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-06-06T03:49:21.442 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-06-06T03:49:21.442 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.442 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-06-06T03:49:21.443 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-06-06T03:49:21.443 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-06-06T03:49:21.443 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-06-06T03:49:21.443 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 0, 2024-06-06T03:49:21.443 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-06-06T03:49:21.443 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-06-06T03:49:21.443 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-06-06T03:49:21.443 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-06-06T03:49:21.443 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-06-06T03:49:21.444 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-06-06T03:49:21.444 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-06-06T03:49:21.444 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-06-06T03:49:21.444 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-06-06T03:49:21.444 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-06-06T03:49:21.444 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-06-06T03:49:21.444 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-06-06T03:49:21.444 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-06-06T03:49:21.445 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-06-06T03:49:21.445 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-06-06T03:49:21.445 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T03:49:21.445 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-06-06T03:49:21.445 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-06-06T03:49:21.445 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-06-06T03:49:21.445 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-06-06T03:49:21.445 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-06-06T03:49:21.446 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.446 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-06-06T03:49:21.446 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-06-06T03:49:21.446 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 0, 2024-06-06T03:49:21.446 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-06-06T03:49:21.446 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-06-06T03:49:21.446 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 146237, 2024-06-06T03:49:21.446 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-06-06T03:49:21.447 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-06-06T03:49:21.447 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 0, 2024-06-06T03:49:21.447 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-06-06T03:49:21.447 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-06-06T03:49:21.447 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 132, 2024-06-06T03:49:21.447 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 13, 2024-06-06T03:49:21.447 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-06-06T03:49:21.447 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-06-06T03:49:21.448 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 85886036, 2024-06-06T03:49:21.448 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 359349042, 2024-06-06T03:49:21.448 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 359334606, 2024-06-06T03:49:21.448 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-06-06T03:49:21.448 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.448 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.448 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.448 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.448 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 146237 2024-06-06T03:49:21.449 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.449 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-06-06T03:49:21.449 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 17429, 2024-06-06T03:49:21.449 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 93357, 2024-06-06T03:49:21.449 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 75928, 2024-06-06T03:49:21.449 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 1264, 2024-06-06T03:49:21.449 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 6449, 2024-06-06T03:49:21.450 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 5185, 2024-06-06T03:49:21.450 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 18324, 2024-06-06T03:49:21.450 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 93964, 2024-06-06T03:49:21.450 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 75640, 2024-06-06T03:49:21.450 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-06-06T03:49:21.450 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-06-06T03:49:21.450 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-06-06T03:49:21.450 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 126556, 2024-06-06T03:49:21.451 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 215420 2024-06-06T03:49:21.451 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.451 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-06-06T03:49:21.451 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-06-06T03:49:21.451 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-06-06T03:49:21.451 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-06-06T03:49:21.451 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 0, 2024-06-06T03:49:21.452 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-06-06T03:49:21.452 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-06-06T03:49:21.452 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.452 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.452 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.452 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.452 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-06-06T03:49:21.453 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.453 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.453 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.453 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.453 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-06-06T03:49:21.453 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.453 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.453 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.454 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.454 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-06-06T03:49:21.454 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.454 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.454 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.454 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.454 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-06-06T03:49:21.454 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.455 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.455 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.455 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.455 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-06-06T03:49:21.455 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.455 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.455 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.455 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.456 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-06-06T03:49:21.456 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.456 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.456 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.456 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.456 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-06-06T03:49:21.456 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.456 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.457 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.457 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.457 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-06-06T03:49:21.457 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.457 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.457 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.457 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.457 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-06-06T03:49:21.458 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.458 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.458 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.458 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.458 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-06-06T03:49:21.458 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.458 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.459 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.459 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.459 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-06-06T03:49:21.460 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.460 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.460 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.460 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.460 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-06-06T03:49:21.460 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.460 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.460 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.461 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.461 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-06-06T03:49:21.461 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.461 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.461 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.461 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.461 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-06-06T03:49:21.461 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.462 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.462 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.462 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.462 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-06-06T03:49:21.462 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.462 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.462 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.463 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.463 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-06-06T03:49:21.463 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.463 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.463 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.463 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.463 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-06-06T03:49:21.463 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.464 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.464 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.464 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.464 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-06-06T03:49:21.464 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.464 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.464 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.464 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.465 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-06-06T03:49:21.465 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.465 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.465 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.465 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.465 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-06-06T03:49:21.465 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.465 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.465 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.466 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.466 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-06-06T03:49:21.466 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.466 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.466 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.466 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.466 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-06-06T03:49:21.466 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.467 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.467 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.467 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.467 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-06-06T03:49:21.467 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.467 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.467 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.467 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.468 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-06-06T03:49:21.468 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.468 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.468 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.468 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.468 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-06-06T03:49:21.468 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.468 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.469 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.469 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.469 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-06-06T03:49:21.469 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.469 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.469 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.469 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.470 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-06-06T03:49:21.470 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.470 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.470 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.471 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.471 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-06-06T03:49:21.471 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.471 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.471 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.471 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.472 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-06-06T03:49:21.472 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-06-06T03:49:21.472 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-06-06T03:49:21.472 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.472 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.472 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.472 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:21.472 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.473 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-06-06T03:49:21.473 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 2, 2024-06-06T03:49:21.473 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 5, 2024-06-06T03:49:21.473 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 3, 2024-06-06T03:49:21.473 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 4, 2024-06-06T03:49:21.473 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-06-06T03:49:21.473 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-06-06T03:49:21.473 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-06-06T03:49:21.474 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 3468, 2024-06-06T03:49:21.474 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-06-06T03:49:21.474 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.474 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-06-06T03:49:21.474 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-06-06T03:49:21.474 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-06-06T03:49:21.474 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T03:49:21.474 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-06-06T03:49:21.474 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T03:49:21.475 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-06-06T03:49:21.475 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T03:49:21.475 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T03:49:21.475 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T03:49:21.475 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T03:49:21.475 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T03:49:21.475 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-06-06T03:49:21.475 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T03:49:21.476 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T03:49:21.476 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-06-06T03:49:21.476 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-06-06T03:49:21.476 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-06-06T03:49:21.476 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-06-06T03:49:21.476 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T03:49:21.476 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T03:49:21.476 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T03:49:21.477 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T03:49:21.477 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T03:49:21.477 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-06-06T03:49:21.477 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-06-06T03:49:21.477 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-06-06T03:49:21.477 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T03:49:21.477 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T03:49:21.477 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-06-06T03:49:21.478 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-06-06T03:49:21.478 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-06-06T03:49:21.478 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-06-06T03:49:21.478 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T03:49:21.478 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-06-06T03:49:21.478 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T03:49:21.478 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-06-06T03:49:21.479 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 929941, 2024-06-06T03:49:21.479 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 18741, 2024-06-06T03:49:21.479 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-06-06T03:49:21.479 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-06-06T03:49:21.479 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-06-06T03:49:21.479 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-06-06T03:49:21.479 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-06-06T03:49:21.479 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-06-06T03:49:21.479 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-06-06T03:49:21.480 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-06-06T03:49:21.480 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7744, 2024-06-06T03:49:21.480 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 161, 2024-06-06T03:49:21.480 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T03:49:21.480 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-06-06T03:49:21.480 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-06-06T03:49:21.480 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-06-06T03:49:21.480 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 55598652, 2024-06-06T03:49:21.481 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 578569, 2024-06-06T03:49:21.481 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-06-06T03:49:21.481 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-06-06T03:49:21.481 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-06-06T03:49:21.481 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-06-06T03:49:21.481 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.481 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-06-06T03:49:21.481 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-06-06T03:49:21.481 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-06-06T03:49:21.482 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 3234, 2024-06-06T03:49:21.482 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 26, 2024-06-06T03:49:21.482 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 4, 2024-06-06T03:49:21.482 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 3234, 2024-06-06T03:49:21.482 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-06-06T03:49:21.482 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3234, 2024-06-06T03:49:21.482 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 5.951879106, 2024-06-06T03:49:21.482 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001840407 2024-06-06T03:49:21.483 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.483 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-06-06T03:49:21.483 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-06-06T03:49:21.483 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3234, 2024-06-06T03:49:21.483 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 3235 2024-06-06T03:49:21.483 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.483 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 3234, 2024-06-06T03:49:21.483 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 3234, 2024-06-06T03:49:21.484 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 0, 2024-06-06T03:49:21.484 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-06-06T03:49:21.484 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-06-06T03:49:21.484 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 1950, 2024-06-06T03:49:21.484 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 0, 2024-06-06T03:49:21.484 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 1281, 2024-06-06T03:49:21.484 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 0, 2024-06-06T03:49:21.484 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 0, 2024-06-06T03:49:21.485 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-06-06T03:49:21.485 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-06-06T03:49:21.485 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-06-06T03:49:21.485 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-06-06T03:49:21.485 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 0, 2024-06-06T03:49:21.485 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-06-06T03:49:21.485 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-06-06T03:49:21.485 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-06-06T03:49:21.486 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 2, 2024-06-06T03:49:21.486 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 0, 2024-06-06T03:49:21.486 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-06-06T03:49:21.486 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-06-06T03:49:21.486 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-06-06T03:49:21.486 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-06-06T03:49:21.486 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-06-06T03:49:21.486 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-06-06T03:49:21.487 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T03:49:21.487 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-06-06T03:49:21.487 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-06-06T03:49:21.487 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-06-06T03:49:21.487 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-06-06T03:49:21.487 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-06-06T03:49:21.487 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-06-06T03:49:21.487 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-06-06T03:49:21.488 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-06-06T03:49:21.488 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-06-06T03:49:21.488 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-06-06T03:49:21.488 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-06-06T03:49:21.488 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-06-06T03:49:21.488 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-06-06T03:49:21.488 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-06-06T03:49:21.488 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-06-06T03:49:21.489 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-06-06T03:49:21.489 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-06-06T03:49:21.489 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-06-06T03:49:21.489 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-06-06T03:49:21.489 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 101, 2024-06-06T03:49:21.489 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-06-06T03:49:21.489 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 13, 2024-06-06T03:49:21.489 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 11, 2024-06-06T03:49:21.490 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 11, 2024-06-06T03:49:21.490 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-06-06T03:49:21.490 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-06-06T03:49:21.490 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 0, 2024-06-06T03:49:21.490 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 2, 2024-06-06T03:49:21.490 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 0 2024-06-06T03:49:21.490 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.490 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-06-06T03:49:21.491 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 0, 2024-06-06T03:49:21.491 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-06-06T03:49:21.491 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-06-06T03:49:21.491 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-06-06T03:49:21.491 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.491 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-06-06T03:49:21.491 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-06-06T03:49:21.492 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.492 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-06-06T03:49:21.492 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-06-06T03:49:21.492 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T03:49:21.492 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-06-06T03:49:21.492 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-06-06T03:49:21.492 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-06-06T03:49:21.493 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-06-06T03:49:21.493 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-06-06T03:49:21.493 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.493 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T03:49:21.493 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:21.493 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-06-06T03:49:21.493 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:21.494 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 3523, 2024-06-06T03:49:21.494 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 357509934, 2024-06-06T03:49:21.494 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:21.494 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 3523, 2024-06-06T03:49:21.494 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-06-06T03:49:21.494 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-06-06T03:49:21.494 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3523, 2024-06-06T03:49:21.495 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 357509934, 2024-06-06T03:49:21.495 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:21.495 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.495 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.495 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.495 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:21.495 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.496 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-06-06T03:49:21.496 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:21.496 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-06-06T03:49:21.496 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:21.496 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-06-06T03:49:21.496 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-06-06T03:49:21.496 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:21.496 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:21.497 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 3234, 2024-06-06T03:49:21.497 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 355195032, 2024-06-06T03:49:21.497 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3234, 2024-06-06T03:49:21.497 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 355195032, 2024-06-06T03:49:21.497 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:21.497 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.497 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.497 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.498 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:21.498 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.498 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-06-06T03:49:21.498 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:21.498 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-06-06T03:49:21.498 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:21.498 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-06-06T03:49:21.498 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-06-06T03:49:21.499 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:21.499 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:21.499 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 3234, 2024-06-06T03:49:21.499 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 3234, 2024-06-06T03:49:21.499 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3234, 2024-06-06T03:49:21.499 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 3234, 2024-06-06T03:49:21.499 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:21.499 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.500 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.500 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.500 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:21.500 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.500 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-06-06T03:49:21.500 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:21.500 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-06-06T03:49:21.500 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:21.500 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-06-06T03:49:21.501 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-06-06T03:49:21.501 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:21.501 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:21.501 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-06-06T03:49:21.501 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-06-06T03:49:21.501 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-06-06T03:49:21.501 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-06-06T03:49:21.502 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:21.502 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.502 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.502 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.502 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:21.502 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:21.502 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x55fca7d886e0": { 2024-06-06T03:49:21.502 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:21.503 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-06-06T03:49:21.503 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:21.503 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-06-06T03:49:21.503 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-06-06T03:49:21.503 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:21.503 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:21.503 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-06-06T03:49:21.503 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-06-06T03:49:21.504 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-06-06T03:49:21.504 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-06-06T03:49:21.504 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:21.504 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:21.504 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:21.504 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:21.504 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:21.504 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:21.505 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-06-06T03:49:22.196 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-06-06T03:49:22.197 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-06-06T03:49:22.197 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-06-06T03:49:22.197 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-06-06T03:49:22.197 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T03:49:22.197 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-06-06T03:49:22.197 DEBUG:tasks.check_counter:Getting stats from f 2024-06-06T03:49:22.197 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-06T03:49:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:49:23.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:22 smithi104 ceph-mon[36569]: pgmap v335: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-06-06T03:49:23.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:22 smithi131 ceph-mon[36687]: pgmap v335: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-06-06T03:49:23.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:22 smithi040 ceph-mon[32220]: pgmap v335: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-06-06T03:49:24.140 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-06-06T03:49:24.141 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T03:49:24.141 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1672, 2024-06-06T03:49:24.141 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1674, 2024-06-06T03:49:24.141 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 38245937, 2024-06-06T03:49:24.141 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 531703, 2024-06-06T03:49:24.141 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 8, 2024-06-06T03:49:24.141 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-06-06T03:49:24.142 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.171250860, 2024-06-06T03:49:24.142 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.049349868, 2024-06-06T03:49:24.142 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.058135644, 2024-06-06T03:49:24.142 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.050478258, 2024-06-06T03:49:24.142 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:24.142 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1672, 2024-06-06T03:49:24.143 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.062023056, 2024-06-06T03:49:24.143 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000037095 2024-06-06T03:49:24.143 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.143 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:24.143 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.143 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.143 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.143 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.144 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T03:49:24.144 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T03:49:24.144 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.144 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T03:49:24.144 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 629, 2024-06-06T03:49:24.144 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 596, 2024-06-06T03:49:24.144 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 31216406, 2024-06-06T03:49:24.144 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 257604, 2024-06-06T03:49:24.145 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 5, 2024-06-06T03:49:24.145 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-06-06T03:49:24.145 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.097395216, 2024-06-06T03:49:24.145 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.023369298, 2024-06-06T03:49:24.145 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.065664627, 2024-06-06T03:49:24.145 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.016087061, 2024-06-06T03:49:24.145 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:24.145 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 595, 2024-06-06T03:49:24.146 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.025875466, 2024-06-06T03:49:24.146 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000043488 2024-06-06T03:49:24.146 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.146 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:24.146 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.146 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.146 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.146 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.147 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1760320, 2024-06-06T03:49:24.147 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 149104 2024-06-06T03:49:24.147 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.147 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T03:49:24.147 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 636, 2024-06-06T03:49:24.147 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 812, 2024-06-06T03:49:24.147 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 38903140, 2024-06-06T03:49:24.147 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 673222, 2024-06-06T03:49:24.147 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 8, 2024-06-06T03:49:24.148 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 6, 2024-06-06T03:49:24.148 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.087868294, 2024-06-06T03:49:24.148 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.023058249, 2024-06-06T03:49:24.148 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.042189494, 2024-06-06T03:49:24.148 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.016300624, 2024-06-06T03:49:24.148 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:24.148 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 810, 2024-06-06T03:49:24.148 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.048165454, 2024-06-06T03:49:24.149 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000059463 2024-06-06T03:49:24.149 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.149 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:24.149 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.149 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.149 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.149 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.149 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-06-06T03:49:24.149 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 473200 2024-06-06T03:49:24.150 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.150 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-06-06T03:49:24.150 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-06-06T03:49:24.150 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-06-06T03:49:24.150 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.150 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-06-06T03:49:24.150 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-06-06T03:49:24.151 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-06-06T03:49:24.151 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4438, 2024-06-06T03:49:24.151 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.119913878, 2024-06-06T03:49:24.151 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000027019 2024-06-06T03:49:24.151 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:24.151 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.152 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-06-06T03:49:24.152 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-06-06T03:49:24.152 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-06-06T03:49:24.152 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.152 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.152 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.153 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:24.153 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.153 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-06-06T03:49:24.153 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-06-06T03:49:24.153 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-06-06T03:49:24.153 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-06-06T03:49:24.154 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.154 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.154 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.154 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.154 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-06-06T03:49:24.155 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-06-06T03:49:24.155 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-06-06T03:49:24.155 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-06-06T03:49:24.155 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 0, 2024-06-06T03:49:24.155 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-06-06T03:49:24.155 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-06-06T03:49:24.156 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 28906, 2024-06-06T03:49:24.156 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 161, 2024-06-06T03:49:24.156 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 28745, 2024-06-06T03:49:24.156 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-06-06T03:49:24.156 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 28677, 2024-06-06T03:49:24.157 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-06-06T03:49:24.157 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-06-06T03:49:24.157 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-06-06T03:49:24.157 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 61, 2024-06-06T03:49:24.157 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-06-06T03:49:24.157 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-06-06T03:49:24.158 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-06-06T03:49:24.158 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-06-06T03:49:24.158 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-06-06T03:49:24.158 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-06-06T03:49:24.158 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-06-06T03:49:24.158 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-06-06T03:49:24.158 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-06-06T03:49:24.159 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-06-06T03:49:24.159 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-06-06T03:49:24.159 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-06-06T03:49:24.159 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-06-06T03:49:24.159 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-06-06T03:49:24.159 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-06-06T03:49:24.159 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-06-06T03:49:24.159 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 0, 2024-06-06T03:49:24.160 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 0, 2024-06-06T03:49:24.160 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-06-06T03:49:24.160 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T03:49:24.160 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-06-06T03:49:24.160 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T03:49:24.160 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-06-06T03:49:24.160 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-06-06T03:49:24.160 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T03:49:24.161 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T03:49:24.161 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-06-06T03:49:24.161 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-06-06T03:49:24.161 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T03:49:24.161 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-06-06T03:49:24.161 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T03:49:24.161 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T03:49:24.161 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T03:49:24.161 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-06-06T03:49:24.162 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-06-06T03:49:24.162 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-06-06T03:49:24.162 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-06-06T03:49:24.162 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.162 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-06-06T03:49:24.162 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-06-06T03:49:24.162 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-06-06T03:49:24.162 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-06-06T03:49:24.163 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 0, 2024-06-06T03:49:24.163 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-06-06T03:49:24.163 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-06-06T03:49:24.163 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-06-06T03:49:24.163 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-06-06T03:49:24.163 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-06-06T03:49:24.163 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-06-06T03:49:24.163 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-06-06T03:49:24.163 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-06-06T03:49:24.164 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-06-06T03:49:24.164 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-06-06T03:49:24.164 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-06-06T03:49:24.164 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-06-06T03:49:24.164 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-06-06T03:49:24.164 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-06-06T03:49:24.164 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-06-06T03:49:24.164 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T03:49:24.165 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-06-06T03:49:24.165 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-06-06T03:49:24.165 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-06-06T03:49:24.165 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-06-06T03:49:24.165 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-06-06T03:49:24.165 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.165 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-06-06T03:49:24.165 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-06-06T03:49:24.166 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 0, 2024-06-06T03:49:24.166 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-06-06T03:49:24.166 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-06-06T03:49:24.166 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 34454, 2024-06-06T03:49:24.166 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-06-06T03:49:24.166 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-06-06T03:49:24.166 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 0, 2024-06-06T03:49:24.166 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-06-06T03:49:24.166 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-06-06T03:49:24.167 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 49, 2024-06-06T03:49:24.167 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 4, 2024-06-06T03:49:24.167 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-06-06T03:49:24.167 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-06-06T03:49:24.167 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-06-06T03:49:24.167 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 110039653, 2024-06-06T03:49:24.167 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 110037330, 2024-06-06T03:49:24.167 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-06-06T03:49:24.168 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.168 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.168 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.168 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.168 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 34454 2024-06-06T03:49:24.168 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.168 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-06-06T03:49:24.168 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 28466, 2024-06-06T03:49:24.169 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 42132, 2024-06-06T03:49:24.169 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 13666, 2024-06-06T03:49:24.169 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 1585, 2024-06-06T03:49:24.169 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 2466, 2024-06-06T03:49:24.169 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 881, 2024-06-06T03:49:24.169 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 28906, 2024-06-06T03:49:24.169 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 42399, 2024-06-06T03:49:24.169 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 13493, 2024-06-06T03:49:24.170 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-06-06T03:49:24.170 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-06-06T03:49:24.170 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-06-06T03:49:24.170 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 162652, 2024-06-06T03:49:24.170 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 215420 2024-06-06T03:49:24.170 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.170 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-06-06T03:49:24.170 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-06-06T03:49:24.171 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-06-06T03:49:24.171 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-06-06T03:49:24.171 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 0, 2024-06-06T03:49:24.171 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-06-06T03:49:24.171 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-06-06T03:49:24.171 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.171 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.171 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.171 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.172 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-06-06T03:49:24.172 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.172 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.172 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.172 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.172 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-06-06T03:49:24.172 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.172 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.172 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.173 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.173 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-06-06T03:49:24.173 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.173 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.173 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.173 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.173 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-06-06T03:49:24.173 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.174 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.174 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.174 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.174 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-06-06T03:49:24.174 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.174 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.174 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.175 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.175 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-06-06T03:49:24.175 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.175 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.175 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.175 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.175 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-06-06T03:49:24.175 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.175 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.176 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.176 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.176 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-06-06T03:49:24.176 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.176 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.176 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.176 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.176 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-06-06T03:49:24.177 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.177 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.177 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.177 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.177 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-06-06T03:49:24.177 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.177 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.177 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.177 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.178 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-06-06T03:49:24.178 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.178 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.178 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.179 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.179 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-06-06T03:49:24.179 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.179 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.179 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.179 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.179 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-06-06T03:49:24.179 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.180 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.180 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.180 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.180 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-06-06T03:49:24.180 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.180 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.180 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.180 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.181 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-06-06T03:49:24.181 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.181 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.181 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.181 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.181 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-06-06T03:49:24.181 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.181 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.181 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.182 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.182 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-06-06T03:49:24.182 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.182 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.182 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.182 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.182 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-06-06T03:49:24.183 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.183 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.183 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.183 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.183 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-06-06T03:49:24.183 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.183 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.183 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.183 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.184 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-06-06T03:49:24.184 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.184 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.184 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.184 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.184 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-06-06T03:49:24.184 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.184 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.184 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.185 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.185 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-06-06T03:49:24.185 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.185 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.185 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.185 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.185 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-06-06T03:49:24.185 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.186 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.186 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.186 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.186 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-06-06T03:49:24.186 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.186 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.186 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.186 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.186 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-06-06T03:49:24.187 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.187 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.187 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.187 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.187 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-06-06T03:49:24.187 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.187 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.187 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.188 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.188 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-06-06T03:49:24.188 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.188 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.188 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.188 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.188 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-06-06T03:49:24.189 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.189 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.189 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.189 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.189 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-06-06T03:49:24.189 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-06-06T03:49:24.189 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-06-06T03:49:24.189 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.189 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.190 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.190 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:24.190 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.190 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-06-06T03:49:24.190 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-06-06T03:49:24.190 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 3, 2024-06-06T03:49:24.190 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 2, 2024-06-06T03:49:24.190 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-06-06T03:49:24.191 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-06-06T03:49:24.191 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-06-06T03:49:24.191 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-06-06T03:49:24.191 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 2056, 2024-06-06T03:49:24.191 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-06-06T03:49:24.191 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.191 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-06-06T03:49:24.191 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-06-06T03:49:24.192 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-06-06T03:49:24.192 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T03:49:24.192 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-06-06T03:49:24.192 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T03:49:24.192 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-06-06T03:49:24.192 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T03:49:24.192 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T03:49:24.192 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T03:49:24.193 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T03:49:24.193 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T03:49:24.193 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-06-06T03:49:24.193 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T03:49:24.193 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T03:49:24.193 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-06-06T03:49:24.193 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-06-06T03:49:24.193 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-06-06T03:49:24.194 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-06-06T03:49:24.194 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T03:49:24.194 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T03:49:24.194 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T03:49:24.194 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T03:49:24.194 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T03:49:24.194 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-06-06T03:49:24.194 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-06-06T03:49:24.195 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-06-06T03:49:24.195 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T03:49:24.195 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T03:49:24.195 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-06-06T03:49:24.195 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-06-06T03:49:24.195 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-06-06T03:49:24.196 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-06-06T03:49:24.196 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T03:49:24.196 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-06-06T03:49:24.196 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T03:49:24.196 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-06-06T03:49:24.196 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 1313444, 2024-06-06T03:49:24.197 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 29300, 2024-06-06T03:49:24.197 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-06-06T03:49:24.197 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-06-06T03:49:24.197 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-06-06T03:49:24.197 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-06-06T03:49:24.198 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-06-06T03:49:24.198 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-06-06T03:49:24.198 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-06-06T03:49:24.198 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-06-06T03:49:24.198 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7744, 2024-06-06T03:49:24.198 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 161, 2024-06-06T03:49:24.199 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T03:49:24.199 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-06-06T03:49:24.199 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-06-06T03:49:24.199 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-06-06T03:49:24.199 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 88113524, 2024-06-06T03:49:24.200 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 961364, 2024-06-06T03:49:24.200 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-06-06T03:49:24.200 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-06-06T03:49:24.200 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-06-06T03:49:24.200 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-06-06T03:49:24.200 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.201 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-06-06T03:49:24.201 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-06-06T03:49:24.201 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-06-06T03:49:24.201 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 2642, 2024-06-06T03:49:24.201 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 21, 2024-06-06T03:49:24.201 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 3, 2024-06-06T03:49:24.202 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 2642, 2024-06-06T03:49:24.202 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-06-06T03:49:24.202 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2642, 2024-06-06T03:49:24.202 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 7.194853884, 2024-06-06T03:49:24.202 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.002723260 2024-06-06T03:49:24.202 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.202 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-06-06T03:49:24.202 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-06-06T03:49:24.203 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2642, 2024-06-06T03:49:24.203 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 2643 2024-06-06T03:49:24.203 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.203 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 2642, 2024-06-06T03:49:24.203 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 2642, 2024-06-06T03:49:24.203 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 0, 2024-06-06T03:49:24.203 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-06-06T03:49:24.203 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-06-06T03:49:24.203 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 1590, 2024-06-06T03:49:24.204 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 0, 2024-06-06T03:49:24.204 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 1050, 2024-06-06T03:49:24.204 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 0, 2024-06-06T03:49:24.204 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 0, 2024-06-06T03:49:24.204 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-06-06T03:49:24.204 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-06-06T03:49:24.204 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-06-06T03:49:24.204 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-06-06T03:49:24.204 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 0, 2024-06-06T03:49:24.205 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-06-06T03:49:24.205 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-06-06T03:49:24.205 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-06-06T03:49:24.205 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 1, 2024-06-06T03:49:24.205 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 0, 2024-06-06T03:49:24.205 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-06-06T03:49:24.205 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-06-06T03:49:24.205 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-06-06T03:49:24.205 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-06-06T03:49:24.206 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-06-06T03:49:24.206 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-06-06T03:49:24.206 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T03:49:24.206 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-06-06T03:49:24.206 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-06-06T03:49:24.206 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-06-06T03:49:24.206 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-06-06T03:49:24.206 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-06-06T03:49:24.207 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-06-06T03:49:24.207 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-06-06T03:49:24.207 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-06-06T03:49:24.207 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-06-06T03:49:24.207 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-06-06T03:49:24.207 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-06-06T03:49:24.207 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-06-06T03:49:24.208 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-06-06T03:49:24.208 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-06-06T03:49:24.208 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-06-06T03:49:24.208 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-06-06T03:49:24.208 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-06-06T03:49:24.208 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-06-06T03:49:24.208 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-06-06T03:49:24.208 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 101, 2024-06-06T03:49:24.208 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-06-06T03:49:24.209 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 13, 2024-06-06T03:49:24.209 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 10, 2024-06-06T03:49:24.209 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 10, 2024-06-06T03:49:24.209 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-06-06T03:49:24.209 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-06-06T03:49:24.209 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 0, 2024-06-06T03:49:24.209 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 2, 2024-06-06T03:49:24.209 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 0 2024-06-06T03:49:24.210 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.210 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-06-06T03:49:24.210 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 0, 2024-06-06T03:49:24.210 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-06-06T03:49:24.210 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-06-06T03:49:24.210 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-06-06T03:49:24.210 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.210 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-06-06T03:49:24.210 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-06-06T03:49:24.211 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.211 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-06-06T03:49:24.211 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-06-06T03:49:24.211 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T03:49:24.211 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-06-06T03:49:24.211 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-06-06T03:49:24.211 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-06-06T03:49:24.212 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-06-06T03:49:24.212 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-06-06T03:49:24.212 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.212 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T03:49:24.212 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:24.212 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-06-06T03:49:24.212 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:24.212 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 2933, 2024-06-06T03:49:24.212 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 108083845, 2024-06-06T03:49:24.213 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:24.213 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 2933, 2024-06-06T03:49:24.213 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-06-06T03:49:24.213 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-06-06T03:49:24.213 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2933, 2024-06-06T03:49:24.213 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 108083845, 2024-06-06T03:49:24.213 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:24.213 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.214 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.214 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.214 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:24.214 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.214 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-06-06T03:49:24.214 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:24.215 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-06-06T03:49:24.215 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:24.215 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-06-06T03:49:24.215 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-06-06T03:49:24.215 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:24.216 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:24.216 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 2642, 2024-06-06T03:49:24.216 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 105860548, 2024-06-06T03:49:24.216 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2642, 2024-06-06T03:49:24.216 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 105860548, 2024-06-06T03:49:24.216 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:24.217 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.217 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.217 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.217 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:24.217 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.217 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-06-06T03:49:24.218 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:24.218 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-06-06T03:49:24.218 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:24.218 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-06-06T03:49:24.218 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-06-06T03:49:24.219 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:24.219 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:24.219 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 2642, 2024-06-06T03:49:24.219 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 2642, 2024-06-06T03:49:24.219 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2642, 2024-06-06T03:49:24.219 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 2642, 2024-06-06T03:49:24.220 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:24.220 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.220 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.220 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.220 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:24.221 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.221 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-06-06T03:49:24.221 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:24.221 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-06-06T03:49:24.221 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:24.221 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-06-06T03:49:24.221 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-06-06T03:49:24.222 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:24.222 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:24.222 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-06-06T03:49:24.222 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-06-06T03:49:24.222 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-06-06T03:49:24.222 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-06-06T03:49:24.222 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:24.222 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.223 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.223 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.223 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:24.223 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:24.223 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x559788f566e0": { 2024-06-06T03:49:24.223 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:24.223 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-06-06T03:49:24.223 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:24.224 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-06-06T03:49:24.224 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-06-06T03:49:24.224 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:24.224 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:24.224 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-06-06T03:49:24.224 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-06-06T03:49:24.224 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-06-06T03:49:24.224 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-06-06T03:49:24.224 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:24.225 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:24.225 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:24.225 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:24.225 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:24.225 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:24.225 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-06-06T03:49:24.795 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-06-06T03:49:24.796 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-06-06T03:49:24.796 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-06-06T03:49:24.796 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-06-06T03:49:24.796 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T03:49:24.796 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-06-06T03:49:24.796 DEBUG:tasks.check_counter:Getting stats from i 2024-06-06T03:49:24.796 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-06T03:49:25.038 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:49:25.159 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:24 smithi131 ceph-mon[36687]: pgmap v336: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-06-06T03:49:25.159 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:24 smithi131 ceph-mon[36687]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:25.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:24 smithi040 ceph-mon[32220]: pgmap v336: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-06-06T03:49:25.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:24 smithi040 ceph-mon[32220]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:25.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:24 smithi104 ceph-mon[36569]: pgmap v336: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-06-06T03:49:25.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:24 smithi104 ceph-mon[36569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T03:49:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-06-06T03:49:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T03:49:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 48422, 2024-06-06T03:49:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 100412, 2024-06-06T03:49:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 18955452, 2024-06-06T03:49:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 77419722, 2024-06-06T03:49:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 18, 2024-06-06T03:49:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 8, 2024-06-06T03:49:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 2.557640544, 2024-06-06T03:49:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 1.250081989, 2024-06-06T03:49:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 4.264455213, 2024-06-06T03:49:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.057134023, 2024-06-06T03:49:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 100409, 2024-06-06T03:49:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 1.605125455, 2024-06-06T03:49:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000015985 2024-06-06T03:49:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 118311, 2024-06-06T03:49:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.022275944, 2024-06-06T03:49:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000188 2024-06-06T03:49:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1779040, 2024-06-06T03:49:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 154720 2024-06-06T03:49:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T03:49:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 4786, 2024-06-06T03:49:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 3570, 2024-06-06T03:49:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 3695666, 2024-06-06T03:49:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 32625751, 2024-06-06T03:49:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 12, 2024-06-06T03:49:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 7, 2024-06-06T03:49:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.341223333, 2024-06-06T03:49:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.173791263, 2024-06-06T03:49:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.111007271, 2024-06-06T03:49:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.062175152, 2024-06-06T03:49:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3569, 2024-06-06T03:49:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.209588134, 2024-06-06T03:49:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000058724 2024-06-06T03:49:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3331, 2024-06-06T03:49:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000377123, 2024-06-06T03:49:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000113 2024-06-06T03:49:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-06-06T03:49:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 475408 2024-06-06T03:49:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T03:49:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 7033, 2024-06-06T03:49:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 9331, 2024-06-06T03:49:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 31929867, 2024-06-06T03:49:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 26523402, 2024-06-06T03:49:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 9, 2024-06-06T03:49:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 5, 2024-06-06T03:49:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.515623115, 2024-06-06T03:49:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.225333645, 2024-06-06T03:49:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.306252693, 2024-06-06T03:49:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.041108735, 2024-06-06T03:49:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 9330, 2024-06-06T03:49:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.227590407, 2024-06-06T03:49:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000024393 2024-06-06T03:49:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 13624, 2024-06-06T03:49:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.002563204, 2024-06-06T03:49:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000188 2024-06-06T03:49:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-06-06T03:49:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T03:49:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-06-06T03:49:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-06-06T03:49:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-06-06T03:49:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-06-06T03:49:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-06-06T03:49:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-06-06T03:49:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3636, 2024-06-06T03:49:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 20.342388609, 2024-06-06T03:49:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.005594716 2024-06-06T03:49:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-06-06T03:49:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-06-06T03:49:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-06-06T03:49:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 310, 2024-06-06T03:49:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.018009542, 2024-06-06T03:49:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000058095 2024-06-06T03:49:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-06-06T03:49:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 8360, 2024-06-06T03:49:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 8138, 2024-06-06T03:49:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-06-06T03:49:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 8138, 2024-06-06T03:49:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 34.032742333, 2024-06-06T03:49:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.004181954 2024-06-06T03:49:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-06-06T03:49:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 222, 2024-06-06T03:49:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 4, 2024-06-06T03:49:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-06-06T03:49:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 175, 2024-06-06T03:49:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 19, 2024-06-06T03:49:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-06-06T03:49:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 20679, 2024-06-06T03:49:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 947, 2024-06-06T03:49:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 19709, 2024-06-06T03:49:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 23, 2024-06-06T03:49:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 19326, 2024-06-06T03:49:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 16498, 2024-06-06T03:49:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 1073, 2024-06-06T03:49:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 1073, 2024-06-06T03:49:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 83, 2024-06-06T03:49:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 9678, 2024-06-06T03:49:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 7755, 2024-06-06T03:49:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 197, 2024-06-06T03:49:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 54, 2024-06-06T03:49:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-06-06T03:49:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-06-06T03:49:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-06-06T03:49:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 3939, 2024-06-06T03:49:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-06-06T03:49:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 58, 2024-06-06T03:49:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 16498, 2024-06-06T03:49:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 82, 2024-06-06T03:49:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 31090, 2024-06-06T03:49:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-06-06T03:49:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-06-06T03:49:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-06-06T03:49:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 84812, 2024-06-06T03:49:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 1539542236, 2024-06-06T03:49:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 6, 2024-06-06T03:49:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T03:49:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-06-06T03:49:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T03:49:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-06-06T03:49:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-06-06T03:49:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T03:49:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T03:49:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-06-06T03:49:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 20, 2024-06-06T03:49:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 23527, 2024-06-06T03:49:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 8349, 2024-06-06T03:49:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 4, 2024-06-06T03:49:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T03:49:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T03:49:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 37947, 2024-06-06T03:49:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 17465, 2024-06-06T03:49:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 431, 2024-06-06T03:49:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 1791 2024-06-06T03:49:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-06-06T03:49:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-06-06T03:49:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-06-06T03:49:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-06-06T03:49:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 88, 2024-06-06T03:49:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 56, 2024-06-06T03:49:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-06-06T03:49:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-06-06T03:49:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-06-06T03:49:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 132, 2024-06-06T03:49:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 132, 2024-06-06T03:49:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-06-06T03:49:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-06-06T03:49:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-06-06T03:49:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-06-06T03:49:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-06-06T03:49:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-06-06T03:49:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-06-06T03:49:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 7, 2024-06-06T03:49:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 58345, 2024-06-06T03:49:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T03:49:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 112, 2024-06-06T03:49:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-06-06T03:49:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 19, 2024-06-06T03:49:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-06-06T03:49:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-06-06T03:49:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-06-06T03:49:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 7, 2024-06-06T03:49:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 26356, 2024-06-06T03:49:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-06-06T03:49:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-06-06T03:49:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 26356, 2024-06-06T03:49:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-06-06T03:49:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-06-06T03:49:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 34, 2024-06-06T03:49:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-06-06T03:49:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-06-06T03:49:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 34, 2024-06-06T03:49:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 4, 2024-06-06T03:49:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-06-06T03:49:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-06-06T03:49:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-06-06T03:49:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 82375357, 2024-06-06T03:49:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-06-06T03:49:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-06-06T03:49:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4115, 2024-06-06T03:49:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 132.358733207, 2024-06-06T03:49:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.032164941 2024-06-06T03:49:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-06-06T03:49:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-06-06T03:49:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 20155, 2024-06-06T03:49:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 36785, 2024-06-06T03:49:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 16630, 2024-06-06T03:49:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 1625, 2024-06-06T03:49:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 2689, 2024-06-06T03:49:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 1064, 2024-06-06T03:49:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 20679, 2024-06-06T03:49:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 37723, 2024-06-06T03:49:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 17044, 2024-06-06T03:49:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 1073, 2024-06-06T03:49:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 41623, 2024-06-06T03:49:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 40550, 2024-06-06T03:49:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 171468, 2024-06-06T03:49:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 223612 2024-06-06T03:49:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-06-06T03:49:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 8687, 2024-06-06T03:49:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 312, 2024-06-06T03:49:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 8360, 2024-06-06T03:49:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 75, 2024-06-06T03:49:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 915, 2024-06-06T03:49:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-06-06T03:49:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4236, 2024-06-06T03:49:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 27.051946659, 2024-06-06T03:49:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.006386200 2024-06-06T03:49:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-06-06T03:49:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 80, 2024-06-06T03:49:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.025230327, 2024-06-06T03:49:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000315379 2024-06-06T03:49:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-06-06T03:49:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-06-06T03:49:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-06-06T03:49:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 954, 2024-06-06T03:49:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 3.884530584, 2024-06-06T03:49:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.004071834 2024-06-06T03:49:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-06-06T03:49:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-06-06T03:49:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-06-06T03:49:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-06-06T03:49:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-06-06T03:49:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-06-06T03:49:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-06-06T03:49:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 351, 2024-06-06T03:49:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.470676435, 2024-06-06T03:49:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001340958 2024-06-06T03:49:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-06-06T03:49:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-06-06T03:49:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-06-06T03:49:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1153, 2024-06-06T03:49:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.845371317, 2024-06-06T03:49:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000733192 2024-06-06T03:49:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-06-06T03:49:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1135, 2024-06-06T03:49:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 1.171919369, 2024-06-06T03:49:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001032528 2024-06-06T03:49:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-06-06T03:49:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 14, 2024-06-06T03:49:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.015244367, 2024-06-06T03:49:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001088883 2024-06-06T03:49:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-06-06T03:49:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-06-06T03:49:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-06-06T03:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-06-06T03:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-06-06T03:49:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-06-06T03:49:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 44, 2024-06-06T03:49:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.310515999, 2024-06-06T03:49:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.007057181 2024-06-06T03:49:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-06-06T03:49:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-06-06T03:49:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-06-06T03:49:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-06-06T03:49:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-06-06T03:49:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 39, 2024-06-06T03:49:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.026404914, 2024-06-06T03:49:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000677049 2024-06-06T03:49:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-06-06T03:49:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 132, 2024-06-06T03:49:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.230902362, 2024-06-06T03:49:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001749260 2024-06-06T03:49:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-06-06T03:49:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-06-06T03:49:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-06-06T03:49:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-06-06T03:49:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-06-06T03:49:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 3, 2024-06-06T03:49:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 2, 2024-06-06T03:49:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-06-06T03:49:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-06-06T03:49:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 1597, 2024-06-06T03:49:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 1597, 2024-06-06T03:49:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 2059, 2024-06-06T03:49:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-06-06T03:49:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-06-06T03:49:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-06-06T03:49:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-06-06T03:49:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T03:49:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-06-06T03:49:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T03:49:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-06-06T03:49:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T03:49:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T03:49:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T03:49:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T03:49:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T03:49:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-06-06T03:49:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T03:49:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T03:49:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-06-06T03:49:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-06-06T03:49:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-06-06T03:49:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-06-06T03:49:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T03:49:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T03:49:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T03:49:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T03:49:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T03:49:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-06-06T03:49:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-06-06T03:49:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-06-06T03:49:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T03:49:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T03:49:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-06-06T03:49:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-06-06T03:49:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-06-06T03:49:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-06-06T03:49:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T03:49:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-06-06T03:49:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T03:49:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-06-06T03:49:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 1030592, 2024-06-06T03:49:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 19926, 2024-06-06T03:49:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-06-06T03:49:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-06-06T03:49:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-06-06T03:49:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-06-06T03:49:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-06-06T03:49:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-06-06T03:49:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-06-06T03:49:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-06-06T03:49:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7744, 2024-06-06T03:49:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 161, 2024-06-06T03:49:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T03:49:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-06-06T03:49:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-06-06T03:49:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-06-06T03:49:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 64603291, 2024-06-06T03:49:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 808256, 2024-06-06T03:49:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-06-06T03:49:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-06-06T03:49:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-06-06T03:49:26.779 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-06-06T03:49:26.779 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.779 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-06-06T03:49:26.779 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-06-06T03:49:26.779 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-06-06T03:49:26.779 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 4780, 2024-06-06T03:49:26.779 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 82357981, 2024-06-06T03:49:26.779 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 28, 2024-06-06T03:49:26.779 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 4780, 2024-06-06T03:49:26.780 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-06-06T03:49:26.780 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4780, 2024-06-06T03:49:26.780 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 28.810415355, 2024-06-06T03:49:26.780 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.006027283 2024-06-06T03:49:26.780 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.780 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-06-06T03:49:26.780 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-06-06T03:49:26.780 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4780, 2024-06-06T03:49:26.780 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 5195 2024-06-06T03:49:26.781 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.781 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 4780, 2024-06-06T03:49:26.781 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 28, 2024-06-06T03:49:26.781 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 4752, 2024-06-06T03:49:26.781 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-06-06T03:49:26.781 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-06-06T03:49:26.781 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-06-06T03:49:26.781 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 10, 2024-06-06T03:49:26.781 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 18, 2024-06-06T03:49:26.781 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 4133, 2024-06-06T03:49:26.782 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 54, 2024-06-06T03:49:26.782 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-06-06T03:49:26.782 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-06-06T03:49:26.782 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-06-06T03:49:26.782 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 12, 2024-06-06T03:49:26.782 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 169, 2024-06-06T03:49:26.782 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-06-06T03:49:26.782 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-06-06T03:49:26.783 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-06-06T03:49:26.783 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 0, 2024-06-06T03:49:26.783 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 20, 2024-06-06T03:49:26.783 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-06-06T03:49:26.783 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-06-06T03:49:26.783 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-06-06T03:49:26.783 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-06-06T03:49:26.783 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-06-06T03:49:26.783 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-06-06T03:49:26.784 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T03:49:26.784 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-06-06T03:49:26.784 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-06-06T03:49:26.784 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 4, 2024-06-06T03:49:26.784 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-06-06T03:49:26.784 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-06-06T03:49:26.784 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-06-06T03:49:26.784 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-06-06T03:49:26.785 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-06-06T03:49:26.785 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-06-06T03:49:26.785 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-06-06T03:49:26.785 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-06-06T03:49:26.785 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-06-06T03:49:26.785 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-06-06T03:49:26.785 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-06-06T03:49:26.785 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-06-06T03:49:26.785 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-06-06T03:49:26.786 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-06-06T03:49:26.786 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-06-06T03:49:26.786 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-06-06T03:49:26.786 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 101, 2024-06-06T03:49:26.786 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-06-06T03:49:26.786 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 13, 2024-06-06T03:49:26.786 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 12, 2024-06-06T03:49:26.786 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 12, 2024-06-06T03:49:26.787 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-06-06T03:49:26.787 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-06-06T03:49:26.787 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 671, 2024-06-06T03:49:26.787 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 14, 2024-06-06T03:49:26.787 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 90 2024-06-06T03:49:26.787 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.787 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-06-06T03:49:26.787 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-06-06T03:49:26.787 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-06-06T03:49:26.788 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-06-06T03:49:26.788 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-06-06T03:49:26.788 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.788 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-06-06T03:49:26.788 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 58148 2024-06-06T03:49:26.788 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.788 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-06-06T03:49:26.788 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-06-06T03:49:26.789 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 6, 2024-06-06T03:49:26.789 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-06-06T03:49:26.789 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 6, 2024-06-06T03:49:26.789 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 132, 2024-06-06T03:49:26.789 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 132, 2024-06-06T03:49:26.789 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-06-06T03:49:26.789 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.789 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T03:49:26.789 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:26.790 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-06-06T03:49:26.790 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:26.790 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 60237, 2024-06-06T03:49:26.790 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 49140365, 2024-06-06T03:49:26.790 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:26.790 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 60237, 2024-06-06T03:49:26.790 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-06-06T03:49:26.790 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-06-06T03:49:26.791 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 60237, 2024-06-06T03:49:26.791 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 49140365, 2024-06-06T03:49:26.791 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:26.791 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.791 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.791 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.791 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:26.791 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.791 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-06-06T03:49:26.792 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:26.792 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-06-06T03:49:26.792 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:26.792 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-06-06T03:49:26.792 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-06-06T03:49:26.792 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:26.792 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:26.792 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 4780, 2024-06-06T03:49:26.792 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 82371067, 2024-06-06T03:49:26.793 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 4780, 2024-06-06T03:49:26.793 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 82371067, 2024-06-06T03:49:26.793 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:26.793 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.793 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.793 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.793 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:26.794 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.794 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-06-06T03:49:26.794 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:26.794 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-06-06T03:49:26.794 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:26.794 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-06-06T03:49:26.794 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-06-06T03:49:26.795 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:26.795 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:26.795 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 4780, 2024-06-06T03:49:26.795 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 4780, 2024-06-06T03:49:26.795 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 4780, 2024-06-06T03:49:26.795 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 4780, 2024-06-06T03:49:26.795 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:26.795 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.795 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.796 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.796 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:26.796 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.796 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-06-06T03:49:26.796 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:26.796 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-06-06T03:49:26.796 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:26.796 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 132, 2024-06-06T03:49:26.797 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 13348, 2024-06-06T03:49:26.797 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:26.797 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 132, 2024-06-06T03:49:26.797 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-06-06T03:49:26.797 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-06-06T03:49:26.797 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 18, 2024-06-06T03:49:26.797 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 13348, 2024-06-06T03:49:26.797 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:26.798 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.798 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.798 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.798 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:26.798 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:26.798 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x5624cc3aa500": { 2024-06-06T03:49:26.798 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 5166, 2024-06-06T03:49:26.798 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-06-06T03:49:26.799 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:26.799 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 26356, 2024-06-06T03:49:26.799 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 78181053, 2024-06-06T03:49:26.799 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:26.799 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 26356, 2024-06-06T03:49:26.799 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-06-06T03:49:26.799 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-06-06T03:49:26.800 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 4115, 2024-06-06T03:49:26.800 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 78175887, 2024-06-06T03:49:26.800 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:26.800 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:26.800 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:26.800 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:26.800 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:26.800 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:26.800 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-06-06T03:49:27.032 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:26 smithi131 ceph-mon[36687]: pgmap v337: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 85 KiB/s wr, 16 op/s 2024-06-06T03:49:27.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:26 smithi040 ceph-mon[32220]: pgmap v337: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 85 KiB/s wr, 16 op/s 2024-06-06T03:49:27.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:26 smithi104 ceph-mon[36569]: pgmap v337: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 85 KiB/s wr, 16 op/s 2024-06-06T03:49:27.407 INFO:tasks.check_counter:Daemon mds.i mds.exported=58 2024-06-06T03:49:27.408 INFO:tasks.check_counter:Daemon mds.i mds.imported=82 2024-06-06T03:49:27.408 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-06-06T03:49:27.408 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-06-06T03:49:27.408 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T03:49:27.408 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=19 2024-06-06T03:49:27.408 DEBUG:tasks.check_counter:Getting stats from l 2024-06-06T03:49:27.409 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-06T03:49:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T03:49:29.076 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:28 smithi131 ceph-mon[36687]: pgmap v338: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 77 KiB/s wr, 13 op/s 2024-06-06T03:49:29.327 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-06-06T03:49:29.327 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T03:49:29.327 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 429, 2024-06-06T03:49:29.328 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 431, 2024-06-06T03:49:29.328 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 12696764, 2024-06-06T03:49:29.328 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 136486, 2024-06-06T03:49:29.328 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 7, 2024-06-06T03:49:29.328 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-06-06T03:49:29.328 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.047385043, 2024-06-06T03:49:29.328 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.011628341, 2024-06-06T03:49:29.329 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.019382806, 2024-06-06T03:49:29.329 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.011979472, 2024-06-06T03:49:29.329 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:29.329 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 429, 2024-06-06T03:49:29.329 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.017813412, 2024-06-06T03:49:29.329 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000041523 2024-06-06T03:49:29.329 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.329 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:29.330 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.330 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.330 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.330 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.330 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T03:49:29.330 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T03:49:29.330 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.330 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T03:49:29.331 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 996, 2024-06-06T03:49:29.331 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1180, 2024-06-06T03:49:29.331 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 21944010, 2024-06-06T03:49:29.331 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 791345, 2024-06-06T03:49:29.331 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 8, 2024-06-06T03:49:29.332 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 6, 2024-06-06T03:49:29.332 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.097164513, 2024-06-06T03:49:29.332 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.030889849, 2024-06-06T03:49:29.332 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.032356723, 2024-06-06T03:49:29.332 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.025955845, 2024-06-06T03:49:29.332 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:29.332 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1177, 2024-06-06T03:49:29.332 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.048962504, 2024-06-06T03:49:29.333 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000041599 2024-06-06T03:49:29.333 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.333 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:29.333 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.333 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.333 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.333 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.333 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-06T03:49:29.334 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 475248 2024-06-06T03:49:29.334 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.334 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T03:49:29.334 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1381, 2024-06-06T03:49:29.334 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1346, 2024-06-06T03:49:29.334 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 18885562, 2024-06-06T03:49:29.334 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 497482, 2024-06-06T03:49:29.334 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 5, 2024-06-06T03:49:29.335 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-06-06T03:49:29.335 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.174726338, 2024-06-06T03:49:29.335 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.050475244, 2024-06-06T03:49:29.335 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.080080704, 2024-06-06T03:49:29.335 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.043652416, 2024-06-06T03:49:29.335 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T03:49:29.335 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1346, 2024-06-06T03:49:29.335 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.066379752, 2024-06-06T03:49:29.336 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000049316 2024-06-06T03:49:29.336 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.336 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-06-06T03:49:29.336 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.336 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.336 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.336 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.336 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1765568, 2024-06-06T03:49:29.337 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 153088 2024-06-06T03:49:29.337 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.337 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-06-06T03:49:29.337 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-06-06T03:49:29.337 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-06-06T03:49:29.337 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.337 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-06-06T03:49:29.337 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-06-06T03:49:29.338 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-06-06T03:49:29.338 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4282, 2024-06-06T03:49:29.338 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.122719854, 2024-06-06T03:49:29.338 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000028659 2024-06-06T03:49:29.338 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:29.338 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.338 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-06-06T03:49:29.338 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-06-06T03:49:29.338 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-06-06T03:49:29.339 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.339 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.339 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.339 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:29.339 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.339 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-06-06T03:49:29.339 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-06-06T03:49:29.339 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-06-06T03:49:29.340 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-06-06T03:49:29.340 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.340 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.340 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.340 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.340 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-06-06T03:49:29.340 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-06-06T03:49:29.340 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-06-06T03:49:29.341 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-06-06T03:49:29.341 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 0, 2024-06-06T03:49:29.341 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-06-06T03:49:29.341 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-06-06T03:49:29.341 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 29040, 2024-06-06T03:49:29.341 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 70, 2024-06-06T03:49:29.341 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 28970, 2024-06-06T03:49:29.341 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-06-06T03:49:29.342 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 28941, 2024-06-06T03:49:29.342 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-06-06T03:49:29.342 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-06-06T03:49:29.342 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-06-06T03:49:29.342 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 79, 2024-06-06T03:49:29.342 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-06-06T03:49:29.342 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-06-06T03:49:29.342 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-06-06T03:49:29.343 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-06-06T03:49:29.343 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-06-06T03:49:29.343 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-06-06T03:49:29.343 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-06-06T03:49:29.343 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-06-06T03:49:29.343 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-06-06T03:49:29.343 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-06-06T03:49:29.343 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-06-06T03:49:29.344 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-06-06T03:49:29.344 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-06-06T03:49:29.344 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-06-06T03:49:29.344 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-06-06T03:49:29.344 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-06-06T03:49:29.344 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 0, 2024-06-06T03:49:29.344 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 0, 2024-06-06T03:49:29.344 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-06-06T03:49:29.345 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T03:49:29.345 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-06-06T03:49:29.345 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T03:49:29.345 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-06-06T03:49:29.345 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-06-06T03:49:29.345 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T03:49:29.345 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T03:49:29.345 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-06-06T03:49:29.346 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-06-06T03:49:29.346 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T03:49:29.346 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-06-06T03:49:29.346 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T03:49:29.346 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T03:49:29.346 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T03:49:29.346 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-06-06T03:49:29.346 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-06-06T03:49:29.346 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-06-06T03:49:29.347 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-06-06T03:49:29.347 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.347 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-06-06T03:49:29.347 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-06-06T03:49:29.347 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-06-06T03:49:29.347 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-06-06T03:49:29.347 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 0, 2024-06-06T03:49:29.347 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-06-06T03:49:29.347 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-06-06T03:49:29.348 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-06-06T03:49:29.348 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-06-06T03:49:29.348 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-06-06T03:49:29.348 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-06-06T03:49:29.348 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-06-06T03:49:29.348 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-06-06T03:49:29.348 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-06-06T03:49:29.348 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-06-06T03:49:29.348 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-06-06T03:49:29.349 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-06-06T03:49:29.349 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-06-06T03:49:29.349 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-06-06T03:49:29.349 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-06-06T03:49:29.349 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T03:49:29.349 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-06-06T03:49:29.349 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-06-06T03:49:29.349 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-06-06T03:49:29.350 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-06-06T03:49:29.350 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-06-06T03:49:29.350 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.350 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-06-06T03:49:29.350 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-06-06T03:49:29.351 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 0, 2024-06-06T03:49:29.351 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-06-06T03:49:29.351 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-06-06T03:49:29.351 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 15104, 2024-06-06T03:49:29.351 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-06-06T03:49:29.351 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-06-06T03:49:29.351 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 0, 2024-06-06T03:49:29.351 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-06-06T03:49:29.351 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-06-06T03:49:29.352 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 24, 2024-06-06T03:49:29.352 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 3, 2024-06-06T03:49:29.352 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-06-06T03:49:29.352 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-06-06T03:49:29.352 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-06-06T03:49:29.352 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 55241323, 2024-06-06T03:49:29.352 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 55240749, 2024-06-06T03:49:29.352 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-06-06T03:49:29.353 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.353 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.353 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.353 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.353 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 15104 2024-06-06T03:49:29.353 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.353 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-06-06T03:49:29.354 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 28647, 2024-06-06T03:49:29.354 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 28754, 2024-06-06T03:49:29.354 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 107, 2024-06-06T03:49:29.354 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 1721, 2024-06-06T03:49:29.354 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 1735, 2024-06-06T03:49:29.354 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 14, 2024-06-06T03:49:29.355 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 29040, 2024-06-06T03:49:29.355 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 29040, 2024-06-06T03:49:29.355 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 0, 2024-06-06T03:49:29.355 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-06-06T03:49:29.355 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-06-06T03:49:29.355 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-06-06T03:49:29.355 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 154052, 2024-06-06T03:49:29.355 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 215420 2024-06-06T03:49:29.355 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.356 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-06-06T03:49:29.356 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-06-06T03:49:29.356 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-06-06T03:49:29.356 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-06-06T03:49:29.356 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 0, 2024-06-06T03:49:29.356 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-06-06T03:49:29.356 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-06-06T03:49:29.356 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.357 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.357 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.357 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.357 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-06-06T03:49:29.357 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.357 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.357 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.357 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.357 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-06-06T03:49:29.358 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.358 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.358 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.358 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.358 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-06-06T03:49:29.358 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.358 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.358 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.359 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.359 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-06-06T03:49:29.359 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.359 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.359 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.359 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.359 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-06-06T03:49:29.359 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.360 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.360 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.360 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.360 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-06-06T03:49:29.360 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.360 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.360 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.360 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.360 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-06-06T03:49:29.361 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.361 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.361 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.361 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.361 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-06-06T03:49:29.361 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.361 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.361 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.362 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.362 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-06-06T03:49:29.362 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.362 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.362 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.362 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.362 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-06-06T03:49:29.362 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.363 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.363 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.363 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.364 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:28 smithi104 ceph-mon[36569]: pgmap v338: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 77 KiB/s wr, 13 op/s 2024-06-06T03:49:29.364 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:28 smithi040 ceph-mon[32220]: pgmap v338: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 77 KiB/s wr, 13 op/s 2024-06-06T03:49:29.365 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-06-06T03:49:29.365 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.365 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.365 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.366 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.366 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-06-06T03:49:29.367 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.367 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.367 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.368 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.368 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-06-06T03:49:29.368 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.368 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.368 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.369 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.369 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-06-06T03:49:29.369 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.369 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.370 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.370 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.370 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-06-06T03:49:29.370 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.371 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.371 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.371 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.371 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-06-06T03:49:29.371 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.372 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.372 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.372 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.372 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-06-06T03:49:29.372 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.372 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.372 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.372 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.373 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-06-06T03:49:29.373 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.373 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.373 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.373 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.373 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-06-06T03:49:29.373 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.373 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.374 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.374 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.374 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-06-06T03:49:29.374 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.374 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.374 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.374 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.374 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-06-06T03:49:29.375 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.375 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.375 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.375 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.375 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-06-06T03:49:29.375 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.375 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.375 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.376 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.376 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-06-06T03:49:29.376 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.376 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.376 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.376 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.376 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-06-06T03:49:29.376 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.376 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.377 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.377 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.377 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-06-06T03:49:29.377 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.377 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.377 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.377 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.377 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-06-06T03:49:29.378 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.378 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.378 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.378 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.378 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-06-06T03:49:29.378 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.378 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.378 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.379 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.379 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-06-06T03:49:29.379 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.379 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.379 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.379 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.379 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-06-06T03:49:29.379 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-06-06T03:49:29.380 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-06-06T03:49:29.380 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.380 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.380 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.380 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:29.380 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.380 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-06-06T03:49:29.380 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-06-06T03:49:29.380 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 3, 2024-06-06T03:49:29.381 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 2, 2024-06-06T03:49:29.381 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-06-06T03:49:29.381 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-06-06T03:49:29.381 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-06-06T03:49:29.381 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-06-06T03:49:29.381 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 2064, 2024-06-06T03:49:29.381 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-06-06T03:49:29.381 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.382 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-06-06T03:49:29.382 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-06-06T03:49:29.382 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-06-06T03:49:29.382 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T03:49:29.382 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-06-06T03:49:29.382 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T03:49:29.382 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-06-06T03:49:29.382 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T03:49:29.382 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T03:49:29.383 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T03:49:29.383 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T03:49:29.383 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T03:49:29.383 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-06-06T03:49:29.383 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T03:49:29.383 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T03:49:29.383 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-06-06T03:49:29.383 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-06-06T03:49:29.384 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-06-06T03:49:29.384 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-06-06T03:49:29.384 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T03:49:29.384 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T03:49:29.384 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T03:49:29.384 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T03:49:29.384 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T03:49:29.384 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-06-06T03:49:29.384 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-06-06T03:49:29.385 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-06-06T03:49:29.385 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T03:49:29.385 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T03:49:29.385 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-06-06T03:49:29.385 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-06-06T03:49:29.385 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-06-06T03:49:29.385 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-06-06T03:49:29.386 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T03:49:29.386 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-06-06T03:49:29.386 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T03:49:29.386 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-06-06T03:49:29.386 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 1330688, 2024-06-06T03:49:29.386 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 29981, 2024-06-06T03:49:29.386 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-06-06T03:49:29.386 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-06-06T03:49:29.387 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-06-06T03:49:29.387 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-06-06T03:49:29.387 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-06-06T03:49:29.387 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-06-06T03:49:29.387 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-06-06T03:49:29.387 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-06-06T03:49:29.387 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7744, 2024-06-06T03:49:29.387 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 161, 2024-06-06T03:49:29.387 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T03:49:29.388 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-06-06T03:49:29.388 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-06-06T03:49:29.388 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-06-06T03:49:29.388 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 89250310, 2024-06-06T03:49:29.388 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 984781, 2024-06-06T03:49:29.388 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-06-06T03:49:29.388 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-06-06T03:49:29.388 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-06-06T03:49:29.389 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-06-06T03:49:29.389 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.389 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-06-06T03:49:29.389 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-06-06T03:49:29.389 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-06-06T03:49:29.389 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 2506, 2024-06-06T03:49:29.389 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 21, 2024-06-06T03:49:29.389 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 3, 2024-06-06T03:49:29.390 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 2506, 2024-06-06T03:49:29.390 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-06-06T03:49:29.390 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2506, 2024-06-06T03:49:29.390 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 4.995898536, 2024-06-06T03:49:29.390 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001993574 2024-06-06T03:49:29.390 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.390 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-06-06T03:49:29.390 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-06-06T03:49:29.391 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2506, 2024-06-06T03:49:29.391 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 2507 2024-06-06T03:49:29.391 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.391 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 2506, 2024-06-06T03:49:29.391 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 2506, 2024-06-06T03:49:29.391 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 0, 2024-06-06T03:49:29.391 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-06-06T03:49:29.391 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-06-06T03:49:29.391 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 1496, 2024-06-06T03:49:29.392 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 0, 2024-06-06T03:49:29.392 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 1008, 2024-06-06T03:49:29.392 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 0, 2024-06-06T03:49:29.392 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 0, 2024-06-06T03:49:29.392 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-06-06T03:49:29.392 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-06-06T03:49:29.392 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-06-06T03:49:29.392 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-06-06T03:49:29.392 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 0, 2024-06-06T03:49:29.393 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-06-06T03:49:29.393 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-06-06T03:49:29.393 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-06-06T03:49:29.393 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 1, 2024-06-06T03:49:29.393 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 0, 2024-06-06T03:49:29.393 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-06-06T03:49:29.393 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-06-06T03:49:29.393 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-06-06T03:49:29.394 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-06-06T03:49:29.394 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-06-06T03:49:29.394 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-06-06T03:49:29.394 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T03:49:29.394 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-06-06T03:49:29.394 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-06-06T03:49:29.394 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-06-06T03:49:29.394 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-06-06T03:49:29.395 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-06-06T03:49:29.395 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-06-06T03:49:29.395 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-06-06T03:49:29.395 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-06-06T03:49:29.395 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-06-06T03:49:29.395 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-06-06T03:49:29.395 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-06-06T03:49:29.395 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-06-06T03:49:29.396 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-06-06T03:49:29.396 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-06-06T03:49:29.396 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-06-06T03:49:29.396 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-06-06T03:49:29.396 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-06-06T03:49:29.396 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-06-06T03:49:29.396 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-06-06T03:49:29.396 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 101, 2024-06-06T03:49:29.396 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-06-06T03:49:29.397 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 13, 2024-06-06T03:49:29.397 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 9, 2024-06-06T03:49:29.397 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 9, 2024-06-06T03:49:29.397 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-06-06T03:49:29.397 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-06-06T03:49:29.397 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 0, 2024-06-06T03:49:29.397 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 2, 2024-06-06T03:49:29.397 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 0 2024-06-06T03:49:29.398 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.398 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-06-06T03:49:29.398 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 0, 2024-06-06T03:49:29.398 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-06-06T03:49:29.398 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-06-06T03:49:29.398 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-06-06T03:49:29.398 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.398 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-06-06T03:49:29.399 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-06-06T03:49:29.399 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.399 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-06-06T03:49:29.399 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-06-06T03:49:29.399 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T03:49:29.399 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-06-06T03:49:29.399 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-06-06T03:49:29.399 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-06-06T03:49:29.399 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-06-06T03:49:29.400 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-06-06T03:49:29.400 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.400 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T03:49:29.400 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:29.400 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-06-06T03:49:29.400 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:29.400 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 2802, 2024-06-06T03:49:29.400 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 53256189, 2024-06-06T03:49:29.400 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:29.401 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 2802, 2024-06-06T03:49:29.401 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-06-06T03:49:29.401 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-06-06T03:49:29.401 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2802, 2024-06-06T03:49:29.401 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 53256189, 2024-06-06T03:49:29.401 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:29.401 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.401 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.402 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.402 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:29.402 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.402 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-06-06T03:49:29.402 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:29.402 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-06-06T03:49:29.402 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:29.402 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-06-06T03:49:29.403 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-06-06T03:49:29.403 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:29.403 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:29.403 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 2506, 2024-06-06T03:49:29.403 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 51053660, 2024-06-06T03:49:29.403 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2506, 2024-06-06T03:49:29.403 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 51053660, 2024-06-06T03:49:29.403 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:29.404 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.404 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.404 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.404 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:29.404 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.404 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-06-06T03:49:29.404 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:29.404 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-06-06T03:49:29.405 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:29.405 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-06-06T03:49:29.405 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-06-06T03:49:29.405 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:29.405 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:29.405 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 2506, 2024-06-06T03:49:29.405 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 2506, 2024-06-06T03:49:29.405 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2506, 2024-06-06T03:49:29.405 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 2506, 2024-06-06T03:49:29.406 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:29.406 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.406 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.406 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.406 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:29.406 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.406 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-06-06T03:49:29.406 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:29.407 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-06-06T03:49:29.407 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:29.407 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-06-06T03:49:29.407 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-06-06T03:49:29.407 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:29.407 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:29.407 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-06-06T03:49:29.407 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-06-06T03:49:29.407 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-06-06T03:49:29.408 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-06-06T03:49:29.408 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:29.408 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.408 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.408 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.408 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:29.408 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-06-06T03:49:29.408 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x55f066e3a6e0": { 2024-06-06T03:49:29.409 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-06-06T03:49:29.409 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-06-06T03:49:29.409 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-06-06T03:49:29.409 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-06-06T03:49:29.409 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-06-06T03:49:29.409 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-06-06T03:49:29.409 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-06-06T03:49:29.409 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-06-06T03:49:29.409 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-06-06T03:49:29.410 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-06-06T03:49:29.410 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-06-06T03:49:29.410 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-06-06T03:49:29.410 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-06-06T03:49:29.410 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-06-06T03:49:29.410 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-06-06T03:49:29.410 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:29.410 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-06-06T03:49:29.411 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-06-06T03:49:30.125 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-06-06T03:49:30.126 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-06-06T03:49:30.126 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-06-06T03:49:30.126 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-06-06T03:49:30.126 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T03:49:30.126 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-06-06T03:49:30.127 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-06-06T03:49:30.139 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-06T03:49:30.139 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:49:30.139 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T03:49:30.162 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-06-06T03:49:30.162 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T03:49:30.223 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T03:49:30.523 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:49:30.915 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:30 smithi040 ceph-mon[32220]: pgmap v339: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 91 KiB/s wr, 16 op/s 2024-06-06T03:49:31.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:30 smithi131 ceph-mon[36687]: pgmap v339: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 91 KiB/s wr, 16 op/s 2024-06-06T03:49:31.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:30 smithi104 ceph-mon[36569]: pgmap v339: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 91 KiB/s wr, 16 op/s 2024-06-06T03:49:32.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.141+0000 7fb7799b5700 1 -- 172.21.15.40:0/2311713775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb774113120 msgr2=0x7fb774115510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:32.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.141+0000 7fb7799b5700 1 --2- 172.21.15.40:0/2311713775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb774113120 0x7fb774115510 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00b0a0 tx=0x7fb76c00b3b0 comp rx=0 tx=0).stop 2024-06-06T03:49:32.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.141+0000 7fb7799b5700 1 -- 172.21.15.40:0/2311713775 shutdown_connections 2024-06-06T03:49:32.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.141+0000 7fb7799b5700 1 --2- 172.21.15.40:0/2311713775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7740764c0 0x7fb7740746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:32.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.141+0000 7fb7799b5700 1 --2- 172.21.15.40:0/2311713775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb774113120 0x7fb774115510 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:32.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.141+0000 7fb7799b5700 1 --2- 172.21.15.40:0/2311713775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb774074c00 0x7fb774075080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:32.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.141+0000 7fb7799b5700 1 -- 172.21.15.40:0/2311713775 >> 172.21.15.40:0/2311713775 conn(0x7fb774079530 msgr2=0x7fb774079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:32.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.141+0000 7fb7799b5700 1 -- 172.21.15.40:0/2311713775 shutdown_connections 2024-06-06T03:49:32.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.141+0000 7fb7799b5700 1 -- 172.21.15.40:0/2311713775 wait complete. 2024-06-06T03:49:32.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.142+0000 7fb7799b5700 1 Processor -- start 2024-06-06T03:49:32.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.142+0000 7fb7799b5700 1 -- start start 2024-06-06T03:49:32.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.142+0000 7fb7799b5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb774074c00 0x7fb77419f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:32.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.142+0000 7fb7799b5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7740764c0 0x7fb77419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:32.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.142+0000 7fb7799b5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774113120 0x7fb7741a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:32.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.142+0000 7fb7799b5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb774118d60 con 0x7fb7740764c0 2024-06-06T03:49:32.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.142+0000 7fb7799b5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb774118be0 con 0x7fb774074c00 2024-06-06T03:49:32.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.142+0000 7fb7799b5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb774118ee0 con 0x7fb774113120 2024-06-06T03:49:32.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.142+0000 7fb7727fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7740764c0 0x7fb77419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:32.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.142+0000 7fb7727fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7740764c0 0x7fb77419f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52882/0 (socket says 172.21.15.40:52882) 2024-06-06T03:49:32.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.142+0000 7fb7727fc700 1 -- 172.21.15.40:0/1105429195 learned_addr learned my addr 172.21.15.40:0/1105429195 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:49:32.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.142+0000 7fb7737fe700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774113120 0x7fb7741a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:32.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.142+0000 7fb772ffd700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb774074c00 0x7fb77419f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:32.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.143+0000 7fb7727fc700 1 -- 172.21.15.40:0/1105429195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774113120 msgr2=0x7fb7741a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:32.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.143+0000 7fb7727fc700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774113120 0x7fb7741a3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:32.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.143+0000 7fb7727fc700 1 -- 172.21.15.40:0/1105429195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb774074c00 msgr2=0x7fb77419f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:32.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.143+0000 7fb7727fc700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb774074c00 0x7fb77419f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:32.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.143+0000 7fb7727fc700 1 -- 172.21.15.40:0/1105429195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7741a4220 con 0x7fb7740764c0 2024-06-06T03:49:32.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.143+0000 7fb772ffd700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb774074c00 0x7fb77419f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:49:32.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.143+0000 7fb7727fc700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7740764c0 0x7fb77419f750 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb76800e9a0 tx=0x7fb76800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:32.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.143+0000 7fb7789b3700 1 -- 172.21.15.40:0/1105429195 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76800cbd0 con 0x7fb7740764c0 2024-06-06T03:49:32.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.143+0000 7fb7799b5700 1 -- 172.21.15.40:0/1105429195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7741a4510 con 0x7fb7740764c0 2024-06-06T03:49:32.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.143+0000 7fb7789b3700 1 -- 172.21.15.40:0/1105429195 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb76800cd70 con 0x7fb7740764c0 2024-06-06T03:49:32.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.143+0000 7fb7789b3700 1 -- 172.21.15.40:0/1105429195 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768018750 con 0x7fb7740764c0 2024-06-06T03:49:32.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.143+0000 7fb7799b5700 1 -- 172.21.15.40:0/1105429195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7741abd90 con 0x7fb7740764c0 2024-06-06T03:49:32.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.144+0000 7fb7799b5700 1 -- 172.21.15.40:0/1105429195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb774069180 con 0x7fb7740764c0 2024-06-06T03:49:32.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.145+0000 7fb7789b3700 1 -- 172.21.15.40:0/1105429195 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb768010430 con 0x7fb7740764c0 2024-06-06T03:49:32.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.146+0000 7fb7789b3700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fb75c0a2d90 0x7fb75c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:32.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.146+0000 7fb772ffd700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fb75c0a2d90 0x7fb75c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:32.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.146+0000 7fb7789b3700 1 -- 172.21.15.40:0/1105429195 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8426+0+0 (secure 0 0 0) 0x7fb7680cfb70 con 0x7fb7740764c0 2024-06-06T03:49:32.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.147+0000 7fb772ffd700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fb75c0a2d90 0x7fb75c0a5250 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb76400bb90 tx=0x7fb764005d80 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:32.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.150+0000 7fb7789b3700 1 -- 172.21.15.40:0/1105429195 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb76809c7f0 con 0x7fb7740764c0 2024-06-06T03:49:32.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.376+0000 7fb7799b5700 1 -- 172.21.15.40:0/1105429195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb7741abfc0 con 0x7fb7740764c0 2024-06-06T03:49:32.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.376+0000 7fb7789b3700 1 -- 172.21.15.40:0/1105429195 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v101) ==== 81+0+940 (secure 0 0 0) 0x7fb76809c0f0 con 0x7fb7740764c0 2024-06-06T03:49:32.377 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/889893607 2024-06-07T03:38:37.622492+0000 2024-06-06T03:49:32.378 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/869712339 2024-06-07T03:38:37.622492+0000 2024-06-06T03:49:32.378 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1294024108 2024-06-07T03:38:37.622492+0000 2024-06-06T03:49:32.378 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/23942578 2024-06-07T03:27:47.578838+0000 2024-06-06T03:49:32.378 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1915158516 2024-06-07T03:38:37.622492+0000 2024-06-06T03:49:32.378 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/869712339 2024-06-07T03:38:37.622492+0000 2024-06-06T03:49:32.378 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2234980855 2024-06-07T03:27:47.578838+0000 2024-06-06T03:49:32.378 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/1920198673 2024-06-07T03:27:19.799499+0000 2024-06-06T03:49:32.378 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/3930494734 2024-06-07T03:27:47.578838+0000 2024-06-06T03:49:32.379 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2301695357 2024-06-07T03:27:47.578838+0000 2024-06-06T03:49:32.379 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1676880727 2024-06-07T03:27:19.799499+0000 2024-06-06T03:49:32.379 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2237379034 2024-06-07T03:38:37.622492+0000 2024-06-06T03:49:32.379 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/3930494734 2024-06-07T03:27:47.578838+0000 2024-06-06T03:49:32.379 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/1920198673 2024-06-07T03:27:19.799499+0000 2024-06-06T03:49:32.379 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1597665640 2024-06-07T03:27:19.799499+0000 2024-06-06T03:49:32.380 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/513375601 2024-06-07T03:27:19.799499+0000 2024-06-06T03:49:32.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.379+0000 7fb7799b5700 1 -- 172.21.15.40:0/1105429195 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fb75c0a2d90 msgr2=0x7fb75c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:32.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.379+0000 7fb7799b5700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fb75c0a2d90 0x7fb75c0a5250 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb76400bb90 tx=0x7fb764005d80 comp rx=0 tx=0).stop 2024-06-06T03:49:32.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.379+0000 7fb7799b5700 1 -- 172.21.15.40:0/1105429195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7740764c0 msgr2=0x7fb77419f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:32.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.379+0000 7fb7799b5700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7740764c0 0x7fb77419f750 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb76800e9a0 tx=0x7fb76800ecb0 comp rx=0 tx=0).stop 2024-06-06T03:49:32.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.379+0000 7fb7799b5700 1 -- 172.21.15.40:0/1105429195 shutdown_connections 2024-06-06T03:49:32.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.379+0000 7fb7799b5700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fb75c0a2d90 0x7fb75c0a5250 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:32.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.379+0000 7fb7799b5700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774113120 0x7fb7741a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:32.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.379+0000 7fb7799b5700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb774074c00 0x7fb77419f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:32.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.379+0000 7fb7799b5700 1 --2- 172.21.15.40:0/1105429195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7740764c0 0x7fb77419f750 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:32.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.379+0000 7fb7799b5700 1 -- 172.21.15.40:0/1105429195 >> 172.21.15.40:0/1105429195 conn(0x7fb774079530 msgr2=0x7fb774111e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:32.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.379+0000 7fb7799b5700 1 -- 172.21.15.40:0/1105429195 shutdown_connections 2024-06-06T03:49:32.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:32.379+0000 7fb7799b5700 1 -- 172.21.15.40:0/1105429195 wait complete. 2024-06-06T03:49:32.382 INFO:teuthology.orchestra.run.smithi040.stderr:listed 16 entries 2024-06-06T03:49:33.021 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:32 smithi040 ceph-mon[32220]: pgmap v340: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 14 op/s 2024-06-06T03:49:33.021 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:32 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1105429195' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T03:49:33.049 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-06-06T03:49:33.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T03:49:33.050 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-06-06T03:49:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:32 smithi131 ceph-mon[36687]: pgmap v340: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 14 op/s 2024-06-06T03:49:33.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:32 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1105429195' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T03:49:33.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:32 smithi104 ceph-mon[36569]: pgmap v340: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 14 op/s 2024-06-06T03:49:33.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:32 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1105429195' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T03:49:33.407 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-06-06T03:49:33.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T03:49:33.407 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-06T03:49:33.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T03:49:33.453 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-06-06T03:49:33.484 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-06T03:49:33.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T03:49:33.484 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T03:49:33.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T03:49:33.559 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-06-06T03:49:33.651 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-06T03:49:33.665 INFO:tasks.cephadm:Teardown begin 2024-06-06T03:49:33.665 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-06T03:49:33.666 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:49:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:49:35.249 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:34 smithi040 ceph-mon[32220]: pgmap v341: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 86 KiB/s wr, 12 op/s 2024-06-06T03:49:35.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:34 smithi131 ceph-mon[36687]: pgmap v341: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 86 KiB/s wr, 12 op/s 2024-06-06T03:49:35.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:34 smithi104 ceph-mon[36569]: pgmap v341: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 86 KiB/s wr, 12 op/s 2024-06-06T03:49:35.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.984+0000 7ff965992700 1 -- 172.21.15.40:0/3353324801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff960101390 msgr2=0x7ff960101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:35.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.984+0000 7ff965992700 1 --2- 172.21.15.40:0/3353324801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff960101390 0x7ff960101810 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff954009990 tx=0x7ff954009ca0 comp rx=0 tx=0).stop 2024-06-06T03:49:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.984+0000 7ff965992700 1 -- 172.21.15.40:0/3353324801 shutdown_connections 2024-06-06T03:49:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.984+0000 7ff965992700 1 --2- 172.21.15.40:0/3353324801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff960101d50 0x7ff96010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.984+0000 7ff965992700 1 --2- 172.21.15.40:0/3353324801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff960107400 0x7ff9601077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.984+0000 7ff965992700 1 --2- 172.21.15.40:0/3353324801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff960101390 0x7ff960101810 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.984+0000 7ff965992700 1 -- 172.21.15.40:0/3353324801 >> 172.21.15.40:0/3353324801 conn(0x7ff960078340 msgr2=0x7ff960078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.984+0000 7ff965992700 1 -- 172.21.15.40:0/3353324801 shutdown_connections 2024-06-06T03:49:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.984+0000 7ff965992700 1 -- 172.21.15.40:0/3353324801 wait complete. 2024-06-06T03:49:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.985+0000 7ff965992700 1 Processor -- start 2024-06-06T03:49:35.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.985+0000 7ff965992700 1 -- start start 2024-06-06T03:49:35.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.985+0000 7ff965992700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff960101390 0x7ff96019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:35.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.985+0000 7ff965992700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff960101d50 0x7ff96019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:35.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.985+0000 7ff965992700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff960107400 0x7ff960198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:35.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.985+0000 7ff965992700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff960111740 con 0x7ff960101390 2024-06-06T03:49:35.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.985+0000 7ff965992700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601115c0 con 0x7ff960107400 2024-06-06T03:49:35.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.985+0000 7ff965992700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601118c0 con 0x7ff960101d50 2024-06-06T03:49:35.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.985+0000 7ff95effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff960101390 0x7ff96019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.985+0000 7ff95effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff960101390 0x7ff96019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52894/0 (socket says 172.21.15.40:52894) 2024-06-06T03:49:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.985+0000 7ff95effd700 1 -- 172.21.15.40:0/275194584 learned_addr learned my addr 172.21.15.40:0/275194584 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:49:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.985+0000 7ff95e7fc700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff960101d50 0x7ff96019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.986+0000 7ff95f7fe700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff960107400 0x7ff960198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.986+0000 7ff95effd700 1 -- 172.21.15.40:0/275194584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff960101d50 msgr2=0x7ff96019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.986+0000 7ff95effd700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff960101d50 0x7ff96019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.986+0000 7ff95effd700 1 -- 172.21.15.40:0/275194584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff960107400 msgr2=0x7ff960198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.986+0000 7ff95effd700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff960107400 0x7ff960198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.986+0000 7ff95effd700 1 -- 172.21.15.40:0/275194584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9601991f0 con 0x7ff960101390 2024-06-06T03:49:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.986+0000 7ff95e7fc700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff960101d50 0x7ff96019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T03:49:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.986+0000 7ff95effd700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff960101390 0x7ff96019e8d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff95000b530 tx=0x7ff95000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:35.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.986+0000 7ff964990700 1 -- 172.21.15.40:0/275194584 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950015020 con 0x7ff960101390 2024-06-06T03:49:35.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.986+0000 7ff965992700 1 -- 172.21.15.40:0/275194584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9601994e0 con 0x7ff960101390 2024-06-06T03:49:35.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.986+0000 7ff964990700 1 -- 172.21.15.40:0/275194584 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff950011d10 con 0x7ff960101390 2024-06-06T03:49:35.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.986+0000 7ff964990700 1 -- 172.21.15.40:0/275194584 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95000f2e0 con 0x7ff960101390 2024-06-06T03:49:35.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.986+0000 7ff965992700 1 -- 172.21.15.40:0/275194584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9601a57a0 con 0x7ff960101390 2024-06-06T03:49:35.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.987+0000 7ff965992700 1 -- 172.21.15.40:0/275194584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff960069180 con 0x7ff960101390 2024-06-06T03:49:35.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.989+0000 7ff964990700 1 -- 172.21.15.40:0/275194584 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff950018020 con 0x7ff960101390 2024-06-06T03:49:35.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.989+0000 7ff964990700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7ff9480a2d90 0x7ff9480a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:35.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.989+0000 7ff95e7fc700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7ff9480a2d90 0x7ff9480a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:35.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.990+0000 7ff964990700 1 -- 172.21.15.40:0/275194584 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8426+0+0 (secure 0 0 0) 0x7ff9500ce9e0 con 0x7ff960101390 2024-06-06T03:49:35.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.990+0000 7ff95e7fc700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7ff9480a2d90 0x7ff9480a5250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff9600ffae0 tx=0x7ff95400b540 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:35.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:35.994+0000 7ff964990700 1 -- 172.21.15.40:0/275194584 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff95001f020 con 0x7ff960101390 2024-06-06T03:49:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:36.250+0000 7ff965992700 1 -- 172.21.15.40:0/275194584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9601a5a30 con 0x7ff960101390 2024-06-06T03:49:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:36.251+0000 7ff964990700 1 -- 172.21.15.40:0/275194584 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11944 (secure 0 0 0) 0x7ff95009b770 con 0x7ff960101390 2024-06-06T03:49:36.252 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:49:36.252 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:49:23.429147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24443,24425,24398,24403]},"id":1}]} 2024-06-06T03:49:36.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:36.254+0000 7ff965992700 1 -- 172.21.15.40:0/275194584 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7ff9480a2d90 msgr2=0x7ff9480a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:36.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:36.254+0000 7ff965992700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7ff9480a2d90 0x7ff9480a5250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff9600ffae0 tx=0x7ff95400b540 comp rx=0 tx=0).stop 2024-06-06T03:49:36.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:36.254+0000 7ff965992700 1 -- 172.21.15.40:0/275194584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff960101390 msgr2=0x7ff96019e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:36.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:36.254+0000 7ff965992700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff960101390 0x7ff96019e8d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff95000b530 tx=0x7ff95000b840 comp rx=0 tx=0).stop 2024-06-06T03:49:36.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:36.254+0000 7ff965992700 1 -- 172.21.15.40:0/275194584 shutdown_connections 2024-06-06T03:49:36.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:36.254+0000 7ff965992700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7ff9480a2d90 0x7ff9480a5250 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:36.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:36.254+0000 7ff965992700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff960107400 0x7ff960198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:36.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:36.254+0000 7ff965992700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff960101390 0x7ff96019e8d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:36.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:36.254+0000 7ff965992700 1 --2- 172.21.15.40:0/275194584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff960101d50 0x7ff96019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:36.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:36.254+0000 7ff965992700 1 -- 172.21.15.40:0/275194584 >> 172.21.15.40:0/275194584 conn(0x7ff960078340 msgr2=0x7ff9601005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:36.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:36.254+0000 7ff965992700 1 -- 172.21.15.40:0/275194584 shutdown_connections 2024-06-06T03:49:36.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:36.254+0000 7ff965992700 1 -- 172.21.15.40:0/275194584 wait complete. 2024-06-06T03:49:36.256 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 49 2024-06-06T03:49:36.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T03:49:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:49:37.267 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:36 smithi040 ceph-mon[32220]: pgmap v342: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 206 KiB/s wr, 17 op/s 2024-06-06T03:49:37.267 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:36 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/275194584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:49:37.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:36 smithi131 ceph-mon[36687]: pgmap v342: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 206 KiB/s wr, 17 op/s 2024-06-06T03:49:37.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:36 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/275194584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:49:37.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:36 smithi104 ceph-mon[36569]: pgmap v342: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 206 KiB/s wr, 17 op/s 2024-06-06T03:49:37.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:36 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/275194584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:49:38.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.963+0000 7fc010af9700 1 -- 172.21.15.40:0/216509092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc00c107840 msgr2=0x7fc00c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:38.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.963+0000 7fc010af9700 1 --2- 172.21.15.40:0/216509092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc00c107840 0x7fc00c10f3c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc00400b0d0 tx=0x7fc00400b3e0 comp rx=0 tx=0).stop 2024-06-06T03:49:38.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.963+0000 7fc010af9700 1 -- 172.21.15.40:0/216509092 shutdown_connections 2024-06-06T03:49:38.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.963+0000 7fc010af9700 1 --2- 172.21.15.40:0/216509092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc00c107840 0x7fc00c10f3c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:38.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.963+0000 7fc010af9700 1 --2- 172.21.15.40:0/216509092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc00c102740 0x7fc00c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:38.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.963+0000 7fc010af9700 1 --2- 172.21.15.40:0/216509092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc00c1030f0 0x7fc00c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:38.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.963+0000 7fc010af9700 1 -- 172.21.15.40:0/216509092 >> 172.21.15.40:0/216509092 conn(0x7fc00c0795f0 msgr2=0x7fc00c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:38.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.963+0000 7fc010af9700 1 -- 172.21.15.40:0/216509092 shutdown_connections 2024-06-06T03:49:38.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.963+0000 7fc010af9700 1 -- 172.21.15.40:0/216509092 wait complete. 2024-06-06T03:49:38.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc010af9700 1 Processor -- start 2024-06-06T03:49:38.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc010af9700 1 -- start start 2024-06-06T03:49:38.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc010af9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc00c102740 0x7fc00c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:38.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc010af9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc00c1030f0 0x7fc00c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:38.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc010af9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc00c107840 0x7fc00c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:38.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc010af9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1127d0 con 0x7fc00c102740 2024-06-06T03:49:38.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc010af9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c112650 con 0x7fc00c107840 2024-06-06T03:49:38.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc010af9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c112950 con 0x7fc00c1030f0 2024-06-06T03:49:38.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc00a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc00c102740 0x7fc00c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:38.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc00a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc00c102740 0x7fc00c19e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49204/0 (socket says 172.21.15.40:49204) 2024-06-06T03:49:38.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc00a59c700 1 -- 172.21.15.40:0/3618901134 learned_addr learned my addr 172.21.15.40:0/3618901134 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:49:38.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc00ad9d700 1 --2- 172.21.15.40:0/3618901134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc00c107840 0x7fc00c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:38.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc009d9b700 1 --2- 172.21.15.40:0/3618901134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc00c1030f0 0x7fc00c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:38.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc00a59c700 1 -- 172.21.15.40:0/3618901134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc00c1030f0 msgr2=0x7fc00c19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:38.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.964+0000 7fc00a59c700 1 --2- 172.21.15.40:0/3618901134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc00c1030f0 0x7fc00c19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:38.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.965+0000 7fc00a59c700 1 -- 172.21.15.40:0/3618901134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc00c107840 msgr2=0x7fc00c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:38.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.965+0000 7fc00a59c700 1 --2- 172.21.15.40:0/3618901134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc00c107840 0x7fc00c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:38.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.965+0000 7fc00a59c700 1 -- 172.21.15.40:0/3618901134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c1990f0 con 0x7fc00c102740 2024-06-06T03:49:38.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.965+0000 7fc00a59c700 1 --2- 172.21.15.40:0/3618901134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc00c102740 0x7fc00c19e7d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00d980 tx=0x7fbffc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:38.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.965+0000 7fbffb7fe700 1 -- 172.21.15.40:0/3618901134 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc016070 con 0x7fc00c102740 2024-06-06T03:49:38.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.965+0000 7fbffb7fe700 1 -- 172.21.15.40:0/3618901134 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbffc004d10 con 0x7fc00c102740 2024-06-06T03:49:38.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.965+0000 7fc010af9700 1 -- 172.21.15.40:0/3618901134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c1993e0 con 0x7fc00c102740 2024-06-06T03:49:38.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.965+0000 7fc010af9700 1 -- 172.21.15.40:0/3618901134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c1a56a0 con 0x7fc00c102740 2024-06-06T03:49:38.970 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:38 smithi040 ceph-mon[32220]: pgmap v343: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 150 KiB/s wr, 15 op/s 2024-06-06T03:49:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.965+0000 7fbffb7fe700 1 -- 172.21.15.40:0/3618901134 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc005670 con 0x7fc00c102740 2024-06-06T03:49:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.966+0000 7fc010af9700 1 -- 172.21.15.40:0/3618901134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc00c069180 con 0x7fc00c102740 2024-06-06T03:49:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.968+0000 7fbffb7fe700 1 -- 172.21.15.40:0/3618901134 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbffc004750 con 0x7fc00c102740 2024-06-06T03:49:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.968+0000 7fbffb7fe700 1 --2- 172.21.15.40:0/3618901134 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fbff40a2d90 0x7fbff40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:38.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.969+0000 7fc009d9b700 1 --2- 172.21.15.40:0/3618901134 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fbff40a2d90 0x7fbff40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.969+0000 7fbffb7fe700 1 -- 172.21.15.40:0/3618901134 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8426+0+0 (secure 0 0 0) 0x7fbffc0cf760 con 0x7fc00c102740 2024-06-06T03:49:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.969+0000 7fc009d9b700 1 --2- 172.21.15.40:0/3618901134 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fbff40a2d90 0x7fbff40a5250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc00c108c60 tx=0x7fc000009190 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:38.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:38.972+0000 7fbffb7fe700 1 -- 172.21.15.40:0/3618901134 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbffc01f020 con 0x7fc00c102740 2024-06-06T03:49:39.143 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:38 smithi104 ceph-mon[36569]: pgmap v343: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 150 KiB/s wr, 15 op/s 2024-06-06T03:49:39.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:39.224+0000 7fc010af9700 1 -- 172.21.15.40:0/3618901134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc00c02dca0 con 0x7fc00c102740 2024-06-06T03:49:39.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:39.226+0000 7fbffb7fe700 1 -- 172.21.15.40:0/3618901134 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11944 (secure 0 0 0) 0x7fbffc09c3e0 con 0x7fc00c102740 2024-06-06T03:49:39.227 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:49:39.227 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1980325130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1980325130},{"type":"v1","addr":"172.21.15.40:6835","nonce":1980325130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1373355997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":1373355997},{"type":"v1","addr":"172.21.15.40:6837","nonce":1373355997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T03:35:15.961853+0000","modified":"2024-06-06T03:49:23.429147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24443,"mds_2":24425,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1930069210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1930069210},{"type":"v1","addr":"172.21.15.40:6839","nonce":1930069210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14586":{"gid":14586,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/2460301066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":2460301066},{"type":"v1","addr":"172.21.15.40:6841","nonce":2460301066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24358":{"gid":24358,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/1833802668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1833802668},{"type":"v1","addr":"172.21.15.131:6833","nonce":1833802668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24376":{"gid":24376,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1221528906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1221528906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1221528906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/526159320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":526159320},{"type":"v1","addr":"172.21.15.131:6837","nonce":526159320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.104:6833/1414464971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1414464971},{"type":"v1","addr":"172.21.15.104:6833","nonce":1414464971}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.131:6839/782268782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782268782},{"type":"v1","addr":"172.21.15.131:6839","nonce":782268782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.104:6835/75462915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":75462915},{"type":"v1","addr":"172.21.15.104:6835","nonce":75462915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.104:6837/2498466212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2498466212},{"type":"v1","addr":"172.21.15.104:6837","nonce":2498466212}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.104:6839/489480648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":489480648},{"type":"v1","addr":"172.21.15.104:6839","nonce":489480648}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24443,24425,24398,24403]},"id":1}]} 2024-06-06T03:49:39.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:39.228+0000 7fc010af9700 1 -- 172.21.15.40:0/3618901134 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fbff40a2d90 msgr2=0x7fbff40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:39.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:39.228+0000 7fc010af9700 1 --2- 172.21.15.40:0/3618901134 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fbff40a2d90 0x7fbff40a5250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc00c108c60 tx=0x7fc000009190 comp rx=0 tx=0).stop 2024-06-06T03:49:39.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:39.228+0000 7fc010af9700 1 -- 172.21.15.40:0/3618901134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc00c102740 msgr2=0x7fc00c19e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:39.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:39.228+0000 7fc010af9700 1 --2- 172.21.15.40:0/3618901134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc00c102740 0x7fc00c19e7d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00d980 tx=0x7fbffc00dc90 comp rx=0 tx=0).stop 2024-06-06T03:49:39.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:39.228+0000 7fc010af9700 1 -- 172.21.15.40:0/3618901134 shutdown_connections 2024-06-06T03:49:39.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:39.228+0000 7fc010af9700 1 --2- 172.21.15.40:0/3618901134 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7fbff40a2d90 0x7fbff40a5250 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:39.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:39.228+0000 7fc010af9700 1 --2- 172.21.15.40:0/3618901134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc00c107840 0x7fc00c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:39.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:39.228+0000 7fc010af9700 1 --2- 172.21.15.40:0/3618901134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc00c102740 0x7fc00c19e7d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:39.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:39.228+0000 7fc010af9700 1 --2- 172.21.15.40:0/3618901134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc00c1030f0 0x7fc00c19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:39.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:39.228+0000 7fc010af9700 1 -- 172.21.15.40:0/3618901134 >> 172.21.15.40:0/3618901134 conn(0x7fc00c0795f0 msgr2=0x7fc00c1004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:39.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:39.228+0000 7fc010af9700 1 -- 172.21.15.40:0/3618901134 shutdown_connections 2024-06-06T03:49:39.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:39.228+0000 7fc010af9700 1 -- 172.21.15.40:0/3618901134 wait complete. 2024-06-06T03:49:39.230 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 49 2024-06-06T03:49:39.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:38 smithi131 ceph-mon[36687]: pgmap v343: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 150 KiB/s wr, 15 op/s 2024-06-06T03:49:39.920 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T03:49:40.175 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:49:40.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:39 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3618901134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:49:40.298 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:39 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3618901134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:49:40.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:39 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3618901134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T03:49:41.220 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:40 smithi040 ceph-mon[32220]: pgmap v344: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 168 KiB/s wr, 18 op/s 2024-06-06T03:49:41.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:40 smithi131 ceph-mon[36687]: pgmap v344: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 168 KiB/s wr, 18 op/s 2024-06-06T03:49:41.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:40 smithi104 ceph-mon[36569]: pgmap v344: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 168 KiB/s wr, 18 op/s 2024-06-06T03:49:42.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.110+0000 7f606aafd700 1 -- 172.21.15.40:0/2137820221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6064106720 msgr2=0x7f606410e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:42.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.110+0000 7f606aafd700 1 --2- 172.21.15.40:0/2137820221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6064106720 0x7f606410e2c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f605c00b0d0 tx=0x7f605c00b3e0 comp rx=0 tx=0).stop 2024-06-06T03:49:42.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.110+0000 7f606aafd700 1 -- 172.21.15.40:0/2137820221 shutdown_connections 2024-06-06T03:49:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.110+0000 7f606aafd700 1 --2- 172.21.15.40:0/2137820221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60641019d0 0x7f6064101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.110+0000 7f606aafd700 1 --2- 172.21.15.40:0/2137820221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6064106720 0x7f606410e2c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.110+0000 7f606aafd700 1 --2- 172.21.15.40:0/2137820221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6064102380 0x7f6064105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.110+0000 7f606aafd700 1 -- 172.21.15.40:0/2137820221 >> 172.21.15.40:0/2137820221 conn(0x7f6064078340 msgr2=0x7f6064078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.110+0000 7f606aafd700 1 -- 172.21.15.40:0/2137820221 shutdown_connections 2024-06-06T03:49:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.110+0000 7f606aafd700 1 -- 172.21.15.40:0/2137820221 wait complete. 2024-06-06T03:49:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.110+0000 7f606aafd700 1 Processor -- start 2024-06-06T03:49:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f606aafd700 1 -- start start 2024-06-06T03:49:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f606aafd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60641019d0 0x7f6064197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f606aafd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6064102380 0x7f6064198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f606aafd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6064106720 0x7f606419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f606aafd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6064111760 con 0x7f60641019d0 2024-06-06T03:49:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f606aafd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f60641115e0 con 0x7f6064106720 2024-06-06T03:49:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f606aafd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f60641118e0 con 0x7f6064102380 2024-06-06T03:49:42.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f6068899700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60641019d0 0x7f6064197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:42.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f6068899700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60641019d0 0x7f6064197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49220/0 (socket says 172.21.15.40:49220) 2024-06-06T03:49:42.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f6068899700 1 -- 172.21.15.40:0/2803083847 learned_addr learned my addr 172.21.15.40:0/2803083847 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:49:42.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f6063fff700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6064102380 0x7f6064198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:42.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f6068899700 1 -- 172.21.15.40:0/2803083847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6064102380 msgr2=0x7f6064198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:42.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f606909a700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6064106720 0x7f606419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:42.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f6068899700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6064102380 0x7f6064198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:42.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f6068899700 1 -- 172.21.15.40:0/2803083847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6064106720 msgr2=0x7f606419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:42.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f6068899700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6064106720 0x7f606419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:42.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f6068899700 1 -- 172.21.15.40:0/2803083847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606419cbd0 con 0x7f60641019d0 2024-06-06T03:49:42.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.111+0000 7f6063fff700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6064102380 0x7f6064198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:49:42.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.112+0000 7f6068899700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60641019d0 0x7f6064197bd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f605800b530 tx=0x7f605800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:42.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.112+0000 7f6061ffb700 1 -- 172.21.15.40:0/2803083847 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058015020 con 0x7f60641019d0 2024-06-06T03:49:42.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.112+0000 7f606aafd700 1 -- 172.21.15.40:0/2803083847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606419cec0 con 0x7f60641019d0 2024-06-06T03:49:42.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.112+0000 7f6061ffb700 1 -- 172.21.15.40:0/2803083847 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6058011d10 con 0x7f60641019d0 2024-06-06T03:49:42.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.112+0000 7f6061ffb700 1 -- 172.21.15.40:0/2803083847 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058010e50 con 0x7f60641019d0 2024-06-06T03:49:42.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.112+0000 7f606aafd700 1 -- 172.21.15.40:0/2803083847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60641a4780 con 0x7f60641019d0 2024-06-06T03:49:42.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.113+0000 7f606aafd700 1 -- 172.21.15.40:0/2803083847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6064102ad0 con 0x7f60641019d0 2024-06-06T03:49:42.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.114+0000 7f6061ffb700 1 -- 172.21.15.40:0/2803083847 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6058018020 con 0x7f60641019d0 2024-06-06T03:49:42.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.115+0000 7f6061ffb700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f60540a2f30 0x7f60540a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:42.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.115+0000 7f6063fff700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f60540a2f30 0x7f60540a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:42.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.115+0000 7f6061ffb700 1 -- 172.21.15.40:0/2803083847 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8426+0+0 (secure 0 0 0) 0x7f60580cebb0 con 0x7f60641019d0 2024-06-06T03:49:42.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.115+0000 7f6063fff700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f60540a2f30 0x7f60540a53f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6050005fd0 tx=0x7f6050005dc0 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:42.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.119+0000 7f6061ffb700 1 -- 172.21.15.40:0/2803083847 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f605801f020 con 0x7f60641019d0 2024-06-06T03:49:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.325+0000 7f606aafd700 1 -- 172.21.15.40:0/2803083847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f606402dca0 con 0x7f60641019d0 2024-06-06T03:49:42.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.327+0000 7f6061ffb700 1 -- 172.21.15.40:0/2803083847 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+23002 (secure 0 0 0) 0x7f60580b9e20 con 0x7f60641019d0 2024-06-06T03:49:42.328 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-06-06T03:49:42.328 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":101,"fsid":"77c25d68-23b4-11ef-bc9b-c7b262605968","created":"2024-06-06T03:26:53.165489+0000","modified":"2024-06-06T03:49:13.819955+0000","last_up_change":"2024-06-06T03:34:07.114448+0000","last_in_change":"2024-06-06T03:33:46.152948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T03:30:14.102785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T03:35:08.521025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T03:35:12.369035+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63c0a320-a2f2-4b84-87e7-68501d335285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6803","nonce":3453526793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6805","nonce":3453526793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6809","nonce":3453526793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3453526793},{"type":"v1","addr":"172.21.15.40:6807","nonce":3453526793}]},"public_addr":"172.21.15.40:6803/3453526793","cluster_addr":"172.21.15.40:6805/3453526793","heartbeat_back_addr":"172.21.15.40:6809/3453526793","heartbeat_front_addr":"172.21.15.40:6807/3453526793","state":["exists","up"]},{"osd":1,"uuid":"70a31534-06a1-4ffa-a54d-577d45bccf98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6801","nonce":405854252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6803","nonce":405854252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6807","nonce":405854252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":405854252},{"type":"v1","addr":"172.21.15.104:6805","nonce":405854252}]},"public_addr":"172.21.15.104:6801/405854252","cluster_addr":"172.21.15.104:6803/405854252","heartbeat_back_addr":"172.21.15.104:6807/405854252","heartbeat_front_addr":"172.21.15.104:6805/405854252","state":["exists","up"]},{"osd":2,"uuid":"f5a89f43-e533-45f0-a777-bd3e9bfa8072","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6801","nonce":3919221141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6803","nonce":3919221141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6807","nonce":3919221141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3919221141},{"type":"v1","addr":"172.21.15.131:6805","nonce":3919221141}]},"public_addr":"172.21.15.131:6801/3919221141","cluster_addr":"172.21.15.131:6803/3919221141","heartbeat_back_addr":"172.21.15.131:6807/3919221141","heartbeat_front_addr":"172.21.15.131:6805/3919221141","state":["exists","up"]},{"osd":3,"uuid":"4df19390-6378-4346-a2be-29bce1dd2380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6811","nonce":2008541522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6813","nonce":2008541522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6817","nonce":2008541522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2008541522},{"type":"v1","addr":"172.21.15.40:6815","nonce":2008541522}]},"public_addr":"172.21.15.40:6811/2008541522","cluster_addr":"172.21.15.40:6813/2008541522","heartbeat_back_addr":"172.21.15.40:6817/2008541522","heartbeat_front_addr":"172.21.15.40:6815/2008541522","state":["exists","up"]},{"osd":4,"uuid":"508de408-1cb1-4423-8199-db6bde9034f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6809","nonce":1153394847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6811","nonce":1153394847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6815","nonce":1153394847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1153394847},{"type":"v1","addr":"172.21.15.104:6813","nonce":1153394847}]},"public_addr":"172.21.15.104:6809/1153394847","cluster_addr":"172.21.15.104:6811/1153394847","heartbeat_back_addr":"172.21.15.104:6815/1153394847","heartbeat_front_addr":"172.21.15.104:6813/1153394847","state":["exists","up"]},{"osd":5,"uuid":"a8c0cec9-a8ad-4a5d-800b-8b891ca6d836","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6809","nonce":1051933303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6811","nonce":1051933303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6815","nonce":1051933303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1051933303},{"type":"v1","addr":"172.21.15.131:6813","nonce":1051933303}]},"public_addr":"172.21.15.131:6809/1051933303","cluster_addr":"172.21.15.131:6811/1051933303","heartbeat_back_addr":"172.21.15.131:6815/1051933303","heartbeat_front_addr":"172.21.15.131:6813/1051933303","state":["exists","up"]},{"osd":6,"uuid":"5f90b7ca-3da2-4a32-bc8b-4f543f4a5b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6819","nonce":2670922011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6821","nonce":2670922011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6825","nonce":2670922011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2670922011},{"type":"v1","addr":"172.21.15.40:6823","nonce":2670922011}]},"public_addr":"172.21.15.40:6819/2670922011","cluster_addr":"172.21.15.40:6821/2670922011","heartbeat_back_addr":"172.21.15.40:6825/2670922011","heartbeat_front_addr":"172.21.15.40:6823/2670922011","state":["exists","up"]},{"osd":7,"uuid":"bfc536bb-6ae4-459e-afd9-8197c7420bd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6817","nonce":2353571051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6819","nonce":2353571051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6823","nonce":2353571051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2353571051},{"type":"v1","addr":"172.21.15.104:6821","nonce":2353571051}]},"public_addr":"172.21.15.104:6817/2353571051","cluster_addr":"172.21.15.104:6819/2353571051","heartbeat_back_addr":"172.21.15.104:6823/2353571051","heartbeat_front_addr":"172.21.15.104:6821/2353571051","state":["exists","up"]},{"osd":8,"uuid":"1fe214f2-5ea3-4fc5-83bf-d9bccafe1529","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6817","nonce":1364533794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6819","nonce":1364533794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6823","nonce":1364533794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1364533794},{"type":"v1","addr":"172.21.15.131:6821","nonce":1364533794}]},"public_addr":"172.21.15.131:6817/1364533794","cluster_addr":"172.21.15.131:6819/1364533794","heartbeat_back_addr":"172.21.15.131:6823/1364533794","heartbeat_front_addr":"172.21.15.131:6821/1364533794","state":["exists","up"]},{"osd":9,"uuid":"4b2b3555-6870-4ead-be50-b635799265a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6827","nonce":2604133190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6829","nonce":2604133190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6833","nonce":2604133190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2604133190},{"type":"v1","addr":"172.21.15.40:6831","nonce":2604133190}]},"public_addr":"172.21.15.40:6827/2604133190","cluster_addr":"172.21.15.40:6829/2604133190","heartbeat_back_addr":"172.21.15.40:6833/2604133190","heartbeat_front_addr":"172.21.15.40:6831/2604133190","state":["exists","up"]},{"osd":10,"uuid":"70049e24-94ce-458e-97e3-dbcf9de05444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6825","nonce":2231164832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6827","nonce":2231164832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6831","nonce":2231164832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":2231164832},{"type":"v1","addr":"172.21.15.104:6829","nonce":2231164832}]},"public_addr":"172.21.15.104:6825/2231164832","cluster_addr":"172.21.15.104:6827/2231164832","heartbeat_back_addr":"172.21.15.104:6831/2231164832","heartbeat_front_addr":"172.21.15.104:6829/2231164832","state":["exists","up"]},{"osd":11,"uuid":"80447548-1fbe-4892-ac13-eb4bb6acf10d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6825","nonce":4128185670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6827","nonce":4128185670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6831","nonce":4128185670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":4128185670},{"type":"v1","addr":"172.21.15.131:6829","nonce":4128185670}]},"public_addr":"172.21.15.131:6825/4128185670","cluster_addr":"172.21.15.131:6827/4128185670","heartbeat_back_addr":"172.21.15.131:6831/4128185670","heartbeat_front_addr":"172.21.15.131:6829/4128185670","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:23.289784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:29:47.271677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:10.866792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:30:36.781924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:01.995848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:26.904606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:31:52.997438+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:19.221567+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:32:44.543977+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:11.948545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:33:38.402943+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T03:34:04.712200+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/889893607":"2024-06-07T03:38:37.622492+0000","172.21.15.40:6801/869712339":"2024-06-07T03:38:37.622492+0000","172.21.15.40:0/1294024108":"2024-06-07T03:38:37.622492+0000","172.21.15.40:0/23942578":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/1915158516":"2024-06-07T03:38:37.622492+0000","172.21.15.40:6800/869712339":"2024-06-07T03:38:37.622492+0000","172.21.15.40:0/2234980855":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6801/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:6800/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/2301695357":"2024-06-07T03:27:47.578838+0000","172.21.15.40:0/1676880727":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/2237379034":"2024-06-07T03:38:37.622492+0000","172.21.15.40:6801/3930494734":"2024-06-07T03:27:47.578838+0000","172.21.15.40:6800/1920198673":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/1597665640":"2024-06-07T03:27:19.799499+0000","172.21.15.40:0/513375601":"2024-06-07T03:27:19.799499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T03:49:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.329+0000 7f606aafd700 1 -- 172.21.15.40:0/2803083847 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f60540a2f30 msgr2=0x7f60540a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.329+0000 7f606aafd700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f60540a2f30 0x7f60540a53f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6050005fd0 tx=0x7f6050005dc0 comp rx=0 tx=0).stop 2024-06-06T03:49:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.329+0000 7f606aafd700 1 -- 172.21.15.40:0/2803083847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60641019d0 msgr2=0x7f6064197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.329+0000 7f606aafd700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60641019d0 0x7f6064197bd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f605800b530 tx=0x7f605800b8f0 comp rx=0 tx=0).stop 2024-06-06T03:49:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.329+0000 7f606aafd700 1 -- 172.21.15.40:0/2803083847 shutdown_connections 2024-06-06T03:49:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.329+0000 7f606aafd700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f60540a2f30 0x7f60540a53f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.329+0000 7f606aafd700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6064106720 0x7f606419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.329+0000 7f606aafd700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60641019d0 0x7f6064197bd0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.329+0000 7f606aafd700 1 --2- 172.21.15.40:0/2803083847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6064102380 0x7f6064198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.329+0000 7f606aafd700 1 -- 172.21.15.40:0/2803083847 >> 172.21.15.40:0/2803083847 conn(0x7f6064078340 msgr2=0x7f60640ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.329+0000 7f606aafd700 1 -- 172.21.15.40:0/2803083847 shutdown_connections 2024-06-06T03:49:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:42.329+0000 7f606aafd700 1 -- 172.21.15.40:0/2803083847 wait complete. 2024-06-06T03:49:43.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs fail cephfs 2024-06-06T03:49:43.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:42 smithi040 ceph-mon[32220]: pgmap v345: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 161 KiB/s wr, 17 op/s 2024-06-06T03:49:43.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:42 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/2803083847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:49:43.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:42 smithi131 ceph-mon[36687]: pgmap v345: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 161 KiB/s wr, 17 op/s 2024-06-06T03:49:43.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:42 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/2803083847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:49:43.300 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:49:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:42 smithi104 ceph-mon[36569]: pgmap v345: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 161 KiB/s wr, 17 op/s 2024-06-06T03:49:43.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:42 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/2803083847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T03:49:44.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.991+0000 7f70ede66700 1 -- 172.21.15.40:0/2126761956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70e8102900 msgr2=0x7f70e810e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:44.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.991+0000 7f70ede66700 1 --2- 172.21.15.40:0/2126761956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70e8102900 0x7f70e810e310 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f70e000b0d0 tx=0x7f70e000b3e0 comp rx=0 tx=0).stop 2024-06-06T03:49:44.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.991+0000 7f70ede66700 1 -- 172.21.15.40:0/2126761956 shutdown_connections 2024-06-06T03:49:44.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.991+0000 7f70ede66700 1 --2- 172.21.15.40:0/2126761956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70e8107660 0x7f70e8107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:44.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.991+0000 7f70ede66700 1 --2- 172.21.15.40:0/2126761956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70e8102900 0x7f70e810e310 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:44.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.991+0000 7f70ede66700 1 --2- 172.21.15.40:0/2126761956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70e8101f40 0x7f70e81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.991+0000 7f70ede66700 1 -- 172.21.15.40:0/2126761956 >> 172.21.15.40:0/2126761956 conn(0x7f70e80766b0 msgr2=0x7f70e8076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.991+0000 7f70ede66700 1 -- 172.21.15.40:0/2126761956 shutdown_connections 2024-06-06T03:49:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.991+0000 7f70ede66700 1 -- 172.21.15.40:0/2126761956 wait complete. 2024-06-06T03:49:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.991+0000 7f70ede66700 1 Processor -- start 2024-06-06T03:49:44.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.992+0000 7f70ede66700 1 -- start start 2024-06-06T03:49:44.997 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:44 smithi040 ceph-mon[32220]: pgmap v346: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 160 KiB/s wr, 15 op/s 2024-06-06T03:49:45.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.992+0000 7f70ede66700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70e8101f40 0x7f70e8198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:45.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.992+0000 7f70ede66700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70e8102900 0x7f70e8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:45.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.992+0000 7f70ede66700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70e8107660 0x7f70e819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:45.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.992+0000 7f70ede66700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81118b0 con 0x7f70e8107660 2024-06-06T03:49:45.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.992+0000 7f70ede66700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8111730 con 0x7f70e8101f40 2024-06-06T03:49:45.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.992+0000 7f70ede66700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8111a30 con 0x7f70e8102900 2024-06-06T03:49:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.992+0000 7f70e7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70e8107660 0x7f70e819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.992+0000 7f70e7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70e8107660 0x7f70e819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49248/0 (socket says 172.21.15.40:49248) 2024-06-06T03:49:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.992+0000 7f70e7fff700 1 -- 172.21.15.40:0/3712361677 learned_addr learned my addr 172.21.15.40:0/3712361677 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:49:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.992+0000 7f70e77fe700 1 --2- 172.21.15.40:0/3712361677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70e8101f40 0x7f70e8198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.992+0000 7f70e6ffd700 1 --2- 172.21.15.40:0/3712361677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70e8102900 0x7f70e8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.993+0000 7f70e7fff700 1 -- 172.21.15.40:0/3712361677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70e8102900 msgr2=0x7f70e8199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.993+0000 7f70e7fff700 1 --2- 172.21.15.40:0/3712361677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70e8102900 0x7f70e8199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.993+0000 7f70e7fff700 1 -- 172.21.15.40:0/3712361677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70e8101f40 msgr2=0x7f70e8198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.993+0000 7f70e7fff700 1 --2- 172.21.15.40:0/3712361677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70e8101f40 0x7f70e8198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.993+0000 7f70e7fff700 1 -- 172.21.15.40:0/3712361677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e819de00 con 0x7f70e8107660 2024-06-06T03:49:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.993+0000 7f70e7fff700 1 --2- 172.21.15.40:0/3712361677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70e8107660 0x7f70e819d6c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f70e0004da0 tx=0x7f70e0004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:45.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.993+0000 7f70e4ff9700 1 -- 172.21.15.40:0/3712361677 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e0021020 con 0x7f70e8107660 2024-06-06T03:49:45.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.993+0000 7f70e4ff9700 1 -- 172.21.15.40:0/3712361677 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70e0007b80 con 0x7f70e8107660 2024-06-06T03:49:45.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.993+0000 7f70ede66700 1 -- 172.21.15.40:0/3712361677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e819e090 con 0x7f70e8107660 2024-06-06T03:49:45.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.993+0000 7f70e4ff9700 1 -- 172.21.15.40:0/3712361677 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e00052a0 con 0x7f70e8107660 2024-06-06T03:49:45.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.993+0000 7f70ede66700 1 -- 172.21.15.40:0/3712361677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e81a5970 con 0x7f70e8107660 2024-06-06T03:49:45.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.994+0000 7f70ede66700 1 -- 172.21.15.40:0/3712361677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e8069180 con 0x7f70e8107660 2024-06-06T03:49:45.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.996+0000 7f70e4ff9700 1 -- 172.21.15.40:0/3712361677 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f70e0024020 con 0x7f70e8107660 2024-06-06T03:49:45.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.996+0000 7f70e4ff9700 1 --2- 172.21.15.40:0/3712361677 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f70d40a2e10 0x7f70d40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:45.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.996+0000 7f70e4ff9700 1 -- 172.21.15.40:0/3712361677 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8426+0+0 (secure 0 0 0) 0x7f70e000f020 con 0x7f70e8107660 2024-06-06T03:49:45.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.996+0000 7f70e77fe700 1 --2- 172.21.15.40:0/3712361677 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f70d40a2e10 0x7f70d40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:45.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:44.997+0000 7f70e77fe700 1 --2- 172.21.15.40:0/3712361677 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f70d40a2e10 0x7f70d40a52d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009a10 tx=0x7f70d0009190 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:45.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.001+0000 7f70e4ff9700 1 -- 172.21.15.40:0/3712361677 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f70e0007d30 con 0x7f70e8107660 2024-06-06T03:49:45.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.256+0000 7f70ede66700 1 -- 172.21.15.40:0/3712361677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f70e802dca0 con 0x7f70e8107660 2024-06-06T03:49:45.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.259+0000 7f70e4ff9700 1 -- 172.21.15.40:0/3712361677 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) ==== 161+0+0 (secure 0 0 0) 0x7f70e009d2a0 con 0x7f70e8107660 2024-06-06T03:49:45.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.262+0000 7f70ede66700 1 -- 172.21.15.40:0/3712361677 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f70d40a2e10 msgr2=0x7f70d40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:45.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.262+0000 7f70ede66700 1 --2- 172.21.15.40:0/3712361677 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f70d40a2e10 0x7f70d40a52d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009a10 tx=0x7f70d0009190 comp rx=0 tx=0).stop 2024-06-06T03:49:45.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.262+0000 7f70ede66700 1 -- 172.21.15.40:0/3712361677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70e8107660 msgr2=0x7f70e819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:45.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.262+0000 7f70ede66700 1 --2- 172.21.15.40:0/3712361677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70e8107660 0x7f70e819d6c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f70e0004da0 tx=0x7f70e0004dd0 comp rx=0 tx=0).stop 2024-06-06T03:49:45.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.262+0000 7f70ede66700 1 -- 172.21.15.40:0/3712361677 shutdown_connections 2024-06-06T03:49:45.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.262+0000 7f70ede66700 1 --2- 172.21.15.40:0/3712361677 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f70d40a2e10 0x7f70d40a52d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:45.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.262+0000 7f70ede66700 1 --2- 172.21.15.40:0/3712361677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70e8107660 0x7f70e819d6c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:45.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.262+0000 7f70ede66700 1 --2- 172.21.15.40:0/3712361677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70e8101f40 0x7f70e8198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:45.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.263+0000 7f70ede66700 1 --2- 172.21.15.40:0/3712361677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70e8102900 0x7f70e8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:45.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.263+0000 7f70ede66700 1 -- 172.21.15.40:0/3712361677 >> 172.21.15.40:0/3712361677 conn(0x7f70e80766b0 msgr2=0x7f70e8100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:45.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.263+0000 7f70ede66700 1 -- 172.21.15.40:0/3712361677 shutdown_connections 2024-06-06T03:49:45.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:45.263+0000 7f70ede66700 1 -- 172.21.15.40:0/3712361677 wait complete. 2024-06-06T03:49:45.267 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-06T03:49:45.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:44 smithi131 ceph-mon[36687]: pgmap v346: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 160 KiB/s wr, 15 op/s 2024-06-06T03:49:45.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:44 smithi104 ceph-mon[36569]: pgmap v346: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 160 KiB/s wr, 15 op/s 2024-06-06T03:49:45.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:45 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a[32216]: 2024-06-06T03:49:45.257+0000 7f2c9b96a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T03:49:45.581 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:49:45 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-a[74733]: ignoring --setuser ceph since I am not root 2024-06-06T03:49:45.581 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:49:45 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-a[74733]: ignoring --setgroup ceph since I am not root 2024-06-06T03:49:45.581 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:49:45 smithi040 ceph-mds[74737]: -- 172.21.15.40:0/180544007 <== mon.0 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5649bd6e1a40 con 0x5649bd6b5000 2024-06-06T03:49:45.581 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:49:45 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-a[74733]: starting mds.a at 2024-06-06T03:49:45.582 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jun 06 03:49:45 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-g[74297]: ignoring --setuser ceph since I am not root 2024-06-06T03:49:45.582 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jun 06 03:49:45 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-g[74297]: ignoring --setgroup ceph since I am not root 2024-06-06T03:49:45.582 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jun 06 03:49:45 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-g[74297]: starting mds.g at 2024-06-06T03:49:45.782 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-i[68123]: ignoring --setuser ceph since I am not root 2024-06-06T03:49:45.783 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-i[68123]: ignoring --setgroup ceph since I am not root 2024-06-06T03:49:45.783 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-mds[68127]: -- 172.21.15.131:0/3166696343 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557b63e69a40 con 0x557b63e3c800 2024-06-06T03:49:45.783 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-i[68123]: starting mds.i at 2024-06-06T03:49:45.783 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-c[67822]: ignoring --setuser ceph since I am not root 2024-06-06T03:49:45.784 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-c[67822]: ignoring --setgroup ceph since I am not root 2024-06-06T03:49:45.784 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-mds[67827]: -- 172.21.15.131:0/362086612 <== mon.0 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5632160e5a40 con 0x5632160b8800 2024-06-06T03:49:45.784 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-c[67822]: starting mds.c at 2024-06-06T03:49:45.784 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-f[67516]: ignoring --setuser ceph since I am not root 2024-06-06T03:49:45.784 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-f[67516]: ignoring --setgroup ceph since I am not root 2024-06-06T03:49:45.784 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-mds[67520]: -- 172.21.15.131:0/221329529 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e1648fa40 con 0x558e16462c00 2024-06-06T03:49:45.785 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-f[67516]: starting mds.f at 2024-06-06T03:49:45.785 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-l[67205]: ignoring --setuser ceph since I am not root 2024-06-06T03:49:45.785 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-l[67205]: ignoring --setgroup ceph since I am not root 2024-06-06T03:49:45.785 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-mds[67209]: -- 172.21.15.131:0/85180497 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562597fbba40 con 0x562597f8ec00 2024-06-06T03:49:45.785 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-l[67205]: starting mds.l at 2024-06-06T03:49:45.841 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-e[66322]: ignoring --setuser ceph since I am not root 2024-06-06T03:49:45.842 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-e[66322]: ignoring --setgroup ceph since I am not root 2024-06-06T03:49:45.842 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-mds[66326]: -- 172.21.15.104:0/857744267 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56000a085a40 con 0x56000a058800 2024-06-06T03:49:45.842 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-e[66322]: starting mds.e at 2024-06-06T03:49:45.843 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b[66930]: ignoring --setuser ceph since I am not root 2024-06-06T03:49:45.843 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b[66930]: ignoring --setgroup ceph since I am not root 2024-06-06T03:49:45.843 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-mds[66934]: -- 172.21.15.104:0/1679030098 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560b856a9a40 con 0x560b8567c800 2024-06-06T03:49:45.843 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b[66930]: starting mds.b at 2024-06-06T03:49:45.844 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-h[66627]: ignoring --setuser ceph since I am not root 2024-06-06T03:49:45.844 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-h[66627]: ignoring --setgroup ceph since I am not root 2024-06-06T03:49:45.844 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-mds[66631]: -- 172.21.15.104:0/4089324649 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557701743a40 con 0x557701716800 2024-06-06T03:49:45.844 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-h[66627]: starting mds.h at 2024-06-06T03:49:45.844 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-k[66007]: ignoring --setuser ceph since I am not root 2024-06-06T03:49:45.845 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-k[66007]: ignoring --setgroup ceph since I am not root 2024-06-06T03:49:45.845 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-mds[66011]: -- 172.21.15.104:0/55189053 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5630e537da40 con 0x5630e5350800 2024-06-06T03:49:45.845 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-k[66007]: starting mds.k at 2024-06-06T03:49:46.012 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-06T03:49:46.269 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:49:46.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3712361677' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-06T03:49:46.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-mon[36687]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-06T03:49:46.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-mon[36687]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T03:49:46.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/3712361677' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-06T03:49:46.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-mon[36687]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T03:49:46.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:45 smithi131 ceph-mon[36687]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-06T03:49:46.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:45 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3712361677' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-06T03:49:46.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:45 smithi040 ceph-mon[32220]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-06T03:49:46.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:45 smithi040 ceph-mon[32220]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T03:49:46.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:45 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/3712361677' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-06T03:49:46.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:45 smithi040 ceph-mon[32220]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T03:49:46.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:45 smithi040 ceph-mon[32220]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-06T03:49:46.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3712361677' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-06T03:49:46.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-mon[36569]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-06T03:49:46.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-mon[36569]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T03:49:46.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/3712361677' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-06T03:49:46.341 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-mon[36569]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T03:49:46.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:45 smithi104 ceph-mon[36569]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-06T03:49:47.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: pgmap v348: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-06-06T03:49:47.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.40:6838/1721402981,v1:172.21.15.40:6839/1721402981] up:boot 2024-06-06T03:49:47.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.40:6840/3788253571,v1:172.21.15.40:6841/3788253571] up:boot 2024-06-06T03:49:47.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.131:6833/2367367618,v1:172.21.15.131:6835/2367367618] up:boot 2024-06-06T03:49:47.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.131:6832/3875142804,v1:172.21.15.131:6834/3875142804] up:boot 2024-06-06T03:49:47.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.131:6836/1962021656,v1:172.21.15.131:6837/1962021656] up:boot 2024-06-06T03:49:47.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.131:6838/503992749,v1:172.21.15.131:6839/503992749] up:boot 2024-06-06T03:49:47.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.104:6832/4000310358,v1:172.21.15.104:6834/4000310358] up:boot 2024-06-06T03:49:47.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.104:6833/2724400552,v1:172.21.15.104:6835/2724400552] up:boot 2024-06-06T03:49:47.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.104:6836/3112672320,v1:172.21.15.104:6838/3112672320] up:boot 2024-06-06T03:49:47.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: mds.? [v2:172.21.15.104:6837/4033083839,v1:172.21.15.104:6839/4033083839] up:boot 2024-06-06T03:49:47.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-06T03:49:47.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T03:49:47.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T03:49:47.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T03:49:47.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T03:49:47.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T03:49:47.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T03:49:47.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T03:49:47.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T03:49:47.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T03:49:47.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:47 smithi131 ceph-mon[36687]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T03:49:47.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: pgmap v348: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-06-06T03:49:47.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.40:6838/1721402981,v1:172.21.15.40:6839/1721402981] up:boot 2024-06-06T03:49:47.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.40:6840/3788253571,v1:172.21.15.40:6841/3788253571] up:boot 2024-06-06T03:49:47.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.131:6833/2367367618,v1:172.21.15.131:6835/2367367618] up:boot 2024-06-06T03:49:47.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.131:6832/3875142804,v1:172.21.15.131:6834/3875142804] up:boot 2024-06-06T03:49:47.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.131:6836/1962021656,v1:172.21.15.131:6837/1962021656] up:boot 2024-06-06T03:49:47.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.131:6838/503992749,v1:172.21.15.131:6839/503992749] up:boot 2024-06-06T03:49:47.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.104:6832/4000310358,v1:172.21.15.104:6834/4000310358] up:boot 2024-06-06T03:49:47.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.104:6833/2724400552,v1:172.21.15.104:6835/2724400552] up:boot 2024-06-06T03:49:47.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.104:6836/3112672320,v1:172.21.15.104:6838/3112672320] up:boot 2024-06-06T03:49:47.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: mds.? [v2:172.21.15.104:6837/4033083839,v1:172.21.15.104:6839/4033083839] up:boot 2024-06-06T03:49:47.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-06T03:49:47.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T03:49:47.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T03:49:47.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T03:49:47.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T03:49:47.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T03:49:47.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T03:49:47.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T03:49:47.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T03:49:47.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T03:49:47.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:47 smithi040 ceph-mon[32220]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T03:49:47.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: pgmap v348: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-06-06T03:49:47.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.40:6838/1721402981,v1:172.21.15.40:6839/1721402981] up:boot 2024-06-06T03:49:47.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.40:6840/3788253571,v1:172.21.15.40:6841/3788253571] up:boot 2024-06-06T03:49:47.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.131:6833/2367367618,v1:172.21.15.131:6835/2367367618] up:boot 2024-06-06T03:49:47.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.131:6832/3875142804,v1:172.21.15.131:6834/3875142804] up:boot 2024-06-06T03:49:47.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.131:6836/1962021656,v1:172.21.15.131:6837/1962021656] up:boot 2024-06-06T03:49:47.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.131:6838/503992749,v1:172.21.15.131:6839/503992749] up:boot 2024-06-06T03:49:47.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.104:6832/4000310358,v1:172.21.15.104:6834/4000310358] up:boot 2024-06-06T03:49:47.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.104:6833/2724400552,v1:172.21.15.104:6835/2724400552] up:boot 2024-06-06T03:49:47.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.104:6836/3112672320,v1:172.21.15.104:6838/3112672320] up:boot 2024-06-06T03:49:47.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: mds.? [v2:172.21.15.104:6837/4033083839,v1:172.21.15.104:6839/4033083839] up:boot 2024-06-06T03:49:47.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-06T03:49:47.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T03:49:47.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T03:49:47.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T03:49:47.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T03:49:47.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T03:49:47.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T03:49:47.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T03:49:47.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T03:49:47.594 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T03:49:47.594 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:47 smithi104 ceph-mon[36569]: from='mgr.24790 172.21.15.40:0/2921406847' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T03:49:48.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.030+0000 7f79a7ebf700 1 -- 172.21.15.40:0/94866132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79a006ac40 msgr2=0x7f79a006b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:48.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.030+0000 7f79a7ebf700 1 --2- 172.21.15.40:0/94866132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79a006ac40 0x7f79a006b020 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f799c009990 tx=0x7f799c009ca0 comp rx=0 tx=0).stop 2024-06-06T03:49:48.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.030+0000 7f79a7ebf700 1 -- 172.21.15.40:0/94866132 shutdown_connections 2024-06-06T03:49:48.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.030+0000 7f79a7ebf700 1 --2- 172.21.15.40:0/94866132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79a010eee0 0x7f79a01112d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:48.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.030+0000 7f79a7ebf700 1 --2- 172.21.15.40:0/94866132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79a006ac40 0x7f79a006b020 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:48.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.030+0000 7f79a7ebf700 1 --2- 172.21.15.40:0/94866132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79a006b560 0x7f79a010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:48.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.030+0000 7f79a7ebf700 1 -- 172.21.15.40:0/94866132 >> 172.21.15.40:0/94866132 conn(0x7f79a0078430 msgr2=0x7f79a0078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:48.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.030+0000 7f79a7ebf700 1 -- 172.21.15.40:0/94866132 shutdown_connections 2024-06-06T03:49:48.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.030+0000 7f79a7ebf700 1 -- 172.21.15.40:0/94866132 wait complete. 2024-06-06T03:49:48.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.031+0000 7f79a7ebf700 1 Processor -- start 2024-06-06T03:49:48.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.031+0000 7f79a7ebf700 1 -- start start 2024-06-06T03:49:48.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.031+0000 7f79a7ebf700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79a006ac40 0x7f79a019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:48.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.031+0000 7f79a7ebf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79a006b560 0x7f79a019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:48.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.031+0000 7f79a7ebf700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79a010eee0 0x7f79a019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:48.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.031+0000 7f79a7ebf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0114a60 con 0x7f79a006b560 2024-06-06T03:49:48.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.031+0000 7f79a7ebf700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01148e0 con 0x7f79a010eee0 2024-06-06T03:49:48.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.031+0000 7f79a7ebf700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0114be0 con 0x7f79a006ac40 2024-06-06T03:49:48.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.031+0000 7f79a545a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79a006b560 0x7f79a019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:48.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.032+0000 7f79a545a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79a006b560 0x7f79a019b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49298/0 (socket says 172.21.15.40:49298) 2024-06-06T03:49:48.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.032+0000 7f79a545a700 1 -- 172.21.15.40:0/885595128 learned_addr learned my addr 172.21.15.40:0/885595128 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:49:48.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.032+0000 7f79a645c700 1 --2- 172.21.15.40:0/885595128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79a010eee0 0x7f79a019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:48.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.032+0000 7f79a545a700 1 -- 172.21.15.40:0/885595128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79a006ac40 msgr2=0x7f79a019af50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T03:49:48.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.032+0000 7f79a545a700 1 --2- 172.21.15.40:0/885595128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79a006ac40 0x7f79a019af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:48.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.032+0000 7f79a545a700 1 -- 172.21.15.40:0/885595128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79a010eee0 msgr2=0x7f79a019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:48.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.032+0000 7f79a545a700 1 --2- 172.21.15.40:0/885595128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79a010eee0 0x7f79a019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:48.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.032+0000 7f79a545a700 1 -- 172.21.15.40:0/885595128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a019ff60 con 0x7f79a006b560 2024-06-06T03:49:48.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.032+0000 7f79a545a700 1 --2- 172.21.15.40:0/885595128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79a006b560 0x7f79a019b490 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f799000c920 tx=0x7f799000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:48.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.032+0000 7f7996ffd700 1 -- 172.21.15.40:0/885595128 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990007b00 con 0x7f79a006b560 2024-06-06T03:49:48.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.032+0000 7f7996ffd700 1 -- 172.21.15.40:0/885595128 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7990004d10 con 0x7f79a006b560 2024-06-06T03:49:48.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.032+0000 7f79a7ebf700 1 -- 172.21.15.40:0/885595128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a01a0250 con 0x7f79a006b560 2024-06-06T03:49:48.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.032+0000 7f7996ffd700 1 -- 172.21.15.40:0/885595128 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990005680 con 0x7f79a006b560 2024-06-06T03:49:48.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.032+0000 7f79a7ebf700 1 -- 172.21.15.40:0/885595128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a01a7ad0 con 0x7f79a006b560 2024-06-06T03:49:48.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.033+0000 7f79a7ebf700 1 -- 172.21.15.40:0/885595128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a0069180 con 0x7f79a006b560 2024-06-06T03:49:48.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.034+0000 7f7996ffd700 1 -- 172.21.15.40:0/885595128 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7990004750 con 0x7f79a006b560 2024-06-06T03:49:48.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.035+0000 7f7996ffd700 1 --2- 172.21.15.40:0/885595128 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f798c0a2d90 0x7f798c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:48.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.035+0000 7f7996ffd700 1 -- 172.21.15.40:0/885595128 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8812+0+0 (secure 0 0 0) 0x7f79900cf160 con 0x7f79a006b560 2024-06-06T03:49:48.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.035+0000 7f79a5c5b700 1 --2- 172.21.15.40:0/885595128 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f798c0a2d90 0x7f798c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:48.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.036+0000 7f79a5c5b700 1 --2- 172.21.15.40:0/885595128 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f798c0a2d90 0x7f798c0a5250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f799c005b40 tx=0x7f799c005a90 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:48.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.040+0000 7f7996ffd700 1 -- 172.21.15.40:0/885595128 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7990014030 con 0x7f79a006b560 2024-06-06T03:49:48.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:48.292+0000 7f79a7ebf700 1 -- 172.21.15.40:0/885595128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f79a002dca0 con 0x7f79a006b560 2024-06-06T03:49:49.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:49.276+0000 7f7996ffd700 1 -- 172.21.15.40:0/885595128 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) ==== 104+0+0 (secure 0 0 0) 0x7f799009bc00 con 0x7f79a006b560 2024-06-06T03:49:49.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:49.279+0000 7f79a7ebf700 1 -- 172.21.15.40:0/885595128 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f798c0a2d90 msgr2=0x7f798c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:49.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:49.279+0000 7f79a7ebf700 1 --2- 172.21.15.40:0/885595128 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f798c0a2d90 0x7f798c0a5250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f799c005b40 tx=0x7f799c005a90 comp rx=0 tx=0).stop 2024-06-06T03:49:49.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:49.279+0000 7f79a7ebf700 1 -- 172.21.15.40:0/885595128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79a006b560 msgr2=0x7f79a019b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:49.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:49.279+0000 7f79a7ebf700 1 --2- 172.21.15.40:0/885595128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79a006b560 0x7f79a019b490 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f799000c920 tx=0x7f799000cc30 comp rx=0 tx=0).stop 2024-06-06T03:49:49.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:49.279+0000 7f79a7ebf700 1 -- 172.21.15.40:0/885595128 shutdown_connections 2024-06-06T03:49:49.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:49.279+0000 7f79a7ebf700 1 --2- 172.21.15.40:0/885595128 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f798c0a2d90 0x7f798c0a5250 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:49.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:49.279+0000 7f79a7ebf700 1 --2- 172.21.15.40:0/885595128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79a006ac40 0x7f79a019af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:49.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:49.279+0000 7f79a7ebf700 1 --2- 172.21.15.40:0/885595128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79a010eee0 0x7f79a019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:49.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:49.279+0000 7f79a7ebf700 1 --2- 172.21.15.40:0/885595128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79a006b560 0x7f79a019b490 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:49.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:49.279+0000 7f79a7ebf700 1 -- 172.21.15.40:0/885595128 >> 172.21.15.40:0/885595128 conn(0x7f79a0078430 msgr2=0x7f79a01102b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:49.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:49.280+0000 7f79a7ebf700 1 -- 172.21.15.40:0/885595128 shutdown_connections 2024-06-06T03:49:49.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:49.280+0000 7f79a7ebf700 1 -- 172.21.15.40:0/885595128 wait complete. 2024-06-06T03:49:49.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:49 smithi131 ceph-mon[36687]: pgmap v349: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-06-06T03:49:49.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:49 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/885595128' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T03:49:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:49 smithi040 ceph-mon[32220]: pgmap v349: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-06-06T03:49:49.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:49 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/885595128' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T03:49:49.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:49 smithi104 ceph-mon[36569]: pgmap v349: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-06-06T03:49:49.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:49 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/885595128' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T03:49:50.036 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-06T03:49:50.293 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:49:50.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:50 smithi131 ceph-mon[36687]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-06T03:49:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:50 smithi131 ceph-mon[36687]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-06T03:49:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:50 smithi131 ceph-mon[36687]: Cluster is now healthy 2024-06-06T03:49:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:50 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/885595128' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-06T03:49:50.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:50 smithi131 ceph-mon[36687]: fsmap 12 up:standby 2024-06-06T03:49:50.557 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:50 smithi040 ceph-mon[32220]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-06T03:49:50.558 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:50 smithi040 ceph-mon[32220]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-06T03:49:50.558 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:50 smithi040 ceph-mon[32220]: Cluster is now healthy 2024-06-06T03:49:50.558 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:50 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/885595128' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-06T03:49:50.558 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:50 smithi040 ceph-mon[32220]: fsmap 12 up:standby 2024-06-06T03:49:50.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:50 smithi104 ceph-mon[36569]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-06T03:49:50.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:50 smithi104 ceph-mon[36569]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-06T03:49:50.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:50 smithi104 ceph-mon[36569]: Cluster is now healthy 2024-06-06T03:49:50.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:50 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/885595128' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-06T03:49:50.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:50 smithi104 ceph-mon[36569]: fsmap 12 up:standby 2024-06-06T03:49:51.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:51 smithi040 ceph-mon[32220]: pgmap v350: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-06-06T03:49:51.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:51 smithi131 ceph-mon[36687]: pgmap v350: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-06-06T03:49:51.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:51 smithi104 ceph-mon[36569]: pgmap v350: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-06-06T03:49:52.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.000+0000 7f3c2df9e700 1 -- 172.21.15.40:0/2269893099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c28102f10 msgr2=0x7f3c2810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:52.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.000+0000 7f3c2df9e700 1 --2- 172.21.15.40:0/2269893099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c28102f10 0x7f3c2810f330 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000b0a0 tx=0x7f3c2000b3b0 comp rx=0 tx=0).stop 2024-06-06T03:49:52.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.000+0000 7f3c2df9e700 1 -- 172.21.15.40:0/2269893099 shutdown_connections 2024-06-06T03:49:52.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.000+0000 7f3c2df9e700 1 --2- 172.21.15.40:0/2269893099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c281084f0 0x7f3c281088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:52.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.000+0000 7f3c2df9e700 1 --2- 172.21.15.40:0/2269893099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c28102f10 0x7f3c2810f330 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:52.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.000+0000 7f3c2df9e700 1 --2- 172.21.15.40:0/2269893099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c28102550 0x7f3c281029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:52.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.000+0000 7f3c2df9e700 1 -- 172.21.15.40:0/2269893099 >> 172.21.15.40:0/2269893099 conn(0x7f3c28079650 msgr2=0x7f3c28079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:52.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.000+0000 7f3c2df9e700 1 -- 172.21.15.40:0/2269893099 shutdown_connections 2024-06-06T03:49:52.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.000+0000 7f3c2df9e700 1 -- 172.21.15.40:0/2269893099 wait complete. 2024-06-06T03:49:52.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.001+0000 7f3c2df9e700 1 Processor -- start 2024-06-06T03:49:52.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.001+0000 7f3c2df9e700 1 -- start start 2024-06-06T03:49:52.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.001+0000 7f3c2df9e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c28102550 0x7f3c2819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:52.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.001+0000 7f3c2df9e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c28102f10 0x7f3c2819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:52.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.001+0000 7f3c2df9e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c281084f0 0x7f3c281989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:52.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.001+0000 7f3c2df9e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281127d0 con 0x7f3c28102550 2024-06-06T03:49:52.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.001+0000 7f3c2df9e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3c28112650 con 0x7f3c281084f0 2024-06-06T03:49:52.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.001+0000 7f3c2df9e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3c28112950 con 0x7f3c28102f10 2024-06-06T03:49:52.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.001+0000 7f3c277fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c28102550 0x7f3c2819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:52.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.001+0000 7f3c277fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c28102550 0x7f3c2819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52760/0 (socket says 172.21.15.40:52760) 2024-06-06T03:49:52.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.001+0000 7f3c277fe700 1 -- 172.21.15.40:0/1851397701 learned_addr learned my addr 172.21.15.40:0/1851397701 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:49:52.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.001+0000 7f3c26ffd700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c28102f10 0x7f3c2819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:52.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.001+0000 7f3c27fff700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c281084f0 0x7f3c281989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:52.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.002+0000 7f3c277fe700 1 -- 172.21.15.40:0/1851397701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c28102f10 msgr2=0x7f3c2819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:52.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.002+0000 7f3c277fe700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c28102f10 0x7f3c2819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:52.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.002+0000 7f3c277fe700 1 -- 172.21.15.40:0/1851397701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c281084f0 msgr2=0x7f3c281989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:52.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.002+0000 7f3c277fe700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c281084f0 0x7f3c281989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:52.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.002+0000 7f3c277fe700 1 -- 172.21.15.40:0/1851397701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c28199150 con 0x7f3c28102550 2024-06-06T03:49:52.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.002+0000 7f3c27fff700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c281084f0 0x7f3c281989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T03:49:52.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.002+0000 7f3c277fe700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c28102550 0x7f3c2819e830 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800b530 tx=0x7f3c1800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:52.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.002+0000 7f3c24ff9700 1 -- 172.21.15.40:0/1851397701 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18015020 con 0x7f3c28102550 2024-06-06T03:49:52.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.002+0000 7f3c24ff9700 1 -- 172.21.15.40:0/1851397701 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c18011d10 con 0x7f3c28102550 2024-06-06T03:49:52.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.002+0000 7f3c2df9e700 1 -- 172.21.15.40:0/1851397701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c28199440 con 0x7f3c28102550 2024-06-06T03:49:52.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.002+0000 7f3c24ff9700 1 -- 172.21.15.40:0/1851397701 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1800f2e0 con 0x7f3c28102550 2024-06-06T03:49:52.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.002+0000 7f3c2df9e700 1 -- 172.21.15.40:0/1851397701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c281a5700 con 0x7f3c28102550 2024-06-06T03:49:52.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.003+0000 7f3c2df9e700 1 -- 172.21.15.40:0/1851397701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c28103ba0 con 0x7f3c28102550 2024-06-06T03:49:52.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.005+0000 7f3c24ff9700 1 -- 172.21.15.40:0/1851397701 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3c18018020 con 0x7f3c28102550 2024-06-06T03:49:52.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.006+0000 7f3c24ff9700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f3c100a2d90 0x7f3c100a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:52.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.006+0000 7f3c24ff9700 1 -- 172.21.15.40:0/1851397701 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8812+0+0 (secure 0 0 0) 0x7f3c180cf1e0 con 0x7f3c28102550 2024-06-06T03:49:52.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.006+0000 7f3c26ffd700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f3c100a2d90 0x7f3c100a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:52.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.006+0000 7f3c26ffd700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f3c100a2d90 0x7f3c100a5250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3c28079a90 tx=0x7f3c1c00a380 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:52.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.009+0000 7f3c24ff9700 1 -- 172.21.15.40:0/1851397701 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3c1801f020 con 0x7f3c28102550 2024-06-06T03:49:52.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.214+0000 7f3c2df9e700 1 -- 172.21.15.40:0/1851397701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3c28069180 con 0x7f3c28102550 2024-06-06T03:49:52.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.286+0000 7f3c24ff9700 1 -- 172.21.15.40:0/1851397701 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) ==== 181+0+0 (secure 0 0 0) 0x7f3c1809be90 con 0x7f3c28102550 2024-06-06T03:49:52.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.290+0000 7f3c2df9e700 1 -- 172.21.15.40:0/1851397701 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f3c100a2d90 msgr2=0x7f3c100a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:52.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.290+0000 7f3c2df9e700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f3c100a2d90 0x7f3c100a5250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3c28079a90 tx=0x7f3c1c00a380 comp rx=0 tx=0).stop 2024-06-06T03:49:52.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.290+0000 7f3c2df9e700 1 -- 172.21.15.40:0/1851397701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c28102550 msgr2=0x7f3c2819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:52.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.290+0000 7f3c2df9e700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c28102550 0x7f3c2819e830 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800b530 tx=0x7f3c1800b840 comp rx=0 tx=0).stop 2024-06-06T03:49:52.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.290+0000 7f3c2df9e700 1 -- 172.21.15.40:0/1851397701 shutdown_connections 2024-06-06T03:49:52.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.290+0000 7f3c2df9e700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f3c100a2d90 0x7f3c100a5250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:52.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.290+0000 7f3c2df9e700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c281084f0 0x7f3c281989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:52.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.290+0000 7f3c2df9e700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c28102550 0x7f3c2819e830 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:52.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.290+0000 7f3c2df9e700 1 --2- 172.21.15.40:0/1851397701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c28102f10 0x7f3c2819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:52.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.290+0000 7f3c2df9e700 1 -- 172.21.15.40:0/1851397701 >> 172.21.15.40:0/1851397701 conn(0x7f3c28079650 msgr2=0x7f3c28100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:52.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.291+0000 7f3c2df9e700 1 -- 172.21.15.40:0/1851397701 shutdown_connections 2024-06-06T03:49:52.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:52.292+0000 7f3c2df9e700 1 -- 172.21.15.40:0/1851397701 wait complete. 2024-06-06T03:49:52.294 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-06-06T03:49:52.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:52 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1851397701' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T03:49:52.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:52 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1851397701' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T03:49:52.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:52 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1851397701' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T03:49:53.065 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-06T03:49:53.321 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T03:49:53.568 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:53 smithi040 ceph-mon[32220]: pgmap v351: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 KiB/s wr, 6 op/s 2024-06-06T03:49:53.569 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:53 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/1851397701' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T03:49:53.569 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:53 smithi040 ceph-mon[32220]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T03:49:53.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:53 smithi104 ceph-mon[36569]: pgmap v351: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 KiB/s wr, 6 op/s 2024-06-06T03:49:53.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:53 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/1851397701' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T03:49:53.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:53 smithi104 ceph-mon[36569]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T03:49:53.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:53 smithi131 ceph-mon[36687]: pgmap v351: 97 pgs: 97 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 KiB/s wr, 6 op/s 2024-06-06T03:49:53.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:53 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/1851397701' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T03:49:53.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:53 smithi131 ceph-mon[36687]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T03:49:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.026+0000 7f041d161700 1 -- 172.21.15.40:0/39521545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0418101390 msgr2=0x7f0418101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.026+0000 7f041d161700 1 --2- 172.21.15.40:0/39521545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0418101390 0x7f0418101810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f040c009990 tx=0x7f040c009ca0 comp rx=0 tx=0).stop 2024-06-06T03:49:55.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.026+0000 7f041d161700 1 -- 172.21.15.40:0/39521545 shutdown_connections 2024-06-06T03:49:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.026+0000 7f041d161700 1 --2- 172.21.15.40:0/39521545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0418101d50 0x7f041810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.026+0000 7f041d161700 1 --2- 172.21.15.40:0/39521545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0418107400 0x7f04181077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.026+0000 7f041d161700 1 --2- 172.21.15.40:0/39521545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0418101390 0x7f0418101810 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.026+0000 7f041d161700 1 -- 172.21.15.40:0/39521545 >> 172.21.15.40:0/39521545 conn(0x7f0418078340 msgr2=0x7f0418078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.026+0000 7f041d161700 1 -- 172.21.15.40:0/39521545 shutdown_connections 2024-06-06T03:49:55.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.026+0000 7f041d161700 1 -- 172.21.15.40:0/39521545 wait complete. 2024-06-06T03:49:55.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.027+0000 7f041d161700 1 Processor -- start 2024-06-06T03:49:55.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.027+0000 7f041d161700 1 -- start start 2024-06-06T03:49:55.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.027+0000 7f041d161700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0418101390 0x7f0418199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:55.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.027+0000 7f041d161700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0418101d50 0x7f041819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:55.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.027+0000 7f041d161700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0418107400 0x7f04181a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:55.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.027+0000 7f041d161700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0418111770 con 0x7f0418107400 2024-06-06T03:49:55.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.027+0000 7f041d161700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f04181115f0 con 0x7f0418101d50 2024-06-06T03:49:55.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.027+0000 7f041d161700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f04181118f0 con 0x7f0418101390 2024-06-06T03:49:55.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f041759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0418107400 0x7f04181a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:55.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f041759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0418107400 0x7f04181a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52770/0 (socket says 172.21.15.40:52770) 2024-06-06T03:49:55.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f041759e700 1 -- 172.21.15.40:0/498559513 learned_addr learned my addr 172.21.15.40:0/498559513 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-06-06T03:49:55.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f041659c700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0418101d50 0x7f041819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:55.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f0416d9d700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0418101390 0x7f0418199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:55.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f041759e700 1 -- 172.21.15.40:0/498559513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0418101390 msgr2=0x7f0418199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:55.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f041759e700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0418101390 0x7f0418199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:55.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f041759e700 1 -- 172.21.15.40:0/498559513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0418101d50 msgr2=0x7f041819a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:55.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f041759e700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0418101d50 0x7f041819a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:55.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f041759e700 1 -- 172.21.15.40:0/498559513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04181a5190 con 0x7f0418107400 2024-06-06T03:49:55.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f0416d9d700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0418101390 0x7f0418199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T03:49:55.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f041759e700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0418107400 0x7f04181a4c50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f041000b970 tx=0x7f041000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:55.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f03fffff700 1 -- 172.21.15.40:0/498559513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410019070 con 0x7f0418107400 2024-06-06T03:49:55.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f03fffff700 1 -- 172.21.15.40:0/498559513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f041000ce10 con 0x7f0418107400 2024-06-06T03:49:55.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f041d161700 1 -- 172.21.15.40:0/498559513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04181a53c0 con 0x7f0418107400 2024-06-06T03:49:55.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f03fffff700 1 -- 172.21.15.40:0/498559513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410007370 con 0x7f0418107400 2024-06-06T03:49:55.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.028+0000 7f041d161700 1 -- 172.21.15.40:0/498559513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04181a5880 con 0x7f0418107400 2024-06-06T03:49:55.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.029+0000 7f041d161700 1 -- 172.21.15.40:0/498559513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0418109970 con 0x7f0418107400 2024-06-06T03:49:55.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.031+0000 7f03fffff700 1 -- 172.21.15.40:0/498559513 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f041001c070 con 0x7f0418107400 2024-06-06T03:49:55.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.032+0000 7f03fffff700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f04000a2d90 0x7f04000a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T03:49:55.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.032+0000 7f0416d9d700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f04000a2d90 0x7f04000a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T03:49:55.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.032+0000 7f03fffff700 1 -- 172.21.15.40:0/498559513 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8316+0+0 (secure 0 0 0) 0x7f04100ceb20 con 0x7f0418107400 2024-06-06T03:49:55.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.032+0000 7f0416d9d700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f04000a2d90 0x7f04000a5250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f04080099d0 tx=0x7f0408008040 comp rx=0 tx=0).ready entity=mgr.24790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T03:49:55.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.035+0000 7f03fffff700 1 -- 172.21.15.40:0/498559513 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f041009b8c0 con 0x7f0418107400 2024-06-06T03:49:55.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.243+0000 7f041d161700 1 -- 172.21.15.40:0/498559513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f041806a760 con 0x7f0418107400 2024-06-06T03:49:55.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.297+0000 7f03fffff700 1 -- 172.21.15.40:0/498559513 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) ==== 169+0+0 (secure 0 0 0) 0x7f041009b1c0 con 0x7f0418107400 2024-06-06T03:49:55.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.301+0000 7f041d161700 1 -- 172.21.15.40:0/498559513 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f04000a2d90 msgr2=0x7f04000a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:55.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.301+0000 7f041d161700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f04000a2d90 0x7f04000a5250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f04080099d0 tx=0x7f0408008040 comp rx=0 tx=0).stop 2024-06-06T03:49:55.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.302+0000 7f041d161700 1 -- 172.21.15.40:0/498559513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0418107400 msgr2=0x7f04181a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T03:49:55.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.302+0000 7f041d161700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0418107400 0x7f04181a4c50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f041000b970 tx=0x7f041000bc80 comp rx=0 tx=0).stop 2024-06-06T03:49:55.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.302+0000 7f041d161700 1 -- 172.21.15.40:0/498559513 shutdown_connections 2024-06-06T03:49:55.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.302+0000 7f041d161700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.40:6800/1568651190,v1:172.21.15.40:6801/1568651190] conn(0x7f04000a2d90 0x7f04000a5250 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:55.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.302+0000 7f041d161700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0418101390 0x7f0418199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:55.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.302+0000 7f041d161700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0418107400 0x7f04181a4c50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:55.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.302+0000 7f041d161700 1 --2- 172.21.15.40:0/498559513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0418101d50 0x7f041819a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T03:49:55.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.302+0000 7f041d161700 1 -- 172.21.15.40:0/498559513 >> 172.21.15.40:0/498559513 conn(0x7f0418078340 msgr2=0x7f04180fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T03:49:55.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.302+0000 7f041d161700 1 -- 172.21.15.40:0/498559513 shutdown_connections 2024-06-06T03:49:55.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-06-06T03:49:55.303+0000 7f041d161700 1 -- 172.21.15.40:0/498559513 wait complete. 2024-06-06T03:49:55.306 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-06-06T03:49:55.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:55 smithi040 ceph-mon[32220]: pgmap v353: 65 pgs: 65 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:49:55.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:55 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/498559513' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T03:49:55.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:55 smithi104 ceph-mon[36569]: pgmap v353: 65 pgs: 65 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:49:55.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:55 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/498559513' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T03:49:55.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:55 smithi131 ceph-mon[36687]: pgmap v353: 65 pgs: 65 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T03:49:55.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:55 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/498559513' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T03:49:55.921 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T03:49:55.948 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T03:49:55.981 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T03:49:56.009 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-06T03:49:56.009 DEBUG:teuthology.orchestra.run.smithi040:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T03:49:56.025 DEBUG:teuthology.orchestra.run.smithi104:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T03:49:56.043 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T03:49:56.064 INFO:tasks.cephadm:Stopping all daemons... 2024-06-06T03:49:56.064 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-06T03:49:56.064 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mon.a 2024-06-06T03:49:56.477 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:56 smithi040 systemd[1]: Stopping Ceph mon.a for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:49:56.477 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:56 smithi040 ceph-mon[32220]: from='client.? 172.21.15.40:0/498559513' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T03:49:56.477 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:56 smithi040 ceph-mon[32220]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T03:49:56.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:56 smithi104 ceph-mon[36569]: from='client.? 172.21.15.40:0/498559513' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T03:49:56.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:56 smithi104 ceph-mon[36569]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T03:49:56.731 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:56 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a[32216]: 2024-06-06T03:49:56.475+0000 7f2ca1175700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:49:56.731 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:56 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a[32216]: 2024-06-06T03:49:56.475+0000 7f2ca1175700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-06T03:49:56.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:56 smithi131 ceph-mon[36687]: from='client.? 172.21.15.40:0/498559513' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T03:49:56.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:56 smithi131 ceph-mon[36687]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T03:49:57.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:56 smithi040 podman[115755]: 2024-06-06 03:49:56.731544284 +0000 UTC m=+0.414960773 container died f7dcb07db357d152afe758de41ab9ed6bcd6a6e4896708ab56508ef10cefe9eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-06T03:49:57.427 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:57 smithi040 podman[115755]: 2024-06-06 03:49:57.201502139 +0000 UTC m=+0.884918616 container cleanup f7dcb07db357d152afe758de41ab9ed6bcd6a6e4896708ab56508ef10cefe9eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:49:57.427 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:57 smithi040 bash[115755]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a 2024-06-06T03:49:57.685 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jun 06 03:49:57 smithi040 podman[115765]: 2024-06-06 03:49:57.427962001 +0000 UTC m=+0.695738925 container remove f7dcb07db357d152afe758de41ab9ed6bcd6a6e4896708ab56508ef10cefe9eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-a, ceph=True, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-06T03:49:57.690 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-06T03:49:57.740 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:49:57.740 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-06T03:49:57.741 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-06T03:49:57.741 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mon.b 2024-06-06T03:49:58.091 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:57 smithi104 systemd[1]: Stopping Ceph mon.b for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:49:58.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:58 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-b[36565]: 2024-06-06T03:49:58.103+0000 7f04352d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:49:58.507 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:58 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-b[36565]: 2024-06-06T03:49:58.103+0000 7f04352d2700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-06T03:49:58.507 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:58 smithi104 podman[72983]: 2024-06-06 03:49:58.145299216 +0000 UTC m=+0.146550762 container died 7589498360b12698598a574efbd470a9eb193fac0346367d30d75d51cb9b6e94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-06T03:49:58.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:58 smithi104 podman[72983]: 2024-06-06 03:49:58.597023642 +0000 UTC m=+0.598275176 container cleanup 7589498360b12698598a574efbd470a9eb193fac0346367d30d75d51cb9b6e94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-06T03:49:58.841 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jun 06 03:49:58 smithi104 bash[72983]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-b 2024-06-06T03:49:59.058 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-06T03:49:59.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:49:59.103 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-06T03:49:59.103 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-06T03:49:59.104 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mon.c 2024-06-06T03:49:59.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:59 smithi131 systemd[1]: Stopping Ceph mon.c for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:49:59.986 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:59 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-c[36670]: 2024-06-06T03:49:59.601+0000 7f67971b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:49:59.987 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:59 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-c[36670]: 2024-06-06T03:49:59.601+0000 7f67971b7700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-06T03:49:59.987 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:49:59 smithi131 podman[74920]: 2024-06-06 03:49:59.684379587 +0000 UTC m=+0.386140404 container died 39bf71dc49191517ecfd7711c7ff7df3a114b8c4f99584ffbff615bbf1819023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:50:00.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:50:00 smithi131 podman[74920]: 2024-06-06 03:50:00.078157244 +0000 UTC m=+0.779918052 container cleanup 39bf71dc49191517ecfd7711c7ff7df3a114b8c4f99584ffbff615bbf1819023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-c, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-06T03:50:00.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Jun 06 03:50:00 smithi131 bash[74920]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mon-c 2024-06-06T03:50:00.515 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-06T03:50:00.556 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:50:00.557 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-06T03:50:00.557 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-06T03:50:00.557 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.x 2024-06-06T03:50:00.831 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:50:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:50:00.449+0000 7f67db676700 -1 client.15234: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-06-06T03:50:00.831 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:50:00 smithi040 systemd[1]: Stopping Ceph mgr.x for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:50:01.295 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:50:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:50:00.938+0000 7f6875016700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:50:01.295 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:50:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x[32450]: 2024-06-06T03:50:00.938+0000 7f6875016700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-06-06T03:50:01.295 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:50:00 smithi040 podman[115878]: 2024-06-06 03:50:00.970921103 +0000 UTC m=+0.218490876 container died 941bccf24d8aa9c112faa180156dc093987316da6f59d3f88d9814def7570238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T03:50:01.580 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:50:01 smithi040 podman[115878]: 2024-06-06 03:50:01.407405377 +0000 UTC m=+0.654975139 container cleanup 941bccf24d8aa9c112faa180156dc093987316da6f59d3f88d9814def7570238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-06T03:50:01.581 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:50:01 smithi040 bash[115878]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x 2024-06-06T03:50:01.884 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jun 06 03:50:01 smithi040 podman[115890]: 2024-06-06 03:50:01.634264254 +0000 UTC m=+0.658179065 container remove 941bccf24d8aa9c112faa180156dc093987316da6f59d3f88d9814def7570238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-06T03:50:01.887 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-06T03:50:01.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:50:01.925 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-06T03:50:01.925 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-06T03:50:01.925 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.y 2024-06-06T03:50:02.272 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:50:01 smithi104 systemd[1]: Stopping Ceph mgr.y for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:50:02.577 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:50:02 smithi104 podman[73106]: 2024-06-06 03:50:02.272430707 +0000 UTC m=+0.150821806 container died 87df9e888403d5bb5c9cc4ea49d2b726571f1e2085a9822e3c326adca4a1f12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-06T03:50:02.841 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:50:02 smithi104 podman[73106]: 2024-06-06 03:50:02.693990965 +0000 UTC m=+0.572382105 container cleanup 87df9e888403d5bb5c9cc4ea49d2b726571f1e2085a9822e3c326adca4a1f12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-06T03:50:02.841 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:50:02 smithi104 bash[73106]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y 2024-06-06T03:50:03.179 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-06T03:50:03.204 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:50:02 smithi104 podman[73118]: 2024-06-06 03:50:02.954153361 +0000 UTC m=+0.672892698 container remove 87df9e888403d5bb5c9cc4ea49d2b726571f1e2085a9822e3c326adca4a1f12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-y, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-06T03:50:03.205 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:50:02 smithi104 systemd[1]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-06-06T03:50:03.205 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:50:03 smithi104 systemd[1]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-06-06T03:50:03.205 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:50:03 smithi104 systemd[1]: Stopped Ceph mgr.y for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:50:03.205 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jun 06 03:50:03 smithi104 systemd[1]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.y.service: Consumed 20.364s CPU time. 2024-06-06T03:50:03.216 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:50:03.216 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-06T03:50:03.216 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-06T03:50:03.216 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.z 2024-06-06T03:50:03.532 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:50:03 smithi131 systemd[1]: Stopping Ceph mgr.z for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:50:03.850 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:50:03 smithi131 podman[75043]: 2024-06-06 03:50:03.554398638 +0000 UTC m=+0.140780252 container died c1118b82cadb00dc66aba6f1235b9da042e00d3dfb47a854adc1a79830e3bc69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS) 2024-06-06T03:50:04.185 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:50:03 smithi131 podman[75043]: 2024-06-06 03:50:03.94266908 +0000 UTC m=+0.529050639 container cleanup c1118b82cadb00dc66aba6f1235b9da042e00d3dfb47a854adc1a79830e3bc69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-06-06T03:50:04.186 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Jun 06 03:50:03 smithi131 bash[75043]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mgr-z 2024-06-06T03:50:04.329 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mgr.z.service' 2024-06-06T03:50:04.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:50:04.362 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-06T03:50:04.362 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-06T03:50:04.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.0 2024-06-06T03:50:04.580 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:50:04 smithi040 systemd[1]: Stopping Ceph osd.0 for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:50:05.080 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:50:04 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-0[46886]: 2024-06-06T03:50:04.679+0000 7fbc7c658700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:50:05.081 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:50:04 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-0[46886]: 2024-06-06T03:50:04.679+0000 7fbc7c658700 -1 osd.0 104 *** Got signal Terminated *** 2024-06-06T03:50:05.081 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:50:04 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-0[46886]: 2024-06-06T03:50:04.679+0000 7fbc7c658700 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T03:50:10.514 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:50:10 smithi040 podman[116008]: 2024-06-06 03:50:10.224500918 +0000 UTC m=+5.664118260 container died 55d4eb08ab050477df3b12e02ca3d3dbd8227dd4df891e1177dff60046a5a6c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-06T03:50:10.831 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:50:10 smithi040 podman[116008]: 2024-06-06 03:50:10.622406231 +0000 UTC m=+6.062023548 container cleanup 55d4eb08ab050477df3b12e02ca3d3dbd8227dd4df891e1177dff60046a5a6c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-06T03:50:10.831 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:50:10 smithi040 bash[116008]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-0 2024-06-06T03:50:11.235 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:50:10 smithi040 podman[116027]: 2024-06-06 03:50:10.907553758 +0000 UTC m=+0.676936662 container remove 55d4eb08ab050477df3b12e02ca3d3dbd8227dd4df891e1177dff60046a5a6c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-06T03:50:11.502 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:50:11 smithi040 podman[116115]: 2024-06-06 03:50:11.137528107 +0000 UTC m=+0.019064857 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:50:11.502 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:50:11 smithi040 podman[116115]: 2024-06-06 03:50:11.258448453 +0000 UTC m=+0.139985184 container create 3888fed82ca68bd27a6679e6f06ff86a814e69431673a436415b1219adaefd8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-06T03:50:11.810 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:50:11 smithi040 podman[116115]: 2024-06-06 03:50:11.502977494 +0000 UTC m=+0.384514233 container init 3888fed82ca68bd27a6679e6f06ff86a814e69431673a436415b1219adaefd8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T03:50:11.811 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:50:11 smithi040 podman[116115]: 2024-06-06 03:50:11.506958138 +0000 UTC m=+0.388494874 container start 3888fed82ca68bd27a6679e6f06ff86a814e69431673a436415b1219adaefd8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-06T03:50:11.811 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:50:11 smithi040 podman[116115]: 2024-06-06 03:50:11.628099042 +0000 UTC m=+0.509635785 container attach 3888fed82ca68bd27a6679e6f06ff86a814e69431673a436415b1219adaefd8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-06T03:50:12.080 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jun 06 03:50:11 smithi040 podman[116115]: 2024-06-06 03:50:11.81180632 +0000 UTC m=+0.693343065 container died 3888fed82ca68bd27a6679e6f06ff86a814e69431673a436415b1219adaefd8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-06T03:50:12.709 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-06T03:50:12.753 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:50:12.753 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-06T03:50:12.754 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-06T03:50:12.754 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.1 2024-06-06T03:50:13.073 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:50:12 smithi104 systemd[1]: Stopping Ceph osd.1 for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:50:13.341 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:50:13 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-1[41142]: 2024-06-06T03:50:13.072+0000 7ff53e653700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:50:13.341 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:50:13 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-1[41142]: 2024-06-06T03:50:13.072+0000 7ff53e653700 -1 osd.1 104 *** Got signal Terminated *** 2024-06-06T03:50:13.341 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:50:13 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-1[41142]: 2024-06-06T03:50:13.072+0000 7ff53e653700 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T03:50:18.749 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:50:18 smithi104 podman[73245]: 2024-06-06 03:50:18.436567404 +0000 UTC m=+5.499889844 container died 18abb4ec4423c4eed9ba7dd5619a3bb03e64b4f4d67f5d751ee831e17edb1d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-1, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0) 2024-06-06T03:50:19.091 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:50:18 smithi104 podman[73245]: 2024-06-06 03:50:18.847600352 +0000 UTC m=+5.910922791 container cleanup 18abb4ec4423c4eed9ba7dd5619a3bb03e64b4f4d67f5d751ee831e17edb1d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-1, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-06T03:50:19.091 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:50:18 smithi104 bash[73245]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-1 2024-06-06T03:50:19.413 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:50:19 smithi104 podman[73261]: 2024-06-06 03:50:19.099150092 +0000 UTC m=+0.658840820 container remove 18abb4ec4423c4eed9ba7dd5619a3bb03e64b4f4d67f5d751ee831e17edb1d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T03:50:19.702 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:50:19 smithi104 podman[73353]: 2024-06-06 03:50:19.316801346 +0000 UTC m=+0.016850626 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:50:19.702 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:50:19 smithi104 podman[73353]: 2024-06-06 03:50:19.458832868 +0000 UTC m=+0.158882138 container create 5c7c09d4371cb8840dd9dae7377b46ead0b5e5f2b5b2c064d09c21720cdd55bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-06T03:50:20.091 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:50:19 smithi104 podman[73353]: 2024-06-06 03:50:19.702920622 +0000 UTC m=+0.402969892 container init 5c7c09d4371cb8840dd9dae7377b46ead0b5e5f2b5b2c064d09c21720cdd55bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-1-deactivate, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-06-06T03:50:20.091 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:50:19 smithi104 podman[73353]: 2024-06-06 03:50:19.706100297 +0000 UTC m=+0.406149568 container start 5c7c09d4371cb8840dd9dae7377b46ead0b5e5f2b5b2c064d09c21720cdd55bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, ceph=True) 2024-06-06T03:50:20.092 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:50:19 smithi104 podman[73353]: 2024-06-06 03:50:19.861372348 +0000 UTC m=+0.561421621 container attach 5c7c09d4371cb8840dd9dae7377b46ead0b5e5f2b5b2c064d09c21720cdd55bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-1-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-06T03:50:20.591 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jun 06 03:50:20 smithi104 podman[73353]: 2024-06-06 03:50:20.232922234 +0000 UTC m=+0.932971514 container died 5c7c09d4371cb8840dd9dae7377b46ead0b5e5f2b5b2c064d09c21720cdd55bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-06T03:50:20.913 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-06T03:50:20.950 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:50:20.951 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-06T03:50:20.951 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-06T03:50:20.951 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.2 2024-06-06T03:50:21.255 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:50:21 smithi131 systemd[1]: Stopping Ceph osd.2 for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:50:21.532 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:50:21 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-2[41779]: 2024-06-06T03:50:21.255+0000 7fa293a55700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:50:21.533 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:50:21 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-2[41779]: 2024-06-06T03:50:21.255+0000 7fa293a55700 -1 osd.2 104 *** Got signal Terminated *** 2024-06-06T03:50:21.533 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:50:21 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-2[41779]: 2024-06-06T03:50:21.255+0000 7fa293a55700 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T03:50:26.915 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:50:26 smithi131 podman[75178]: 2024-06-06 03:50:26.612628276 +0000 UTC m=+5.459433747 container died a64c72ab59e7413d2426bc06ccc4188be61c5f4aea1cce88ddfc1562677182e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-2, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:50:27.232 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:50:27 smithi131 podman[75178]: 2024-06-06 03:50:27.00639172 +0000 UTC m=+5.853197188 container cleanup a64c72ab59e7413d2426bc06ccc4188be61c5f4aea1cce88ddfc1562677182e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-3d89acc) 2024-06-06T03:50:27.232 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:50:27 smithi131 bash[75178]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-2 2024-06-06T03:50:27.533 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:50:27 smithi131 podman[75196]: 2024-06-06 03:50:27.232767433 +0000 UTC m=+0.614136799 container remove a64c72ab59e7413d2426bc06ccc4188be61c5f4aea1cce88ddfc1562677182e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-06T03:50:27.861 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:50:27 smithi131 podman[75287]: 2024-06-06 03:50:27.481146692 +0000 UTC m=+0.017954154 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:50:27.861 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:50:27 smithi131 podman[75287]: 2024-06-06 03:50:27.60059043 +0000 UTC m=+0.137397876 container create c27744e144ba125432cd546c2c1c1542624596d0fb64b8fb32a67a3207cb20d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-06T03:50:27.861 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:50:27 smithi131 podman[75287]: 2024-06-06 03:50:27.861615815 +0000 UTC m=+0.398423261 container init c27744e144ba125432cd546c2c1c1542624596d0fb64b8fb32a67a3207cb20d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:50:28.154 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:50:27 smithi131 podman[75287]: 2024-06-06 03:50:27.864954275 +0000 UTC m=+0.401761727 container start c27744e144ba125432cd546c2c1c1542624596d0fb64b8fb32a67a3207cb20d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:50:28.154 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:50:27 smithi131 podman[75287]: 2024-06-06 03:50:27.961742644 +0000 UTC m=+0.498550107 container attach c27744e144ba125432cd546c2c1c1542624596d0fb64b8fb32a67a3207cb20d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-06T03:50:28.422 INFO:journalctl@ceph.osd.2.smithi131.stdout:Jun 06 03:50:28 smithi131 podman[75287]: 2024-06-06 03:50:28.154591201 +0000 UTC m=+0.691398658 container died c27744e144ba125432cd546c2c1c1542624596d0fb64b8fb32a67a3207cb20d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-06T03:50:28.771 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-06T03:50:28.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:50:28.816 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-06T03:50:28.816 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-06T03:50:28.817 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.3 2024-06-06T03:50:29.120 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:50:28 smithi040 systemd[1]: Stopping Ceph osd.3 for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:50:29.581 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:50:29 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-3[53559]: 2024-06-06T03:50:29.118+0000 7fd96a980700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:50:29.581 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:50:29 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-3[53559]: 2024-06-06T03:50:29.118+0000 7fd96a980700 -1 osd.3 104 *** Got signal Terminated *** 2024-06-06T03:50:29.581 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:50:29 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-3[53559]: 2024-06-06T03:50:29.118+0000 7fd96a980700 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T03:50:30.591 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:30 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:30.501+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:31.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:31 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:31.507+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:32.532 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:32 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:32.275+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:32.579 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:32 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:32.174+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:32.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:32 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:32.492+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:33.532 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:33 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:33.255+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:33.581 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:33 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:33.222+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:33.581 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:33 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:33.166+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:33.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:33 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:33.479+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:34.282 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:34 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:34.240+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:34.283 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:34 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:34.028+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:34.521 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:34 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:34.151+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:34.521 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:34 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:34.267+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:34.830 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:50:34 smithi040 podman[116223]: 2024-06-06 03:50:34.521593687 +0000 UTC m=+5.498381519 container died 26474bff4d32bc729f225698c5780a28a2394e9f5bcf4df7e60554a2f6198544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-3, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-06T03:50:34.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:34 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:34.450+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:35.177 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:50:35 smithi040 podman[116223]: 2024-06-06 03:50:35.119726492 +0000 UTC m=+6.096514384 container cleanup 26474bff4d32bc729f225698c5780a28a2394e9f5bcf4df7e60554a2f6198544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-3, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-06T03:50:35.177 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:50:35 smithi040 bash[116223]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-3 2024-06-06T03:50:35.532 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:35.236+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:35.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:35.056+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:35.580 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:50:35 smithi040 podman[116249]: 2024-06-06 03:50:35.387880139 +0000 UTC m=+0.860271932 container remove 26474bff4d32bc729f225698c5780a28a2394e9f5bcf4df7e60554a2f6198544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-3, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-06T03:50:35.581 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:35 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:35.176+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:35.581 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:35 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:35.266+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:35.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:35 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:35.404+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:36.024 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:50:35 smithi040 podman[116339]: 2024-06-06 03:50:35.622953332 +0000 UTC m=+0.017493542 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:50:36.024 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:50:35 smithi040 podman[116339]: 2024-06-06 03:50:35.747261875 +0000 UTC m=+0.141802079 container create 5fb47012d73f2dd70d208c5a5294f1d9c1a432b72647260d3c1761942630380a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-06-06T03:50:36.024 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:50:35 smithi040 podman[116339]: 2024-06-06 03:50:35.957898783 +0000 UTC m=+0.352438993 container init 5fb47012d73f2dd70d208c5a5294f1d9c1a432b72647260d3c1761942630380a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-06T03:50:36.024 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:50:35 smithi040 podman[116339]: 2024-06-06 03:50:35.961343345 +0000 UTC m=+0.355883560 container start 5fb47012d73f2dd70d208c5a5294f1d9c1a432b72647260d3c1761942630380a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-06T03:50:36.025 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:50:36 smithi040 podman[116339]: 2024-06-06 03:50:36.024598511 +0000 UTC m=+0.419138721 container attach 5fb47012d73f2dd70d208c5a5294f1d9c1a432b72647260d3c1761942630380a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-06T03:50:36.282 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:36 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:36.154+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:36.282 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:36 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:36.235+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:36.283 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:36 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:36.023+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:36.581 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:36 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:36.295+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:36.582 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jun 06 03:50:36 smithi040 podman[116339]: 2024-06-06 03:50:36.28287681 +0000 UTC m=+0.677417033 container died 5fb47012d73f2dd70d208c5a5294f1d9c1a432b72647260d3c1761942630380a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-06T03:50:36.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:36 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:36.363+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:37.023 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-06T03:50:37.068 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:50:37.068 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-06T03:50:37.069 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-06T03:50:37.069 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.4 2024-06-06T03:50:37.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:37.199+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:37.331 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:37 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:37.252+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:37.389 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:37 smithi104 systemd[1]: Stopping Ceph osd.4 for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:50:37.532 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:37 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:37.204+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:37.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:37 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:37.061+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:37.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:37 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:37.388+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:37.841 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:37 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4[48033]: 2024-06-06T03:50:37.612+0000 7f9fdadb5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:50:37.841 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:37 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4[48033]: 2024-06-06T03:50:37.612+0000 7f9fdadb5700 -1 osd.4 104 *** Got signal Terminated *** 2024-06-06T03:50:37.842 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:37 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4[48033]: 2024-06-06T03:50:37.612+0000 7f9fdadb5700 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T03:50:38.532 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:38 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:38.187+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:38.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:38 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:38.109+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:38.581 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:38 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:38.198+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:38.581 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:38 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:38.299+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:38.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:38 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:38.377+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:39.199 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:39 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:39.099+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:39.532 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:39 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:39.199+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:39.581 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:39 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:39.207+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:39.581 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:39 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:39.347+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:39.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:39 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:39.384+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:39.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:39 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:39.384+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:40.173 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:39 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:39.907+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:40.173 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:40 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:40.118+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:40.174 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:40 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:40.118+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:40.532 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:40 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:40.173+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:40.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:40 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:40.173+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.958412+0000 front 2024-06-06T03:50:17.958520+0000 (oldest deadline 2024-06-06T03:50:39.658479+0000) 2024-06-06T03:50:40.580 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:40 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:40.188+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:40.581 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:40 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:40.344+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:40.841 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:40 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4[48033]: 2024-06-06T03:50:40.370+0000 7f9fd73a9700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.565731+0000 front 2024-06-06T03:50:15.565758+0000 (oldest deadline 2024-06-06T03:50:39.665932+0000) 2024-06-06T03:50:40.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:40 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:40.441+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:40.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:40 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:40.344+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:40.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:40 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:40.344+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:41.156 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:40 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:40.860+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:41.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:41 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:41.212+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:41.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:41 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:41.212+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.958412+0000 front 2024-06-06T03:50:17.958520+0000 (oldest deadline 2024-06-06T03:50:39.658479+0000) 2024-06-06T03:50:41.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:41 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:41.157+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:41.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:41 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:41.157+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:41.581 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:41 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:41.204+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:41.581 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:41 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:41.392+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:41.581 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:41 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:41.392+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:41.742 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:41 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:41.459+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:41.743 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:41 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:41.374+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:41.743 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:41 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:41.374+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:41.743 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:41 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4[48033]: 2024-06-06T03:50:41.411+0000 7f9fd73a9700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.565731+0000 front 2024-06-06T03:50:15.565758+0000 (oldest deadline 2024-06-06T03:50:39.665932+0000) 2024-06-06T03:50:42.138 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:41 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:41.812+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:42.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:42 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:42.138+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:42.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:42 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:42.138+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:42.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:42 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:42.199+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:42.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:42 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:42.199+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.958412+0000 front 2024-06-06T03:50:17.958520+0000 (oldest deadline 2024-06-06T03:50:39.658479+0000) 2024-06-06T03:50:42.581 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:42 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:42.218+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:42.581 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:42 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:42.218+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:42.581 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:42 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:42.424+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:42.581 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:42 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:42.424+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:42.758 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:42 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:42.466+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:42.759 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:42 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4[48033]: 2024-06-06T03:50:42.394+0000 7f9fd73a9700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.565731+0000 front 2024-06-06T03:50:15.565758+0000 (oldest deadline 2024-06-06T03:50:39.665932+0000) 2024-06-06T03:50:42.759 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:42 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:42.422+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:42.759 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:42 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:42.422+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:43.026 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:42 smithi104 podman[73459]: 2024-06-06 03:50:42.997018718 +0000 UTC m=+5.731625787 container died 0ccd6676136550c466fc18dae8ca3c1232371c35013ad638073f0c1a59a6ee40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc) 2024-06-06T03:50:43.148 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:42 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:42.800+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:43.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:43 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:43.211+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:43.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:43 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:43.211+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:43.532 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:43 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:43.170+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:43.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:43 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:43.170+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.958412+0000 front 2024-06-06T03:50:17.958520+0000 (oldest deadline 2024-06-06T03:50:39.658479+0000) 2024-06-06T03:50:43.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:43 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:43.147+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:43.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:43 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:43.147+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:43.591 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:43 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:43.419+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:43.592 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:43 smithi104 podman[73459]: 2024-06-06 03:50:43.408046424 +0000 UTC m=+6.142653497 container cleanup 0ccd6676136550c466fc18dae8ca3c1232371c35013ad638073f0c1a59a6ee40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-06T03:50:43.592 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:43 smithi104 bash[73459]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4 2024-06-06T03:50:43.592 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:43 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:43.372+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:43.592 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:43 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:43.372+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:43.657 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:43 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:43.396+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:43.657 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:43 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:43.396+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:43.978 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:43 smithi104 podman[73475]: 2024-06-06 03:50:43.650919797 +0000 UTC m=+0.652266971 container remove 0ccd6676136550c466fc18dae8ca3c1232371c35013ad638073f0c1a59a6ee40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-06T03:50:44.133 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:43 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:43.839+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:44.280 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:43 smithi104 podman[73573]: 2024-06-06 03:50:43.881254043 +0000 UTC m=+0.017000101 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:50:44.281 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:44 smithi104 podman[73573]: 2024-06-06 03:50:44.018518762 +0000 UTC m=+0.154264804 container create 6f484f6bb47d4872088b4211cbcd485f4b588a04a37848218a84eb08e4d00c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T03:50:44.281 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:44 smithi104 podman[73573]: 2024-06-06 03:50:44.280730767 +0000 UTC m=+0.416476818 container init 6f484f6bb47d4872088b4211cbcd485f4b588a04a37848218a84eb08e4d00c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-06T03:50:44.532 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:44 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:44.132+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:44.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:44 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:44.132+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.958412+0000 front 2024-06-06T03:50:17.958520+0000 (oldest deadline 2024-06-06T03:50:39.658479+0000) 2024-06-06T03:50:44.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:44 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:44.160+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:44.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:44 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:44.160+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:44.574 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:44 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:44.397+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:44.574 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:44 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:44.413+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:44.574 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:44 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:44.413+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:44.574 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:44 smithi104 podman[73573]: 2024-06-06 03:50:44.28410172 +0000 UTC m=+0.419847770 container start 6f484f6bb47d4872088b4211cbcd485f4b588a04a37848218a84eb08e4d00c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-06T03:50:44.575 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:44 smithi104 podman[73573]: 2024-06-06 03:50:44.404934568 +0000 UTC m=+0.540680619 container attach 6f484f6bb47d4872088b4211cbcd485f4b588a04a37848218a84eb08e4d00c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T03:50:44.580 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:44 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:44.185+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:44.580 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:44 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:44.185+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:44.581 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:44 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:44.422+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:44.581 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:44 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:44.422+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:44.841 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jun 06 03:50:44 smithi104 podman[73573]: 2024-06-06 03:50:44.574753698 +0000 UTC m=+0.710499753 container died 6f484f6bb47d4872088b4211cbcd485f4b588a04a37848218a84eb08e4d00c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True) 2024-06-06T03:50:45.150 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:44 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:44.820+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:45.283 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-06T03:50:45.322 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:50:45.323 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-06T03:50:45.323 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-06T03:50:45.323 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.5 2024-06-06T03:50:45.426 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:45 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:45.157+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:45.426 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:45 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:45.157+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:45.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:45.150+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:45.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:45.150+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:45.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:45.155+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:45.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:45.155+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.958412+0000 front 2024-06-06T03:50:17.958520+0000 (oldest deadline 2024-06-06T03:50:39.658479+0000) 2024-06-06T03:50:45.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:45 smithi131 systemd[1]: Stopping Ceph osd.5 for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:50:45.591 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:45.435+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:45.591 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:45.453+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:45.591 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:45 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:45.453+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:45.830 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:45 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:45.426+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:45.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:45 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:45.426+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:46.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:45.837+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:46.033 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:45.643+0000 7f3a2ee41700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:50:46.033 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:45.643+0000 7f3a2ee41700 -1 osd.5 104 *** Got signal Terminated *** 2024-06-06T03:50:46.033 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:45 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:45.643+0000 7f3a2ee41700 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T03:50:46.464 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:46 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:46.169+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:46.465 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:46 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:46.169+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:46.532 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:46 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:46.109+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:46.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:46 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:46.109+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.958412+0000 front 2024-06-06T03:50:17.958520+0000 (oldest deadline 2024-06-06T03:50:39.658479+0000) 2024-06-06T03:50:46.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:46 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:46.174+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:46.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:46 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:46.174+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:46.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:46 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:46.464+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:46.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:46 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:46.464+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:46.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:46 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:46.471+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:46.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:46 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:46.407+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:46.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:46 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:46.407+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:47.097 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:46 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:46.807+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:47.437 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:47 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:47.166+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:47.438 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:47 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:47.166+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:47.532 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:47 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:47.096+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:47.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:47 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:47.096+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.958412+0000 front 2024-06-06T03:50:17.958520+0000 (oldest deadline 2024-06-06T03:50:39.658479+0000) 2024-06-06T03:50:47.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:47 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:47.217+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:47.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:47 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:47.217+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:47.830 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:47 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:47.435+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:47.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:47 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:47.435+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:47.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:47 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:47.453+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:47.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:47 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:47.382+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:47.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:47 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:47.382+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:47.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:47 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:47.382+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:50:48.096 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:47 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:47.798+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:48.096 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:47 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:47.798+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:50:48.420 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:48 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:48.147+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:48.420 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:48 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:48.147+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:48.420 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:48 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:48.147+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:22.157697+0000 front 2024-06-06T03:50:22.157857+0000 (oldest deadline 2024-06-06T03:50:47.457856+0000) 2024-06-06T03:50:48.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:48 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:48.095+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:48.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:48 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:48.095+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.958412+0000 front 2024-06-06T03:50:17.958520+0000 (oldest deadline 2024-06-06T03:50:39.658479+0000) 2024-06-06T03:50:48.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:48 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:48.095+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.960735+0000 front 2024-06-06T03:50:25.960717+0000 (oldest deadline 2024-06-06T03:50:47.660759+0000) 2024-06-06T03:50:48.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:48 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:48.253+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:48.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:48 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:48.253+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:48.830 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:48 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:48.419+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:48.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:48 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:48.419+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:48.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:48 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:48.419+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:50:48.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:48 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:48.437+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:48.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:48 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:48.388+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:48.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:48 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:48.388+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:48.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:48 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:48.388+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:50:49.064 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:48 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:48.789+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:49.065 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:48 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:48.789+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:50:49.428 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:49 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:49.132+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:49.429 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:49 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:49.132+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:49.429 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:49 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:49.132+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:22.157697+0000 front 2024-06-06T03:50:22.157857+0000 (oldest deadline 2024-06-06T03:50:47.457856+0000) 2024-06-06T03:50:49.532 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:49 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:49.063+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:49.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:49 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:49.063+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.958412+0000 front 2024-06-06T03:50:17.958520+0000 (oldest deadline 2024-06-06T03:50:39.658479+0000) 2024-06-06T03:50:49.533 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:49 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:49.063+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.960735+0000 front 2024-06-06T03:50:25.960717+0000 (oldest deadline 2024-06-06T03:50:47.660759+0000) 2024-06-06T03:50:49.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:49 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:49.267+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:49.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:49 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:49.267+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:49.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:49 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:49.428+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:49.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:49 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:49.428+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:49.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:49 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:49.428+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:50:49.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:49 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:49.429+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:49.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:49 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:49.411+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:49.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:49 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:49.411+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:49.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:49 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:49.411+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:50:50.027 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:49 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:49.742+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:50.027 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:49 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:49.742+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:50:50.282 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:50 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:50.026+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:07.456844+0000 front 2024-06-06T03:50:07.456612+0000 (oldest deadline 2024-06-06T03:50:31.556578+0000) 2024-06-06T03:50:50.283 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:50 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:50.026+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.958412+0000 front 2024-06-06T03:50:17.958520+0000 (oldest deadline 2024-06-06T03:50:39.658479+0000) 2024-06-06T03:50:50.283 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:50 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5[48740]: 2024-06-06T03:50:50.026+0000 7f3a2742d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.960735+0000 front 2024-06-06T03:50:25.960717+0000 (oldest deadline 2024-06-06T03:50:47.660759+0000) 2024-06-06T03:50:50.478 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:50 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:50.104+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:50.478 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:50 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:50.104+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:50.478 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:50 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:50.104+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:22.157697+0000 front 2024-06-06T03:50:22.157857+0000 (oldest deadline 2024-06-06T03:50:47.457856+0000) 2024-06-06T03:50:50.772 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:50 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:50.299+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:50.772 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:50 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:50.299+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:50.772 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:50 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:50.299+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:50:50.830 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:50 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:50.477+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:50.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:50 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:50.477+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:50.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:50 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:50.477+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:50:50.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:50 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:50.439+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:50.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:50 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:50.439+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:50:50.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:50 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:50.444+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:50.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:50 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:50.444+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:50.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:50 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:50.444+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:50:51.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:50 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:50.770+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:51.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:50 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:50.770+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:50:51.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:51 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:51.063+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:51.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:51 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:51.063+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:51.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:51 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:51.063+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:22.157697+0000 front 2024-06-06T03:50:22.157857+0000 (oldest deadline 2024-06-06T03:50:47.457856+0000) 2024-06-06T03:50:51.403 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:51 smithi131 podman[75404]: 2024-06-06 03:50:51.117688202 +0000 UTC m=+5.600421022 container died 68048391c66cf6fc81ed9e1b619335b9031056e10a33973f82e1dfdc68bfada9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-06T03:50:51.404 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:51 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:51.294+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:51.404 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:51 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:51.294+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:51.404 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:51 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:51.294+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:50:51.728 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:51 smithi131 podman[75404]: 2024-06-06 03:50:51.486887895 +0000 UTC m=+5.969620715 container cleanup 68048391c66cf6fc81ed9e1b619335b9031056e10a33973f82e1dfdc68bfada9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-06T03:50:51.728 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:51 smithi131 bash[75404]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5 2024-06-06T03:50:51.830 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:51 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:51.503+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:51.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:51 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:51.503+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:51.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:51 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:51.503+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:50:51.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:51 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:51.475+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:51.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:51 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:51.475+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:50:51.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:51 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:51.415+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:51.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:51 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:51.415+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:51.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:51 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:51.415+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:50:52.032 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:51 smithi131 podman[75423]: 2024-06-06 03:50:51.771888363 +0000 UTC m=+0.656358881 container remove 68048391c66cf6fc81ed9e1b619335b9031056e10a33973f82e1dfdc68bfada9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-06T03:50:52.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:51 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:51.727+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:52.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:51 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:51.727+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:50:52.330 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:52 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:52.060+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:52.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:52 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:52.060+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:52.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:52 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:52.060+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:22.157697+0000 front 2024-06-06T03:50:22.157857+0000 (oldest deadline 2024-06-06T03:50:47.457856+0000) 2024-06-06T03:50:52.367 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:52 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:52.276+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:52.367 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:52 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:52.276+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:52.367 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:52 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:52.276+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:50:52.367 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:52 smithi131 podman[75511]: 2024-06-06 03:50:52.018911193 +0000 UTC m=+0.017412784 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:50:52.368 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:52 smithi131 podman[75511]: 2024-06-06 03:50:52.13112185 +0000 UTC m=+0.129623425 container create e2601f5d8b45e19027df34922c82dcb58a733f4889b04449b9df5455946bb1ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-06T03:50:52.368 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:52 smithi131 podman[75511]: 2024-06-06 03:50:52.367670589 +0000 UTC m=+0.366172173 container init e2601f5d8b45e19027df34922c82dcb58a733f4889b04449b9df5455946bb1ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5-deactivate, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T03:50:52.656 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:52 smithi131 podman[75511]: 2024-06-06 03:50:52.370835445 +0000 UTC m=+0.369337020 container start e2601f5d8b45e19027df34922c82dcb58a733f4889b04449b9df5455946bb1ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:50:52.657 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:52 smithi131 podman[75511]: 2024-06-06 03:50:52.467146259 +0000 UTC m=+0.465647842 container attach e2601f5d8b45e19027df34922c82dcb58a733f4889b04449b9df5455946bb1ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T03:50:52.830 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:52 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:52.508+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:52.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:52 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:52.508+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:52.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:52 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:52.508+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:50:52.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:52 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:52.467+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:52.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:52 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:52.467+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:50:52.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:52 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:52.454+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:52.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:52 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:52.454+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:52.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:52 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:52.454+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:50:52.963 INFO:journalctl@ceph.osd.5.smithi131.stdout:Jun 06 03:50:52 smithi131 podman[75511]: 2024-06-06 03:50:52.657709627 +0000 UTC m=+0.656211210 container died e2601f5d8b45e19027df34922c82dcb58a733f4889b04449b9df5455946bb1ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-06-06T03:50:52.964 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:52 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:52.748+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:52.964 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:52 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:52.748+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:50:53.286 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-06T03:50:53.326 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:50:53.326 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-06T03:50:53.326 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-06T03:50:53.326 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.6 2024-06-06T03:50:53.376 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:53 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:53.106+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:53.376 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:53 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:53.106+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:53.377 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:53 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:53.106+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:22.157697+0000 front 2024-06-06T03:50:22.157857+0000 (oldest deadline 2024-06-06T03:50:47.457856+0000) 2024-06-06T03:50:53.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:53 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:53.230+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:53.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:53 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:53.230+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:53.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:53 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:53.230+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:50:53.631 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:53 smithi040 systemd[1]: Stopping Ceph osd.6 for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:50:53.632 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:53 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:53.508+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:53.632 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:53 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:53.508+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:53.632 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:53 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:53.508+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:50:53.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:53 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:53.423+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:53.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:53 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:53.423+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:50:53.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:53 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:53.426+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:53.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:53 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:53.426+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:53.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:53 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:53.426+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:50:54.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:53 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:53.777+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:54.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:53 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:53.777+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:50:54.081 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:53 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:53.631+0000 7fcd35b54700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:50:54.081 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:53 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:53.631+0000 7fcd35b54700 -1 osd.6 104 *** Got signal Terminated *** 2024-06-06T03:50:54.081 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:53 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:53.631+0000 7fcd35b54700 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T03:50:54.477 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:54 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:54.114+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:54.477 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:54 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:54.114+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:54.478 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:54 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:54.114+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:22.157697+0000 front 2024-06-06T03:50:22.157857+0000 (oldest deadline 2024-06-06T03:50:47.457856+0000) 2024-06-06T03:50:54.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:54 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:54.238+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:54.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:54 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:54.238+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:54.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:54 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:54.238+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:50:54.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:54 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:54.476+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:54.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:54 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:54.476+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:54.832 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:54 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:54.476+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:50:54.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:54 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:54.454+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:54.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:54 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:54.454+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:50:54.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:54 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:54.419+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:54.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:54 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:54.419+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:54.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:54 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:54.419+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:50:55.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:54 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:54.781+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:55.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:54 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:54.781+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:50:55.330 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:55 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:55.080+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:55.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:55 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:55.080+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:55.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:55 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:55.080+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:22.157697+0000 front 2024-06-06T03:50:22.157857+0000 (oldest deadline 2024-06-06T03:50:47.457856+0000) 2024-06-06T03:50:55.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:55 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:55.080+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.358770+0000 front 2024-06-06T03:50:30.358844+0000 (oldest deadline 2024-06-06T03:50:55.058744+0000) 2024-06-06T03:50:55.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:55 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:55.217+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:55.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:55 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:55.217+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:55.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:55 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:55.217+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:50:55.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:55 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:55.521+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:55.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:55 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:55.521+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:55.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:55 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:55.521+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:50:55.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:55 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:55.521+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:50:55.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:55 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:55.459+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:55.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:55 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:55.459+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:50:55.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:55 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:55.459+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.224518+0000 front 2024-06-06T03:50:30.224558+0000 (oldest deadline 2024-06-06T03:50:54.924666+0000) 2024-06-06T03:50:55.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:55 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:55.457+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:55.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:55 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:55.457+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:55.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:55 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:55.457+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:50:55.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:55 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:55.457+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:50:56.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:55 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:55.743+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:56.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:55 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:55.743+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:50:56.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:55 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:55.743+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:50:56.528 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:56 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:56.102+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:56.528 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:56 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:56.102+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:56.528 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:56 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:56.102+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:22.157697+0000 front 2024-06-06T03:50:22.157857+0000 (oldest deadline 2024-06-06T03:50:47.457856+0000) 2024-06-06T03:50:56.529 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:56 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:56.102+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.358770+0000 front 2024-06-06T03:50:30.358844+0000 (oldest deadline 2024-06-06T03:50:55.058744+0000) 2024-06-06T03:50:56.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:56 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:56.239+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:56.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:56 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:56.239+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:56.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:56 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:56.239+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:50:56.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:56 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:56.239+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:50:56.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:56 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:56.528+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:56.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:56 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:56.528+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:56.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:56 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:56.528+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:50:56.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:56 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:56.528+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:50:56.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:56 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:56.449+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:56.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:56 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:56.449+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:50:56.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:56 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:56.449+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.224518+0000 front 2024-06-06T03:50:30.224558+0000 (oldest deadline 2024-06-06T03:50:54.924666+0000) 2024-06-06T03:50:56.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:56 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:56.483+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:56.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:56 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:56.483+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:56.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:56 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:56.483+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:50:56.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:56 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:56.483+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:50:57.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:56 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:56.719+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:57.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:56 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:56.719+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:50:57.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:56 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:56.719+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:50:57.532 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:57 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:57.087+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:57.532 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:57 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:57.087+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:57.532 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:57 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:57.087+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:22.157697+0000 front 2024-06-06T03:50:22.157857+0000 (oldest deadline 2024-06-06T03:50:47.457856+0000) 2024-06-06T03:50:57.533 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:57 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:57.087+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.358770+0000 front 2024-06-06T03:50:30.358844+0000 (oldest deadline 2024-06-06T03:50:55.058744+0000) 2024-06-06T03:50:57.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:57 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:57.228+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:57.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:57 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:57.228+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:57.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:57 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:57.228+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:50:57.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:57 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:57.228+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:50:57.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:57 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:57.531+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:57.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:57 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:57.531+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:57.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:57 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:57.531+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:50:57.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:57 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:57.531+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:50:57.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:57 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:57.433+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:57.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:57 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:57.433+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:50:57.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:57 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:57.433+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.224518+0000 front 2024-06-06T03:50:30.224558+0000 (oldest deadline 2024-06-06T03:50:54.924666+0000) 2024-06-06T03:50:57.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:57 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:57.469+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:57.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:57 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:57.469+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:57.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:57 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:57.469+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:50:57.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:57 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:57.469+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:50:58.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:57 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:57.747+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:58.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:57 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:57.747+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:50:58.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:57 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:57.747+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:50:58.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:58 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:58.078+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.755150+0000 front 2024-06-06T03:50:08.755229+0000 (oldest deadline 2024-06-06T03:50:32.255280+0000) 2024-06-06T03:50:58.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:58 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:58.078+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.856909+0000 front 2024-06-06T03:50:16.856958+0000 (oldest deadline 2024-06-06T03:50:42.157088+0000) 2024-06-06T03:50:58.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:58 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:58.078+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:22.157697+0000 front 2024-06-06T03:50:22.157857+0000 (oldest deadline 2024-06-06T03:50:47.457856+0000) 2024-06-06T03:50:58.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:58 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6[60341]: 2024-06-06T03:50:58.078+0000 7fcd32148700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.358770+0000 front 2024-06-06T03:50:30.358844+0000 (oldest deadline 2024-06-06T03:50:55.058744+0000) 2024-06-06T03:50:58.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:58 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:58.197+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:58.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:58 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:58.197+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:58.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:58 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:58.197+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:50:58.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:58 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:58.197+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:50:58.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:58 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:58.557+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:58.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:58 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:58.557+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:58.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:58 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:58.557+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:50:58.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:58 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:58.557+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:50:58.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:58 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:58.393+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:58.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:58 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:58.393+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:50:58.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:58 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:58.393+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.224518+0000 front 2024-06-06T03:50:30.224558+0000 (oldest deadline 2024-06-06T03:50:54.924666+0000) 2024-06-06T03:50:58.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:58 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:58.441+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:58.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:58 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:58.441+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:58.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:58 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:58.441+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:50:58.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:58 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:58.441+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:50:59.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:58 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:58.710+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:50:59.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:58 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:58.710+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:50:59.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:58 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:58.710+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:50:59.331 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:58 smithi040 podman[116445]: 2024-06-06 03:50:58.989184635 +0000 UTC m=+5.457854556 container died 13a950e0cbf8786ba452e18cdeedb765d6504058f39162f85fce2a47cc641c5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-06T03:50:59.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:59 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:59.234+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:50:59.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:59 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:59.234+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:50:59.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:59 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:59.234+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:50:59.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:50:59 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:50:59.234+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:50:59.658 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:59 smithi040 podman[116445]: 2024-06-06 03:50:59.382081079 +0000 UTC m=+5.850751019 container cleanup 13a950e0cbf8786ba452e18cdeedb765d6504058f39162f85fce2a47cc641c5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-06-06T03:50:59.659 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:59 smithi040 bash[116445]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6 2024-06-06T03:50:59.659 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:59 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:59.540+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:50:59.659 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:59 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:59.540+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:50:59.659 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:59 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:59.540+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:50:59.659 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:50:59 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:50:59.540+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:50:59.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:59 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:59.364+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:50:59.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:59 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:59.364+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:50:59.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:50:59 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:50:59.364+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.224518+0000 front 2024-06-06T03:50:30.224558+0000 (oldest deadline 2024-06-06T03:50:54.924666+0000) 2024-06-06T03:50:59.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:59 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:59.445+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:50:59.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:59 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:59.445+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:50:59.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:59 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:59.445+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:50:59.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:50:59 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:50:59.445+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:50:59.980 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:59 smithi040 podman[116460]: 2024-06-06 03:50:59.658768573 +0000 UTC m=+0.661990447 container remove 13a950e0cbf8786ba452e18cdeedb765d6504058f39162f85fce2a47cc641c5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-06T03:51:00.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:59 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:59.700+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:00.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:59 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:59.700+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:00.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:50:59 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:50:59.700+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:00.278 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:50:59 smithi040 podman[116550]: 2024-06-06 03:50:59.883759653 +0000 UTC m=+0.017772038 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:51:00.278 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:51:00 smithi040 podman[116550]: 2024-06-06 03:51:00.017843578 +0000 UTC m=+0.151855940 container create 6e44747e5572a2037a20ca5d69c506f71164c63d30c7f2d730e647eec4ca1ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-06T03:51:00.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:00 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:00.236+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:00.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:00 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:00.236+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:00.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:00 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:00.236+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:00.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:00 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:00.236+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:00.578 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:51:00 smithi040 podman[116550]: 2024-06-06 03:51:00.279013907 +0000 UTC m=+0.413026287 container init 6e44747e5572a2037a20ca5d69c506f71164c63d30c7f2d730e647eec4ca1ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-06T03:51:00.579 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:51:00 smithi040 podman[116550]: 2024-06-06 03:51:00.283041071 +0000 UTC m=+0.417053443 container start 6e44747e5572a2037a20ca5d69c506f71164c63d30c7f2d730e647eec4ca1ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-06-06T03:51:00.579 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:51:00 smithi040 podman[116550]: 2024-06-06 03:51:00.387511638 +0000 UTC m=+0.521524010 container attach 6e44747e5572a2037a20ca5d69c506f71164c63d30c7f2d730e647eec4ca1ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:51:00.831 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jun 06 03:51:00 smithi040 podman[116550]: 2024-06-06 03:51:00.586446514 +0000 UTC m=+0.720458885 container died 6e44747e5572a2037a20ca5d69c506f71164c63d30c7f2d730e647eec4ca1ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-06-06T03:51:00.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:00.577+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:00.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:00.577+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:00.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:00.577+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:00.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:00 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:00.577+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:00.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:00 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:00.389+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:51:00.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:00 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:00.389+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:51:00.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:00 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:00.389+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.224518+0000 front 2024-06-06T03:50:30.224558+0000 (oldest deadline 2024-06-06T03:50:54.924666+0000) 2024-06-06T03:51:00.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:00 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:00.396+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:00.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:00 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:00.396+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:00.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:00 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:00.396+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:00.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:00 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:00.396+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:01.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:00 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:00.686+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:01.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:00 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:00.686+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:01.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:00 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:00.686+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:01.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-06T03:51:01.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:51:01.405 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-06T03:51:01.405 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-06T03:51:01.405 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.7 2024-06-06T03:51:01.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:01 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:01.199+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:01.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:01 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:01.199+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:01.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:01 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:01.199+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:01.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:01 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:01.199+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:01.726 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:01 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:01.349+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:01.726 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:01 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:01.349+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:01.727 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:01 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:01.349+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:01.727 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:01 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:01.349+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:01.727 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:01 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:01.383+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:51:01.727 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:01 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:01.383+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:51:01.727 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:01 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:01.383+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.224518+0000 front 2024-06-06T03:50:30.224558+0000 (oldest deadline 2024-06-06T03:50:54.924666+0000) 2024-06-06T03:51:01.727 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:01 smithi104 systemd[1]: Stopping Ceph osd.7 for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:51:02.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:01 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:01.694+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:02.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:01 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:01.694+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:02.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:01 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:01.694+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:02.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:01 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:01.588+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:02.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:01 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:01.588+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:02.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:01 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:01.588+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:02.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:01 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:01.588+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:02.091 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:01 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:01.725+0000 7f6f13edc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:51:02.091 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:01 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:01.725+0000 7f6f13edc700 -1 osd.7 104 *** Got signal Terminated *** 2024-06-06T03:51:02.091 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:01 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:01.725+0000 7f6f13edc700 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T03:51:02.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:02 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:02.169+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:02.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:02 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:02.169+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:02.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:02 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:02.169+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:02.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:02 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:02.169+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:02.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:02 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:02.578+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:02.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:02 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:02.578+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:02.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:02 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:02.578+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:02.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:02 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:02.578+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:02.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:02 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:02.416+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:51:02.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:02 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:02.416+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:51:02.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:02 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:02.416+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.224518+0000 front 2024-06-06T03:50:30.224558+0000 (oldest deadline 2024-06-06T03:50:54.924666+0000) 2024-06-06T03:51:02.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:02 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:02.391+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:02.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:02 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:02.391+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:02.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:02 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:02.391+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:02.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:02 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:02.391+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:03.034 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:02 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:02.681+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:03.035 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:02 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:02.681+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:03.036 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:02 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:02.681+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:03.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:03 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:03.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:03.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:03 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:03.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:03.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:03 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:03.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:03.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:03 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:03.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:03.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:03 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:03.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:03.830 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:03 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:03.549+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:03.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:03 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:03.549+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:03.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:03 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:03.549+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:03.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:03 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:03.549+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:03.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:03 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:03.408+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:51:03.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:03 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:03.408+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:51:03.842 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:03 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:03.408+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.224518+0000 front 2024-06-06T03:50:30.224558+0000 (oldest deadline 2024-06-06T03:50:54.924666+0000) 2024-06-06T03:51:03.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:03 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:03.389+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:03.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:03 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:03.389+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:03.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:03 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:03.389+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:03.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:03 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:03.389+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:04.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:03 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:03.728+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:04.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:03 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:03.728+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:04.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:03 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:03.728+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:04.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:04 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:04.145+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:04.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:04 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:04.145+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:04.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:04 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:04.145+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:04.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:04 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:04.145+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:04.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:04 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:04.145+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:04.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:04 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:04.544+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:04.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:04 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:04.544+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:04.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:04 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:04.544+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:04.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:04 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:04.544+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:04.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:04 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:04.423+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:51:04.842 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:04 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:04.423+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:51:04.842 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:04 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:04.423+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.224518+0000 front 2024-06-06T03:50:30.224558+0000 (oldest deadline 2024-06-06T03:50:54.924666+0000) 2024-06-06T03:51:04.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:04 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:04.382+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:04.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:04 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:04.382+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:04.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:04 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:04.382+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:04.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:04 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:04.382+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:04.843 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:04 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:04.382+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:05.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:04 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:04.720+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:05.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:04 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:04.720+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:05.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:04 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:04.720+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:05.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:05 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:05.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:05.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:05 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:05.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:05.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:05 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:05.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:05.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:05 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:05.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:05.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:05 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:05.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:05.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:05 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:05.399+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:51:05.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:05 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:05.399+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:51:05.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:05 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:05.399+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.224518+0000 front 2024-06-06T03:50:30.224558+0000 (oldest deadline 2024-06-06T03:50:54.924666+0000) 2024-06-06T03:51:05.842 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:05 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:05.399+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.526061+0000 front 2024-06-06T03:50:42.526027+0000 (oldest deadline 2024-06-06T03:51:04.826090+0000) 2024-06-06T03:51:05.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:05 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:05.428+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:05.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:05 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:05.428+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:05.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:05 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:05.428+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:05.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:05 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:05.428+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:05.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:05 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:05.428+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:06.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:05 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:05.749+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:06.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:05 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:05.749+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:06.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:05 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:05.749+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:06.080 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:05 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:05.584+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:06.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:05 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:05.584+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:06.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:05 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:05.584+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:06.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:05 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:05.584+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:06.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:05 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:05.584+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:06.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:06 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:06.135+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:06.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:06 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:06.135+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:06.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:06 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:06.135+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:06.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:06 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:06.135+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:06.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:06 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:06.135+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:06.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:06.429+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:16.821660+0000 front 2024-06-06T03:50:16.821814+0000 (oldest deadline 2024-06-06T03:50:40.321826+0000) 2024-06-06T03:51:06.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:06.429+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:26.223060+0000 front 2024-06-06T03:50:26.223088+0000 (oldest deadline 2024-06-06T03:50:49.723262+0000) 2024-06-06T03:51:06.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:06.429+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.224518+0000 front 2024-06-06T03:50:30.224558+0000 (oldest deadline 2024-06-06T03:50:54.924666+0000) 2024-06-06T03:51:06.841 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7[55005]: 2024-06-06T03:51:06.429+0000 7f6f0c4c8700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.526061+0000 front 2024-06-06T03:50:42.526027+0000 (oldest deadline 2024-06-06T03:51:04.826090+0000) 2024-06-06T03:51:06.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:06.464+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:06.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:06.464+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:06.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:06.464+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:06.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:06.464+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:06.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:06.464+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:07.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:06 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:06.775+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:07.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:06 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:06.775+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:07.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:06 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:06.775+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:07.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:06 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:06.775+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:41.609110+0000 front 2024-06-06T03:50:41.609072+0000 (oldest deadline 2024-06-06T03:51:06.308916+0000) 2024-06-06T03:51:07.080 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:06 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:06.616+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:07.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:06 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:06.616+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:07.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:06 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:06.616+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:07.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:06 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:06.616+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:07.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:06 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:06.616+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:07.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:07 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:07.155+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:07.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:07 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:07.155+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:07.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:07 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:07.155+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:07.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:07 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:07.155+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:07.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:07 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:07.155+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:07.591 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:07 smithi104 podman[73680]: 2024-06-06 03:51:07.290026663 +0000 UTC m=+5.686379546 container died ce22a21785d0ba3de139f781c5cf7b212d7041e56e55dd3ce25f5e4dd887873b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-06T03:51:07.591 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:07 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:07.507+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:07.592 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:07 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:07.507+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:07.592 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:07 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:07.507+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:07.592 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:07 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:07.507+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:07.592 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:07 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:07.507+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:07.960 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:07 smithi104 podman[73680]: 2024-06-06 03:51:07.709460338 +0000 UTC m=+6.105813216 container cleanup ce22a21785d0ba3de139f781c5cf7b212d7041e56e55dd3ce25f5e4dd887873b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-06T03:51:07.960 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:07 smithi104 bash[73680]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7 2024-06-06T03:51:08.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:07 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:07.775+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:08.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:07 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:07.775+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:08.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:07 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:07.775+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:08.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:07 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:07.775+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:41.609110+0000 front 2024-06-06T03:50:41.609072+0000 (oldest deadline 2024-06-06T03:51:06.308916+0000) 2024-06-06T03:51:08.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:07 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:07.581+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:08.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:07 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:07.581+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:08.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:07 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:07.581+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:08.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:07 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:07.581+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:08.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:07 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:07.581+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:08.341 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:07 smithi104 podman[73700]: 2024-06-06 03:51:07.960875139 +0000 UTC m=+0.663688063 container remove ce22a21785d0ba3de139f781c5cf7b212d7041e56e55dd3ce25f5e4dd887873b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-06T03:51:08.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:08 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:08.136+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:08.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:08 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:08.136+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:08.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:08 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:08.136+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:08.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:08 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:08.136+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:08.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:08 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:08.136+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:08.713 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:08 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:08.495+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:08.713 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:08 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:08.495+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:08.714 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:08 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:08.495+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:08.714 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:08 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:08.495+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:08.714 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:08 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:08.495+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:08.714 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:08 smithi104 podman[73788]: 2024-06-06 03:51:08.264821765 +0000 UTC m=+0.017239850 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:51:08.714 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:08 smithi104 podman[73788]: 2024-06-06 03:51:08.470130147 +0000 UTC m=+0.222548265 container create 3d5c6fe597630f9c7afdb5d08830a872957a293816d5b534834e21cec0e6c452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7-deactivate, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-06T03:51:08.714 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:08 smithi104 podman[73788]: 2024-06-06 03:51:08.713917519 +0000 UTC m=+0.466335594 container init 3d5c6fe597630f9c7afdb5d08830a872957a293816d5b534834e21cec0e6c452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-06T03:51:09.000 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:08 smithi104 podman[73788]: 2024-06-06 03:51:08.717415596 +0000 UTC m=+0.469833671 container start 3d5c6fe597630f9c7afdb5d08830a872957a293816d5b534834e21cec0e6c452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7-deactivate, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-06T03:51:09.000 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:08 smithi104 podman[73788]: 2024-06-06 03:51:08.797193598 +0000 UTC m=+0.549611677 container attach 3d5c6fe597630f9c7afdb5d08830a872957a293816d5b534834e21cec0e6c452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-06-06T03:51:09.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:08 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:08.772+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:09.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:08 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:08.772+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:09.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:08 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:08.772+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:09.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:08 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:08.772+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:41.609110+0000 front 2024-06-06T03:50:41.609072+0000 (oldest deadline 2024-06-06T03:51:06.308916+0000) 2024-06-06T03:51:09.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:08 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:08.623+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:09.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:08 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:08.623+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:09.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:08 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:08.623+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:09.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:08 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:08.623+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:09.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:08 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:08.623+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:09.273 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jun 06 03:51:09 smithi104 podman[73788]: 2024-06-06 03:51:09.000536894 +0000 UTC m=+0.752954970 container died 3d5c6fe597630f9c7afdb5d08830a872957a293816d5b534834e21cec0e6c452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-7-deactivate, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-06T03:51:09.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:09 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:09.148+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:09.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:09 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:09.148+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:09.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:09 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:09.148+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:09.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:09 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:09.148+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:09.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:09 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:09.148+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:09.591 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:09 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:09.493+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:09.592 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:09 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:09.493+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:09.592 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:09 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:09.493+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:09.592 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:09 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:09.493+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:09.592 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:09 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:09.493+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:09.624 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-06T03:51:09.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:51:09.661 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-06T03:51:09.661 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-06T03:51:09.661 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.8 2024-06-06T03:51:09.987 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:09 smithi131 systemd[1]: Stopping Ceph osd.8 for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:51:09.987 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:09 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:09.809+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:09.988 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:09 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:09.809+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:09.988 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:09 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:09.809+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:09.988 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:09 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:09.809+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:41.609110+0000 front 2024-06-06T03:50:41.609072+0000 (oldest deadline 2024-06-06T03:51:06.308916+0000) 2024-06-06T03:51:09.988 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:09 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:09.987+0000 7fed058e2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:51:09.988 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:09 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:09.987+0000 7fed058e2700 -1 osd.8 104 *** Got signal Terminated *** 2024-06-06T03:51:09.988 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:09 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:09.987+0000 7fed058e2700 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T03:51:10.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:09 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:09.625+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:10.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:09 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:09.625+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:10.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:09 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:09.625+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:10.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:09 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:09.625+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:10.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:09 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:09.625+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:10.282 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:10 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:10.173+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:10.283 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:10 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:10.173+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:10.283 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:10 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:10.173+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:10.283 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:10 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:10.173+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:10.283 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:10 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:10.173+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:10.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:10 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:10.505+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:10.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:10 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:10.505+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:10.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:10 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:10.505+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:10.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:10 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:10.505+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:10.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:10 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:10.505+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:11.032 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:10 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:10.774+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:11.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:10 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:10.774+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:11.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:10 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:10.774+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:11.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:10 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:10.774+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:41.609110+0000 front 2024-06-06T03:50:41.609072+0000 (oldest deadline 2024-06-06T03:51:06.308916+0000) 2024-06-06T03:51:11.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:10 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:10.605+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:11.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:10 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:10.605+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:11.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:10 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:10.605+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:11.082 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:10 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:10.605+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:11.082 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:10 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:10.605+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:11.313 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:11 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:11.190+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:11.314 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:11 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:11.190+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:11.314 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:11 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:11.190+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:11.314 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:11 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:11.190+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:11.314 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:11 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:11.190+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:11.743 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:11 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:11.468+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:11.743 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:11 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:11.468+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:11.743 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:11 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:11.468+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:11.743 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:11 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:11.468+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:11.743 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:11 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:11.468+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:11.830 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:11 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:11.575+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:11.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:11 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:11.575+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:11.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:11 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:11.575+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:11.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:11 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:11.575+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:11.831 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:11 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:11.575+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:12.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:11 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:11.773+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:12.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:11 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:11.773+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:12.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:11 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:11.773+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:12.033 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:11 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:11.773+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:41.609110+0000 front 2024-06-06T03:50:41.609072+0000 (oldest deadline 2024-06-06T03:51:06.308916+0000) 2024-06-06T03:51:12.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:12 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:12.216+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:12.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:12 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:12.216+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:12.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:12 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:12.216+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:12.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:12 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:12.216+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:12.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:12 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:12.216+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:12.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:12 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:12.216+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:12.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:12 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:12.478+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:12.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:12 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:12.478+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:12.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:12 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:12.478+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:12.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:12 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:12.478+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:12.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:12 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:12.478+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:13.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:12 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:12.613+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:13.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:12 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:12.613+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:13.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:12 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:12.613+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:13.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:12 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:12.613+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:13.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:12 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:12.613+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:13.212 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:12 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:12.818+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:13.212 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:12 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:12.818+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:13.213 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:12 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:12.818+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:13.213 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:12 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:12.818+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:41.609110+0000 front 2024-06-06T03:50:41.609072+0000 (oldest deadline 2024-06-06T03:51:06.308916+0000) 2024-06-06T03:51:13.213 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:12 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:12.818+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:46.809722+0000 front 2024-06-06T03:50:46.809753+0000 (oldest deadline 2024-06-06T03:51:12.709869+0000) 2024-06-06T03:51:13.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:13 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:13.212+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:13.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:13 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:13.212+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:13.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:13 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:13.212+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:13.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:13 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:13.212+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:13.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:13 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:13.212+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:13.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:13 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:13.212+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:13.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:13 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:13.481+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:13.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:13 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:13.481+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:13.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:13 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:13.481+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:13.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:13 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:13.481+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:13.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:13 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:13.481+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:13.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:13 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:13.481+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:14.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:13 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:13.630+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:14.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:13 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:13.630+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:14.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:13 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:13.630+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:14.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:13 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:13.630+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:14.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:13 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:13.630+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:14.082 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:13 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:13.630+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:50.254077+0000 front 2024-06-06T03:50:50.254089+0000 (oldest deadline 2024-06-06T03:51:13.154047+0000) 2024-06-06T03:51:14.179 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:13 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:13.832+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:14.179 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:13 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:13.832+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:14.179 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:13 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:13.832+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:14.179 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:13 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:13.832+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:41.609110+0000 front 2024-06-06T03:50:41.609072+0000 (oldest deadline 2024-06-06T03:51:06.308916+0000) 2024-06-06T03:51:14.179 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:13 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:13.832+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:46.809722+0000 front 2024-06-06T03:50:46.809753+0000 (oldest deadline 2024-06-06T03:51:12.709869+0000) 2024-06-06T03:51:14.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:14 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:14.178+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:14.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:14 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:14.178+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:14.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:14 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:14.178+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:14.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:14 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:14.178+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:14.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:14 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:14.178+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:14.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:14 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:14.178+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:14.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:14 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:14.458+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:14.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:14 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:14.458+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:14.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:14 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:14.458+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:14.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:14 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:14.458+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:14.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:14 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:14.459+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:14.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:14 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:14.459+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:15.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:14 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:14.628+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:15.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:14 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:14.628+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:15.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:14 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:14.628+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:15.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:14 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:14.628+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:15.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:14 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:14.628+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:15.082 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:14 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:14.628+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:50.254077+0000 front 2024-06-06T03:50:50.254089+0000 (oldest deadline 2024-06-06T03:51:13.154047+0000) 2024-06-06T03:51:15.130 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:14 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:14.811+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.904504+0000 front 2024-06-06T03:50:15.904528+0000 (oldest deadline 2024-06-06T03:50:39.404625+0000) 2024-06-06T03:51:15.130 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:14 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:14.811+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.305814+0000 front 2024-06-06T03:50:25.305793+0000 (oldest deadline 2024-06-06T03:50:47.006011+0000) 2024-06-06T03:51:15.130 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:14 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:14.811+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:29.307150+0000 front 2024-06-06T03:50:29.307184+0000 (oldest deadline 2024-06-06T03:50:55.207199+0000) 2024-06-06T03:51:15.131 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:14 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:14.811+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:41.609110+0000 front 2024-06-06T03:50:41.609072+0000 (oldest deadline 2024-06-06T03:51:06.308916+0000) 2024-06-06T03:51:15.131 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:14 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8[55400]: 2024-06-06T03:51:14.811+0000 7fecfdece700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:46.809722+0000 front 2024-06-06T03:50:46.809753+0000 (oldest deadline 2024-06-06T03:51:12.709869+0000) 2024-06-06T03:51:15.500 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:15 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:15.130+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:15.500 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:15 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:15.130+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:15.501 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:15 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:15.130+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:15.501 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:15 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:15.130+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:15.501 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:15 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:15.130+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:15.501 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:15 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:15.130+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:15.782 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:15 smithi131 podman[75618]: 2024-06-06 03:51:15.500766762 +0000 UTC m=+5.645018760 container died 9818cf9103de59769644bd837a838746ba575ac9a6b5bf9b2d8aa7dd5525c6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:51:15.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:15 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:15.457+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:15.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:15 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:15.457+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:15.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:15 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:15.457+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:15.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:15 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:15.457+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:15.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:15 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:15.457+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:15.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:15 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:15.457+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:16.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:15 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:15.586+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:16.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:15 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:15.586+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:16.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:15 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:15.586+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:16.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:15 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:15.586+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:16.082 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:15 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:15.586+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:16.082 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:15 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:15.586+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:50.254077+0000 front 2024-06-06T03:50:50.254089+0000 (oldest deadline 2024-06-06T03:51:13.154047+0000) 2024-06-06T03:51:16.121 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:15 smithi131 podman[75618]: 2024-06-06 03:51:15.887606915 +0000 UTC m=+6.031858907 container cleanup 9818cf9103de59769644bd837a838746ba575ac9a6b5bf9b2d8aa7dd5525c6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-06T03:51:16.121 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:15 smithi131 bash[75618]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8 2024-06-06T03:51:16.464 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:16 smithi131 podman[75647]: 2024-06-06 03:51:16.139038473 +0000 UTC m=+0.632840468 container remove 9818cf9103de59769644bd837a838746ba575ac9a6b5bf9b2d8aa7dd5525c6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-06T03:51:16.465 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:16 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:16.120+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:16.465 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:16 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:16.120+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:16.465 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:16 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:16.120+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:16.465 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:16 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:16.120+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:16.465 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:16 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:16.120+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:16.465 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:16 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:16.120+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:16.716 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:16 smithi131 podman[75734]: 2024-06-06 03:51:16.367497045 +0000 UTC m=+0.017008038 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:51:16.717 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:16 smithi131 podman[75734]: 2024-06-06 03:51:16.481661772 +0000 UTC m=+0.131172758 container create 90c3f9ede5d2facaca9e1d912708f03add2cb53aa963848553874bc5809e65ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-06T03:51:16.717 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:16 smithi131 podman[75734]: 2024-06-06 03:51:16.717446624 +0000 UTC m=+0.366957603 container init 90c3f9ede5d2facaca9e1d912708f03add2cb53aa963848553874bc5809e65ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-06T03:51:16.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:16 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:16.464+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:16.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:16 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:16.464+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:16.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:16 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:16.464+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:16.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:16 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:16.464+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:16.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:16 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:16.464+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:16.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:16 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:16.464+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:17.009 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:16 smithi131 podman[75734]: 2024-06-06 03:51:16.720520382 +0000 UTC m=+0.370031372 container start 90c3f9ede5d2facaca9e1d912708f03add2cb53aa963848553874bc5809e65ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-06T03:51:17.009 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:16 smithi131 podman[75734]: 2024-06-06 03:51:16.809455576 +0000 UTC m=+0.458966560 container attach 90c3f9ede5d2facaca9e1d912708f03add2cb53aa963848553874bc5809e65ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:51:17.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:16 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:16.624+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:17.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:16 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:16.624+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:17.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:16 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:16.624+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:17.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:16 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:16.624+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:17.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:16 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:16.624+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:17.082 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:16 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:16.624+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:50.254077+0000 front 2024-06-06T03:50:50.254089+0000 (oldest deadline 2024-06-06T03:51:13.154047+0000) 2024-06-06T03:51:17.283 INFO:journalctl@ceph.osd.8.smithi131.stdout:Jun 06 03:51:17 smithi131 podman[75734]: 2024-06-06 03:51:17.009887497 +0000 UTC m=+0.659398484 container died 90c3f9ede5d2facaca9e1d912708f03add2cb53aa963848553874bc5809e65ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:51:17.283 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:17 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:17.108+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:17.283 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:17 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:17.108+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:17.283 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:17 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:17.108+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:17.284 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:17 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:17.108+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:17.284 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:17 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:17.108+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:17.284 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:17 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:17.108+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:17.694 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.8.service' 2024-06-06T03:51:17.733 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:51:17.733 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-06T03:51:17.734 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-06T03:51:17.734 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.9 2024-06-06T03:51:17.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:17 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:17.452+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:17.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:17 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:17.452+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:17.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:17 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:17.452+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:17.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:17 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:17.452+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:17.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:17 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:17.452+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:17.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:17 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:17.452+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:18.069 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:17 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:17.600+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:18.070 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:17 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:17.600+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:18.070 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:17 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:17.600+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:18.070 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:17 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:17.600+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:18.070 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:17 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:17.600+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:18.070 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:17 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:17.600+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:50.254077+0000 front 2024-06-06T03:50:50.254089+0000 (oldest deadline 2024-06-06T03:51:13.154047+0000) 2024-06-06T03:51:18.070 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:17 smithi040 systemd[1]: Stopping Ceph osd.9 for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:51:18.331 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:18.068+0000 7f1a8bd85700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:51:18.331 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:18.068+0000 7f1a8bd85700 -1 osd.9 104 *** Got signal Terminated *** 2024-06-06T03:51:18.331 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:18.068+0000 7f1a8bd85700 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T03:51:18.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:18 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:18.114+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:18.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:18 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:18.114+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:18.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:18 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:18.114+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:18.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:18 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:18.114+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:18.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:18 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:18.114+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:18.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:18 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:18.114+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:18.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:18 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:18.500+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:18.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:18 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:18.500+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:18.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:18 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:18.500+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:18.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:18 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:18.500+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:18.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:18 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:18.500+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:18.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:18 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:18.500+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:19.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:18.617+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:19.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:18.617+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:19.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:18.617+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:19.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:18.617+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:19.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:18.617+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:19.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:18 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:18.617+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:50.254077+0000 front 2024-06-06T03:50:50.254089+0000 (oldest deadline 2024-06-06T03:51:13.154047+0000) 2024-06-06T03:51:19.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:19 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:19.111+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:19.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:19 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:19.111+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:19.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:19 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:19.111+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:19.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:19 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:19.111+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:19.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:19 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:19.111+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:19.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:19 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:19.111+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:19.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:19 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:19.544+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:19.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:19 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:19.544+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:19.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:19 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:19.544+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:19.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:19 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:19.544+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:19.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:19 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:19.544+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:19.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:19 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:19.544+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:20.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:19 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:19.659+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:20.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:19 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:19.659+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:20.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:19 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:19.659+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:20.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:19 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:19.659+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:20.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:19 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:19.659+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:20.082 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:19 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:19.659+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:50.254077+0000 front 2024-06-06T03:50:50.254089+0000 (oldest deadline 2024-06-06T03:51:13.154047+0000) 2024-06-06T03:51:20.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:20 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:20.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:20.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:20 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:20.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:20.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:20 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:20.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:20.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:20 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:20.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:20.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:20 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:20.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:20.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:20 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:20.119+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:20.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:20 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:20.570+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:20.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:20 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:20.570+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:20.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:20 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:20.570+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:20.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:20 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:20.570+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:20.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:20 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:20.570+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:20.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:20 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:20.570+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:20.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:20 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:20.570+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2024-06-06T03:50:57.297019+0000 front 2024-06-06T03:50:57.297083+0000 (oldest deadline 2024-06-06T03:51:19.596852+0000) 2024-06-06T03:51:21.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:20 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:20.647+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:21.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:20 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:20.647+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:21.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:20 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:20.647+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:21.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:20 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:20.647+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:21.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:20 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:20.647+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:21.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:20 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:20.647+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:50.254077+0000 front 2024-06-06T03:50:50.254089+0000 (oldest deadline 2024-06-06T03:51:13.154047+0000) 2024-06-06T03:51:21.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:21 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:21.168+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:21.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:21 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:21.168+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:21.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:21 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:21.168+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:21.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:21 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:21.168+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:21.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:21 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:21.168+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:21.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:21 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:21.168+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:21.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:21 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:21.579+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:21.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:21 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:21.579+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:21.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:21 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:21.579+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:21.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:21 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:21.579+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:21.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:21 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:21.579+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:21.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:21 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:21.580+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:21.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:21 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:21.580+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2024-06-06T03:50:57.297019+0000 front 2024-06-06T03:50:57.297083+0000 (oldest deadline 2024-06-06T03:51:19.596852+0000) 2024-06-06T03:51:22.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:21 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:21.609+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:22.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:21 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:21.609+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:22.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:21 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:21.609+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:22.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:21 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:21.609+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:22.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:21 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:21.609+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:22.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:21 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:21.609+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:50.254077+0000 front 2024-06-06T03:50:50.254089+0000 (oldest deadline 2024-06-06T03:51:13.154047+0000) 2024-06-06T03:51:22.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:22 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:22.186+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:22.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:22 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:22.186+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:22.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:22 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:22.186+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:22.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:22 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:22.186+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:22.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:22 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:22.187+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:22.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:22 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:22.187+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:22.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:22 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:22.560+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:22.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:22 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:22.560+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:22.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:22 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:22.561+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:22.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:22 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:22.561+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:22.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:22 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:22.561+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:22.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:22 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:22.561+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:22.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:22 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:22.561+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2024-06-06T03:50:57.297019+0000 front 2024-06-06T03:50:57.297083+0000 (oldest deadline 2024-06-06T03:51:19.596852+0000) 2024-06-06T03:51:23.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:22 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:22.644+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:08.545014+0000 front 2024-06-06T03:50:08.545042+0000 (oldest deadline 2024-06-06T03:50:32.045380+0000) 2024-06-06T03:51:23.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:22 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:22.644+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:17.946890+0000 front 2024-06-06T03:50:17.946783+0000 (oldest deadline 2024-06-06T03:50:40.846876+0000) 2024-06-06T03:51:23.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:22 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:22.644+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.248677+0000 front 2024-06-06T03:50:24.248699+0000 (oldest deadline 2024-06-06T03:50:47.748707+0000) 2024-06-06T03:51:23.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:22 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:22.644+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.249640+0000 front 2024-06-06T03:50:31.249706+0000 (oldest deadline 2024-06-06T03:50:54.650336+0000) 2024-06-06T03:51:23.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:22 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:22.645+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.053046+0000 front 2024-06-06T03:50:42.053121+0000 (oldest deadline 2024-06-06T03:51:04.952956+0000) 2024-06-06T03:51:23.082 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:22 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9[67337]: 2024-06-06T03:51:22.645+0000 7f1a88379700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:50.254077+0000 front 2024-06-06T03:50:50.254089+0000 (oldest deadline 2024-06-06T03:51:13.154047+0000) 2024-06-06T03:51:23.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:23 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:23.210+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:23.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:23 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:23.210+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:23.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:23 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:23.210+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:23.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:23 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:23.210+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:23.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:23 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:23.210+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:23.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:23 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:23.210+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:23.786 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:23 smithi040 podman[116671]: 2024-06-06 03:51:23.535605236 +0000 UTC m=+5.597491268 container died 92eed75809289a23f23b49d40d96bf30e56514ad7f9736ddb7e88c2e958454a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-06T03:51:23.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:23 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:23.513+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:23.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:23 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:23.513+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:23.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:23 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:23.513+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:23.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:23 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:23.513+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:23.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:23 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:23.513+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:23.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:23 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:23.513+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:23.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:23 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:23.513+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2024-06-06T03:50:57.297019+0000 front 2024-06-06T03:50:57.297083+0000 (oldest deadline 2024-06-06T03:51:19.596852+0000) 2024-06-06T03:51:24.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:23 smithi040 podman[116671]: 2024-06-06 03:51:23.918944238 +0000 UTC m=+5.980830300 container cleanup 92eed75809289a23f23b49d40d96bf30e56514ad7f9736ddb7e88c2e958454a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-06T03:51:24.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:23 smithi040 bash[116671]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9 2024-06-06T03:51:24.445 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:24 smithi040 podman[116686]: 2024-06-06 03:51:24.12850274 +0000 UTC m=+0.582090133 container remove 92eed75809289a23f23b49d40d96bf30e56514ad7f9736ddb7e88c2e958454a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9, org.label-schema.build-date=20240528, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-06T03:51:24.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:24 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:24.241+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:24.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:24 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:24.241+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:24.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:24 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:24.242+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:24.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:24 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:24.242+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:24.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:24 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:24.242+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:24.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:24 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:24.242+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:24.764 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:24 smithi040 podman[116773]: 2024-06-06 03:51:24.44595268 +0000 UTC m=+0.107946393 container create 7d5ad9811dae55fa3cd6fe23f0a80c4ff5365b66111b065efbbe4ef4a57fb452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T03:51:24.765 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:24 smithi040 podman[116773]: 2024-06-06 03:51:24.35592305 +0000 UTC m=+0.017916783 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:51:24.765 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:24 smithi040 podman[116773]: 2024-06-06 03:51:24.681725125 +0000 UTC m=+0.343718847 container init 7d5ad9811dae55fa3cd6fe23f0a80c4ff5365b66111b065efbbe4ef4a57fb452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9-deactivate, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528) 2024-06-06T03:51:24.765 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:24 smithi040 podman[116773]: 2024-06-06 03:51:24.685312865 +0000 UTC m=+0.347306581 container start 7d5ad9811dae55fa3cd6fe23f0a80c4ff5365b66111b065efbbe4ef4a57fb452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-06T03:51:24.765 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:24 smithi040 podman[116773]: 2024-06-06 03:51:24.765072058 +0000 UTC m=+0.427065797 container attach 7d5ad9811dae55fa3cd6fe23f0a80c4ff5365b66111b065efbbe4ef4a57fb452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T03:51:24.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:24 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:24.496+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:24.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:24 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:24.496+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:24.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:24 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:24.496+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:24.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:24 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:24.496+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:24.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:24 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:24.496+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:24.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:24 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:24.496+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:24.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:24 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:24.496+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2024-06-06T03:50:57.297019+0000 front 2024-06-06T03:50:57.297083+0000 (oldest deadline 2024-06-06T03:51:19.596852+0000) 2024-06-06T03:51:25.081 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:24 smithi040 podman[116773]: 2024-06-06 03:51:24.991973648 +0000 UTC m=+0.653967362 container died 7d5ad9811dae55fa3cd6fe23f0a80c4ff5365b66111b065efbbe4ef4a57fb452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-06T03:51:25.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:25 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:25.210+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:25.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:25 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:25.210+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:25.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:25 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:25.210+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:25.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:25 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:25.210+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:25.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:25 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:25.210+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:25.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:25 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:25.210+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:25.537 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.9.service' 2024-06-06T03:51:25.563 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:25 smithi040 podman[116799]: 2024-06-06 03:51:25.518596248 +0000 UTC m=+0.517442418 container remove 7d5ad9811dae55fa3cd6fe23f0a80c4ff5365b66111b065efbbe4ef4a57fb452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-06T03:51:25.563 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:25 smithi040 systemd[1]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.9.service: Deactivated successfully. 2024-06-06T03:51:25.564 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:25 smithi040 systemd[1]: Stopped Ceph osd.9 for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:51:25.564 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jun 06 03:51:25 smithi040 systemd[1]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.9.service: Consumed 52.422s CPU time. 2024-06-06T03:51:25.580 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:51:25.581 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-06T03:51:25.581 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-06T03:51:25.581 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.10 2024-06-06T03:51:25.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:25 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:25.448+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:25.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:25 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:25.448+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:25.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:25 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:25.448+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:25.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:25 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:25.448+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:25.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:25 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:25.448+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:25.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:25 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:25.448+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:25.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:25 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:25.448+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2024-06-06T03:50:57.297019+0000 front 2024-06-06T03:50:57.297083+0000 (oldest deadline 2024-06-06T03:51:19.596852+0000) 2024-06-06T03:51:25.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:25 smithi104 systemd[1]: Stopping Ceph osd.10 for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:51:26.341 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:25 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:25.893+0000 7fd40fb79700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:51:26.341 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:25 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:25.893+0000 7fd40fb79700 -1 osd.10 104 *** Got signal Terminated *** 2024-06-06T03:51:26.341 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:25 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:25.893+0000 7fd40fb79700 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T03:51:26.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:26 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:26.193+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:26.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:26 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:26.193+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:26.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:26 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:26.193+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:26.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:26 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:26.193+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:26.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:26 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:26.193+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:26.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:26 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:26.193+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:26.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:26 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:26.414+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:26.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:26 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:26.414+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:26.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:26 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:26.414+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:26.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:26 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:26.414+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:26.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:26 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:26.414+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:26.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:26 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:26.414+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:26.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:26 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:26.414+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2024-06-06T03:50:57.297019+0000 front 2024-06-06T03:50:57.297083+0000 (oldest deadline 2024-06-06T03:51:19.596852+0000) 2024-06-06T03:51:27.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:27 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:27.150+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:27.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:27 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:27.150+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:27.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:27 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:27.150+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:27.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:27 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:27.150+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:27.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:27 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:27.150+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:27.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:27 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:27.150+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:27.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:27 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:27.409+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:27.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:27 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:27.409+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:27.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:27 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:27.409+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:27.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:27 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:27.409+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:27.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:27 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:27.409+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:27.843 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:27 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:27.409+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:27.843 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:27 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:27.409+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2024-06-06T03:50:57.297019+0000 front 2024-06-06T03:50:57.297083+0000 (oldest deadline 2024-06-06T03:51:19.596852+0000) 2024-06-06T03:51:28.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:28 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:28.159+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:28.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:28 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:28.159+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:28.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:28 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:28.159+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:28.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:28 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:28.159+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:28.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:28 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:28.159+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:28.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:28 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:28.159+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:28.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:28 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:28.448+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:28.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:28 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:28.448+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:28.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:28 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:28.448+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:28.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:28 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:28.448+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:28.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:28 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:28.448+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:28.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:28 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:28.448+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:28.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:28 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:28.448+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2024-06-06T03:50:57.297019+0000 front 2024-06-06T03:50:57.297083+0000 (oldest deadline 2024-06-06T03:51:19.596852+0000) 2024-06-06T03:51:29.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:29 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:29.188+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:29.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:29 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:29.188+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:29.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:29 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:29.188+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:29.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:29 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:29.188+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:29.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:29 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:29.188+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:29.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:29 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:29.188+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:29.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:29 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:29.493+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:29.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:29 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:29.493+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:29.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:29 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:29.493+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:29.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:29 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:29.493+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:29.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:29 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:29.493+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:29.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:29 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:29.493+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:29.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:29 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:29.493+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2024-06-06T03:50:57.297019+0000 front 2024-06-06T03:50:57.297083+0000 (oldest deadline 2024-06-06T03:51:19.596852+0000) 2024-06-06T03:51:30.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:30 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:30.169+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:30.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:30 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:30.169+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:30.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:30 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:30.169+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:30.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:30 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:30.169+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:30.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:30 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:30.169+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:30.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:30 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:30.169+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:30.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:30 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:30.492+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:04.587539+0000 front 2024-06-06T03:50:04.587448+0000 (oldest deadline 2024-06-06T03:50:30.487613+0000) 2024-06-06T03:51:30.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:30 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:30.492+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:15.188658+0000 front 2024-06-06T03:50:15.188688+0000 (oldest deadline 2024-06-06T03:50:39.288912+0000) 2024-06-06T03:51:30.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:30 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:30.492+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:24.390897+0000 front 2024-06-06T03:50:24.390983+0000 (oldest deadline 2024-06-06T03:50:46.690879+0000) 2024-06-06T03:51:30.841 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:30 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:30.492+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:31.292437+0000 front 2024-06-06T03:50:31.292475+0000 (oldest deadline 2024-06-06T03:50:54.792458+0000) 2024-06-06T03:51:30.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:30 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:30.492+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.294382+0000 front 2024-06-06T03:50:42.294399+0000 (oldest deadline 2024-06-06T03:51:03.994432+0000) 2024-06-06T03:51:30.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:30 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:30.492+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:49.895669+0000 front 2024-06-06T03:50:49.895604+0000 (oldest deadline 2024-06-06T03:51:12.795321+0000) 2024-06-06T03:51:30.842 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:30 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10[61733]: 2024-06-06T03:51:30.492+0000 7fd40c16d700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2024-06-06T03:50:57.297019+0000 front 2024-06-06T03:50:57.297083+0000 (oldest deadline 2024-06-06T03:51:19.596852+0000) 2024-06-06T03:51:31.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:31 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:31.137+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:31.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:31 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:31.137+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:31.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:31 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:31.137+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:31.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:31 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:31.137+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:31.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:31 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:31.137+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:31.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:31 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:31.137+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:31.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:31 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:31.137+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-06-06T03:51:05.818872+0000 front 2024-06-06T03:51:05.818921+0000 (oldest deadline 2024-06-06T03:51:30.518829+0000) 2024-06-06T03:51:31.591 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:31 smithi104 podman[73908]: 2024-06-06 03:51:31.329249769 +0000 UTC m=+5.557999798 container died 4d4ec4a286e8e13191dce20aa8be5ebf0d05a9a2b0cb9f2ff45092410c759f81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-06T03:51:32.002 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:31 smithi104 podman[73908]: 2024-06-06 03:51:31.701752736 +0000 UTC m=+5.930502766 container cleanup 4d4ec4a286e8e13191dce20aa8be5ebf0d05a9a2b0cb9f2ff45092410c759f81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5) 2024-06-06T03:51:32.003 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:31 smithi104 bash[73908]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10 2024-06-06T03:51:32.319 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:32 smithi104 podman[73923]: 2024-06-06 03:51:32.00313412 +0000 UTC m=+0.664352578 container remove 4d4ec4a286e8e13191dce20aa8be5ebf0d05a9a2b0cb9f2ff45092410c759f81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:51:32.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:32 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:32.091+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:32.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:32 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:32.091+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:32.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:32 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:32.091+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:32.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:32 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:32.091+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:32.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:32 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:32.091+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:32.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:32 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:32.091+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:32.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:32 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:32.091+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-06-06T03:51:05.818872+0000 front 2024-06-06T03:51:05.818921+0000 (oldest deadline 2024-06-06T03:51:30.518829+0000) 2024-06-06T03:51:32.591 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:32 smithi104 podman[74013]: 2024-06-06 03:51:32.222392687 +0000 UTC m=+0.016958940 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:51:32.591 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:32 smithi104 podman[74013]: 2024-06-06 03:51:32.328862498 +0000 UTC m=+0.123428746 container create 6365cc4c024557f4864511f7b580b1a91fcfd1fea9c3c2bfc0582a90f8aaa366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-06T03:51:32.591 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:32 smithi104 podman[74013]: 2024-06-06 03:51:32.547647668 +0000 UTC m=+0.342213921 container init 6365cc4c024557f4864511f7b580b1a91fcfd1fea9c3c2bfc0582a90f8aaa366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-06T03:51:32.591 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:32 smithi104 podman[74013]: 2024-06-06 03:51:32.550692959 +0000 UTC m=+0.345259213 container start 6365cc4c024557f4864511f7b580b1a91fcfd1fea9c3c2bfc0582a90f8aaa366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:51:33.084 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:32 smithi104 podman[74013]: 2024-06-06 03:51:32.639253974 +0000 UTC m=+0.433820238 container attach 6365cc4c024557f4864511f7b580b1a91fcfd1fea9c3c2bfc0582a90f8aaa366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-06T03:51:33.084 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:32 smithi104 podman[74039]: 2024-06-06 03:51:32.86117747 +0000 UTC m=+0.018164913 container died 6365cc4c024557f4864511f7b580b1a91fcfd1fea9c3c2bfc0582a90f8aaa366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-06T03:51:33.341 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jun 06 03:51:33 smithi104 podman[74039]: 2024-06-06 03:51:33.316881249 +0000 UTC m=+0.473868694 container remove 6365cc4c024557f4864511f7b580b1a91fcfd1fea9c3c2bfc0582a90f8aaa366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-06T03:51:33.350 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.10.service' 2024-06-06T03:51:33.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:51:33.388 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-06T03:51:33.389 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-06T03:51:33.389 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.11 2024-06-06T03:51:33.415 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:33 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:33.062+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:33.415 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:33 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:33.062+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:33.415 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:33 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:33.062+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:33.415 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:33 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:33.062+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:33.415 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:33 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:33.062+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:33.415 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:33 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:33.062+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:33.416 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:33 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:33.062+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-06-06T03:51:05.818872+0000 front 2024-06-06T03:51:05.818921+0000 (oldest deadline 2024-06-06T03:51:30.518829+0000) 2024-06-06T03:51:33.699 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:33 smithi131 systemd[1]: Stopping Ceph osd.11 for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:51:33.699 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:33 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:33.698+0000 7f888e0e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:51:33.700 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:33 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:33.698+0000 7f888e0e0700 -1 osd.11 104 *** Got signal Terminated *** 2024-06-06T03:51:33.700 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:33 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:33.698+0000 7f888e0e0700 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T03:51:34.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:34 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:34.093+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:34.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:34 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:34.093+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:34.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:34 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:34.093+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:34.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:34 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:34.093+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:34.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:34 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:34.093+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:34.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:34 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:34.093+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:34.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:34 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:34.093+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-06-06T03:51:05.818872+0000 front 2024-06-06T03:51:05.818921+0000 (oldest deadline 2024-06-06T03:51:30.518829+0000) 2024-06-06T03:51:35.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:35.141+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:35.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:35.141+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:35.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:35.141+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:35.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:35.141+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:35.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:35.141+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:35.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:35.141+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:35.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:35 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:35.141+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-06-06T03:51:05.818872+0000 front 2024-06-06T03:51:05.818921+0000 (oldest deadline 2024-06-06T03:51:30.518829+0000) 2024-06-06T03:51:36.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:36 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:36.125+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:36.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:36 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:36.125+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:36.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:36 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:36.125+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:36.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:36 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:36.125+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:36.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:36 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:36.125+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:36.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:36 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:36.125+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:36.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:36 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:36.125+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-06-06T03:51:05.818872+0000 front 2024-06-06T03:51:05.818921+0000 (oldest deadline 2024-06-06T03:51:30.518829+0000) 2024-06-06T03:51:37.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:37 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:37.083+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:37.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:37 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:37.083+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:37.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:37 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:37.083+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:37.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:37 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:37.083+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:37.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:37 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:37.083+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:37.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:37 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:37.083+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:37.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:37 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:37.083+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-06-06T03:51:05.818872+0000 front 2024-06-06T03:51:05.818921+0000 (oldest deadline 2024-06-06T03:51:30.518829+0000) 2024-06-06T03:51:38.532 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:38 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:38.079+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-06-06T03:50:06.809907+0000 front 2024-06-06T03:50:06.810054+0000 (oldest deadline 2024-06-06T03:50:33.810658+0000) 2024-06-06T03:51:38.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:38 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:38.079+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-06-06T03:50:13.811475+0000 front 2024-06-06T03:50:13.811328+0000 (oldest deadline 2024-06-06T03:50:39.111250+0000) 2024-06-06T03:51:38.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:38 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:38.079+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-06-06T03:50:25.512746+0000 front 2024-06-06T03:50:25.512727+0000 (oldest deadline 2024-06-06T03:50:50.212998+0000) 2024-06-06T03:51:38.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:38 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:38.079+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2024-06-06T03:50:30.213449+0000 front 2024-06-06T03:50:30.213496+0000 (oldest deadline 2024-06-06T03:50:56.113357+0000) 2024-06-06T03:51:38.533 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:38 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:38.079+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-06-06T03:50:42.014528+0000 front 2024-06-06T03:50:42.014486+0000 (oldest deadline 2024-06-06T03:51:03.114540+0000) 2024-06-06T03:51:38.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:38 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:38.079+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-06-06T03:50:47.215506+0000 front 2024-06-06T03:50:47.215536+0000 (oldest deadline 2024-06-06T03:51:11.915490+0000) 2024-06-06T03:51:38.534 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:38 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11[62416]: 2024-06-06T03:51:38.079+0000 7f88866cc700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-06-06T03:51:05.818872+0000 front 2024-06-06T03:51:05.818921+0000 (oldest deadline 2024-06-06T03:51:30.518829+0000) 2024-06-06T03:51:39.516 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:39 smithi131 podman[75842]: 2024-06-06 03:51:39.25750743 +0000 UTC m=+5.673618196 container died ba2d5cc80df17d3587346bedd20544fdbb4a6c52fdad67bcd9f0e0b653c46f97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T03:51:39.783 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:39 smithi131 podman[75842]: 2024-06-06 03:51:39.599636803 +0000 UTC m=+6.015747591 container cleanup ba2d5cc80df17d3587346bedd20544fdbb4a6c52fdad67bcd9f0e0b653c46f97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-06T03:51:39.783 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:39 smithi131 bash[75842]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11 2024-06-06T03:51:40.216 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:39 smithi131 podman[75860]: 2024-06-06 03:51:39.792440107 +0000 UTC m=+0.530024578 container remove ba2d5cc80df17d3587346bedd20544fdbb4a6c52fdad67bcd9f0e0b653c46f97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-06T03:51:40.503 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:40 smithi131 podman[75947]: 2024-06-06 03:51:40.119631102 +0000 UTC m=+0.017190149 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T03:51:40.503 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:40 smithi131 podman[75947]: 2024-06-06 03:51:40.226724811 +0000 UTC m=+0.124283906 container create 4396aee505dfb39337b7bb7d5cfd4e9ac617b9a9be567c4c47d4b60293d70864 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11-deactivate, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-06T03:51:40.503 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:40 smithi131 podman[75947]: 2024-06-06 03:51:40.429088057 +0000 UTC m=+0.326647093 container init 4396aee505dfb39337b7bb7d5cfd4e9ac617b9a9be567c4c47d4b60293d70864 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-06T03:51:40.504 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:40 smithi131 podman[75947]: 2024-06-06 03:51:40.43215162 +0000 UTC m=+0.329710656 container start 4396aee505dfb39337b7bb7d5cfd4e9ac617b9a9be567c4c47d4b60293d70864 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-06T03:51:40.504 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:40 smithi131 podman[75947]: 2024-06-06 03:51:40.504035066 +0000 UTC m=+0.401594114 container attach 4396aee505dfb39337b7bb7d5cfd4e9ac617b9a9be567c4c47d4b60293d70864 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-06T03:51:40.782 INFO:journalctl@ceph.osd.11.smithi131.stdout:Jun 06 03:51:40 smithi131 podman[75947]: 2024-06-06 03:51:40.72579247 +0000 UTC m=+0.623351506 container died 4396aee505dfb39337b7bb7d5cfd4e9ac617b9a9be567c4c47d4b60293d70864 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T03:51:41.345 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@osd.11.service' 2024-06-06T03:51:41.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:51:41.423 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-06T03:51:41.423 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-06T03:51:41.424 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.a 2024-06-06T03:51:41.738 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:51:41 smithi040 systemd[1]: Stopping Ceph mds.a for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:51:42.080 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:51:41 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-a[74733]: 2024-06-06T03:51:41.736+0000 7fddd55d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:51:42.081 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:51:41 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-a[74733]: 2024-06-06T03:51:41.736+0000 7fddd55d4700 -1 mds.a *** got signal Terminated *** 2024-06-06T03:51:45.581 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:51:45 smithi040 podman[116880]: 2024-06-06 03:51:45.330673201 +0000 UTC m=+3.698191633 container died 8f73f8f00067447387e4ab151be22708f3699a840d83ec470ea5125335c0cb87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-06T03:51:46.054 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:51:45 smithi040 podman[116880]: 2024-06-06 03:51:45.828674453 +0000 UTC m=+4.196192878 container cleanup 8f73f8f00067447387e4ab151be22708f3699a840d83ec470ea5125335c0cb87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-06T03:51:46.055 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jun 06 03:51:45 smithi040 bash[116880]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-a 2024-06-06T03:51:46.187 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.a.service' 2024-06-06T03:51:46.225 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:51:46.226 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-06T03:51:46.226 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-06T03:51:46.226 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.d 2024-06-06T03:51:46.557 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jun 06 03:51:46 smithi040 systemd[1]: Stopping Ceph mds.d for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:51:46.830 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jun 06 03:51:46 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-d[73447]: 2024-06-06T03:51:46.558+0000 7ff7a59f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:51:46.831 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jun 06 03:51:46 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-d[73447]: 2024-06-06T03:51:46.558+0000 7ff7a59f4700 -1 mds.d *** got signal Terminated *** 2024-06-06T03:51:50.331 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jun 06 03:51:50 smithi040 podman[117012]: 2024-06-06 03:51:50.078019317 +0000 UTC m=+3.624805089 container died e566d1e35dc849ec8e64438d4367059f13250b7f5a5910083a0eaa9fa0c17776 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-d, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-06T03:51:50.700 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jun 06 03:51:50 smithi040 podman[117012]: 2024-06-06 03:51:50.491107474 +0000 UTC m=+4.037893235 container cleanup e566d1e35dc849ec8e64438d4367059f13250b7f5a5910083a0eaa9fa0c17776 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5) 2024-06-06T03:51:50.700 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jun 06 03:51:50 smithi040 bash[117012]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-d 2024-06-06T03:51:50.851 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.d.service' 2024-06-06T03:51:50.886 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:51:50.886 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-06T03:51:50.886 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-06T03:51:50.886 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.g 2024-06-06T03:51:51.245 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jun 06 03:51:50 smithi040 systemd[1]: Stopping Ceph mds.g for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:51:51.580 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jun 06 03:51:51 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-g[74297]: 2024-06-06T03:51:51.246+0000 7f88fe92f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:51:51.581 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jun 06 03:51:51 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-g[74297]: 2024-06-06T03:51:51.246+0000 7f88fe92f700 -1 mds.g *** got signal Terminated *** 2024-06-06T03:51:53.581 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jun 06 03:51:53 smithi040 podman[117134]: 2024-06-06 03:51:53.329464027 +0000 UTC m=+2.205307319 container died b0faccf813fd987d70c00733f365932bcc7a35c2c5556f08c32de818e64ee2b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-g, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-06T03:51:53.930 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jun 06 03:51:53 smithi040 podman[117134]: 2024-06-06 03:51:53.779235088 +0000 UTC m=+2.655078406 container cleanup b0faccf813fd987d70c00733f365932bcc7a35c2c5556f08c32de818e64ee2b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-g, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-06T03:51:53.930 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jun 06 03:51:53 smithi040 bash[117134]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-g 2024-06-06T03:51:54.162 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.g.service' 2024-06-06T03:51:54.184 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jun 06 03:51:53 smithi040 podman[117147]: 2024-06-06 03:51:53.930574083 +0000 UTC m=+0.595079148 container remove b0faccf813fd987d70c00733f365932bcc7a35c2c5556f08c32de818e64ee2b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-g, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-06T03:51:54.184 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jun 06 03:51:54 smithi040 systemd[1]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.g.service: Deactivated successfully. 2024-06-06T03:51:54.185 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jun 06 03:51:54 smithi040 systemd[1]: Stopped Ceph mds.g for 77c25d68-23b4-11ef-bc9b-c7b262605968. 2024-06-06T03:51:54.185 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jun 06 03:51:54 smithi040 systemd[1]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.g.service: Consumed 7.855s CPU time. 2024-06-06T03:51:54.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:51:54.196 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-06T03:51:54.196 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-06T03:51:54.197 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.j 2024-06-06T03:51:54.507 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:51:54 smithi040 systemd[1]: Stopping Ceph mds.j for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:51:54.831 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:51:54 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-j[73057]: 2024-06-06T03:51:54.508+0000 7fd434de5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:51:54.831 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:51:54 smithi040 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-j[73057]: 2024-06-06T03:51:54.508+0000 7fd434de5700 -1 mds.j *** got signal Terminated *** 2024-06-06T03:51:58.831 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:51:58 smithi040 podman[117257]: 2024-06-06 03:51:58.520037035 +0000 UTC m=+4.093712876 container died d0b64280c765686e90c59401ab1cc5271ef9bc0e48bab1a7086f17afa48d8923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-06T03:51:59.167 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:51:58 smithi040 podman[117257]: 2024-06-06 03:51:58.96639401 +0000 UTC m=+4.540069891 container cleanup d0b64280c765686e90c59401ab1cc5271ef9bc0e48bab1a7086f17afa48d8923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-j, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-06T03:51:59.167 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jun 06 03:51:58 smithi040 bash[117257]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-j 2024-06-06T03:51:59.391 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.j.service' 2024-06-06T03:51:59.424 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:51:59.424 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-06T03:51:59.424 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-06T03:51:59.425 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.b 2024-06-06T03:51:59.740 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:51:59 smithi104 systemd[1]: Stopping Ceph mds.b for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:52:00.091 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:51:59 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b[66930]: 2024-06-06T03:51:59.740+0000 7f8d3237b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:52:00.091 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:51:59 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b[66930]: 2024-06-06T03:51:59.740+0000 7f8d3237b700 -1 mds.b *** got signal Terminated *** 2024-06-06T03:52:01.674 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:52:01 smithi104 podman[74118]: 2024-06-06 03:52:01.38266815 +0000 UTC m=+1.757403481 container died 0226ae581893891d254fe2536e97a551b0d053a1f131818065eafa570270466a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T03:52:01.965 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:52:01 smithi104 podman[74118]: 2024-06-06 03:52:01.765207659 +0000 UTC m=+2.139942977 container cleanup 0226ae581893891d254fe2536e97a551b0d053a1f131818065eafa570270466a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-06T03:52:01.966 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jun 06 03:52:01 smithi104 bash[74118]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-b 2024-06-06T03:52:02.188 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.b.service' 2024-06-06T03:52:02.222 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:52:02.223 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-06T03:52:02.223 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-06T03:52:02.223 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.e 2024-06-06T03:52:02.536 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:52:02 smithi104 systemd[1]: Stopping Ceph mds.e for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:52:02.841 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:52:02 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-e[66322]: 2024-06-06T03:52:02.535+0000 7fc1f5dec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:52:02.841 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:52:02 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-e[66322]: 2024-06-06T03:52:02.535+0000 7fc1f5dec700 -1 mds.e *** got signal Terminated *** 2024-06-06T03:52:05.677 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:52:05 smithi104 podman[74250]: 2024-06-06 03:52:05.383017097 +0000 UTC m=+2.947899300 container stop 4643e9f7b44930925dc7e40db3b23699e12a8571eaa19f7a471fd497795cada2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-06T03:52:05.677 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:52:05 smithi104 podman[74250]: 2024-06-06 03:52:05.409681693 +0000 UTC m=+2.974563895 container died 4643e9f7b44930925dc7e40db3b23699e12a8571eaa19f7a471fd497795cada2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-06T03:52:05.927 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:52:05 smithi104 podman[74250]: 2024-06-06 03:52:05.752005316 +0000 UTC m=+3.316887518 container cleanup 4643e9f7b44930925dc7e40db3b23699e12a8571eaa19f7a471fd497795cada2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-e, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-06T03:52:05.928 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:52:05 smithi104 bash[74250]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-e 2024-06-06T03:52:06.178 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jun 06 03:52:05 smithi104 podman[74263]: 2024-06-06 03:52:05.928183208 +0000 UTC m=+0.538031948 container remove 4643e9f7b44930925dc7e40db3b23699e12a8571eaa19f7a471fd497795cada2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-06-06T03:52:06.180 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.e.service' 2024-06-06T03:52:06.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:52:06.212 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-06T03:52:06.213 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-06T03:52:06.213 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.h 2024-06-06T03:52:06.530 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jun 06 03:52:06 smithi104 systemd[1]: Stopping Ceph mds.h for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:52:06.841 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jun 06 03:52:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-h[66627]: 2024-06-06T03:52:06.530+0000 7fe0358b1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:52:06.841 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jun 06 03:52:06 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-h[66627]: 2024-06-06T03:52:06.530+0000 7fe0358b1700 -1 mds.h *** got signal Terminated *** 2024-06-06T03:52:09.672 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jun 06 03:52:09 smithi104 podman[74368]: 2024-06-06 03:52:09.38567715 +0000 UTC m=+2.955544113 container died 969a1f2cb41aca43d5658f057407eeddfd27670681563e83fe3c54da777b2946 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-06T03:52:10.006 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jun 06 03:52:09 smithi104 podman[74368]: 2024-06-06 03:52:09.755316719 +0000 UTC m=+3.325183677 container cleanup 969a1f2cb41aca43d5658f057407eeddfd27670681563e83fe3c54da777b2946 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-h, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-06T03:52:10.006 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jun 06 03:52:09 smithi104 bash[74368]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-h 2024-06-06T03:52:10.143 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.h.service' 2024-06-06T03:52:10.174 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:52:10.174 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-06T03:52:10.174 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-06T03:52:10.175 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.k 2024-06-06T03:52:10.591 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jun 06 03:52:10 smithi104 systemd[1]: Stopping Ceph mds.k for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:52:10.591 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jun 06 03:52:10 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-k[66007]: 2024-06-06T03:52:10.491+0000 7f761131d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:52:10.591 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jun 06 03:52:10 smithi104 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-k[66007]: 2024-06-06T03:52:10.491+0000 7f761131d700 -1 mds.k *** got signal Terminated *** 2024-06-06T03:52:13.716 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jun 06 03:52:13 smithi104 podman[74490]: 2024-06-06 03:52:13.386420156 +0000 UTC m=+2.990871505 container died 940392d80d7236a81ad3976aae2cfbbcd2080294743b4d05c4dfdb1e69f3f02e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-k, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T03:52:13.717 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jun 06 03:52:13 smithi104 podman[74490]: 2024-06-06 03:52:13.716812437 +0000 UTC m=+3.321263781 container cleanup 940392d80d7236a81ad3976aae2cfbbcd2080294743b4d05c4dfdb1e69f3f02e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-06T03:52:14.091 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jun 06 03:52:13 smithi104 bash[74490]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-k 2024-06-06T03:52:14.091 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jun 06 03:52:13 smithi104 podman[74504]: 2024-06-06 03:52:13.901299421 +0000 UTC m=+0.508549313 container remove 940392d80d7236a81ad3976aae2cfbbcd2080294743b4d05c4dfdb1e69f3f02e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-06T03:52:14.146 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.k.service' 2024-06-06T03:52:14.176 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:52:14.177 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-06T03:52:14.177 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-06T03:52:14.177 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.c 2024-06-06T03:52:14.495 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:52:14 smithi131 systemd[1]: Stopping Ceph mds.c for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:52:14.759 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:52:14 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-c[67822]: 2024-06-06T03:52:14.495+0000 7f235f6a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:52:14.759 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:52:14 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-c[67822]: 2024-06-06T03:52:14.495+0000 7f235f6a5700 -1 mds.c *** got signal Terminated *** 2024-06-06T03:52:17.665 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:52:17 smithi131 podman[76064]: 2024-06-06 03:52:17.341671087 +0000 UTC m=+2.970276132 container died 0e5f414d00ff75f688ee3aff5e501b9747e9ba28c860f9a0979d5a5eec9c8a74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T03:52:17.665 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:52:17 smithi131 podman[76064]: 2024-06-06 03:52:17.665544499 +0000 UTC m=+3.294149535 container cleanup 0e5f414d00ff75f688ee3aff5e501b9747e9ba28c860f9a0979d5a5eec9c8a74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-c, ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-06T03:52:18.032 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:52:17 smithi131 bash[76064]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-c 2024-06-06T03:52:18.033 INFO:journalctl@ceph.mds.c.smithi131.stdout:Jun 06 03:52:17 smithi131 podman[76090]: 2024-06-06 03:52:17.850117067 +0000 UTC m=+0.499678403 container remove 0e5f414d00ff75f688ee3aff5e501b9747e9ba28c860f9a0979d5a5eec9c8a74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-c, ceph=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-06T03:52:18.098 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.c.service' 2024-06-06T03:52:18.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:52:18.134 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-06T03:52:18.134 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-06T03:52:18.134 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.f 2024-06-06T03:52:18.452 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:52:18 smithi131 systemd[1]: Stopping Ceph mds.f for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:52:18.782 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:52:18 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-f[67516]: 2024-06-06T03:52:18.452+0000 7f9100b54700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:52:18.783 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:52:18 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-f[67516]: 2024-06-06T03:52:18.452+0000 7f9100b54700 -1 mds.f *** got signal Terminated *** 2024-06-06T03:52:21.612 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:52:21 smithi131 podman[76196]: 2024-06-06 03:52:21.340929622 +0000 UTC m=+2.987523662 container died b3bcd351fd22303fe9458b6c2deb6a7b070e7302888b4de81bfe96b6e779ff18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS) 2024-06-06T03:52:21.962 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:52:21 smithi131 podman[76196]: 2024-06-06 03:52:21.694428309 +0000 UTC m=+3.341022347 container cleanup b3bcd351fd22303fe9458b6c2deb6a7b070e7302888b4de81bfe96b6e779ff18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-06T03:52:21.962 INFO:journalctl@ceph.mds.f.smithi131.stdout:Jun 06 03:52:21 smithi131 bash[76196]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-f 2024-06-06T03:52:22.131 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.f.service' 2024-06-06T03:52:22.164 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:52:22.164 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-06T03:52:22.164 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-06T03:52:22.165 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.i 2024-06-06T03:52:22.481 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:52:22 smithi131 systemd[1]: Stopping Ceph mds.i for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:52:22.782 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:52:22 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-i[68123]: 2024-06-06T03:52:22.480+0000 7f0ef39bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:52:22.782 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:52:22 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-i[68123]: 2024-06-06T03:52:22.480+0000 7f0ef39bc700 -1 mds.i *** got signal Terminated *** 2024-06-06T03:52:25.532 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:52:25 smithi131 podman[76318]: 2024-06-06 03:52:25.348250184 +0000 UTC m=+2.962830257 container died 4d9bd292e896d9b3b0cc27a88add1de880213fa2d077824c46cef89365209259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-i, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-06T03:52:26.033 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:52:25 smithi131 podman[76318]: 2024-06-06 03:52:25.898617168 +0000 UTC m=+3.513197235 container cleanup 4d9bd292e896d9b3b0cc27a88add1de880213fa2d077824c46cef89365209259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-06T03:52:26.033 INFO:journalctl@ceph.mds.i.smithi131.stdout:Jun 06 03:52:25 smithi131 bash[76318]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-i 2024-06-06T03:52:26.314 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.i.service' 2024-06-06T03:52:26.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:52:26.345 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-06T03:52:26.345 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-06T03:52:26.345 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.l 2024-06-06T03:52:26.782 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:52:26 smithi131 systemd[1]: Stopping Ceph mds.l for 77c25d68-23b4-11ef-bc9b-c7b262605968... 2024-06-06T03:52:26.783 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:52:26 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-l[67205]: 2024-06-06T03:52:26.651+0000 7f93657e8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T03:52:26.783 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:52:26 smithi131 ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-l[67205]: 2024-06-06T03:52:26.651+0000 7f93657e8700 -1 mds.l *** got signal Terminated *** 2024-06-06T03:52:29.653 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:52:29 smithi131 podman[76439]: 2024-06-06 03:52:29.337935773 +0000 UTC m=+2.779220977 container died 220c34dc6a944942aa87d736ae4a04ca12980ed047019063b32b203a56849078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-l, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-06-06T03:52:29.953 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:52:29 smithi131 podman[76439]: 2024-06-06 03:52:29.727625784 +0000 UTC m=+3.168910987 container cleanup 220c34dc6a944942aa87d736ae4a04ca12980ed047019063b32b203a56849078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-l, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T03:52:29.953 INFO:journalctl@ceph.mds.l.smithi131.stdout:Jun 06 03:52:29 smithi131 bash[76439]: ceph-77c25d68-23b4-11ef-bc9b-c7b262605968-mds-l 2024-06-06T03:52:30.076 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77c25d68-23b4-11ef-bc9b-c7b262605968@mds.l.service' 2024-06-06T03:52:30.107 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T03:52:30.107 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-06T03:52:30.108 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T03:52:30.308 INFO:teuthology.orchestra.run.smithi040.stdout:Deleting cluster with fsid: 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:52:32.820 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T03:52:33.007 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:52:35.794 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T03:52:35.982 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:52:38.224 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T03:52:38.258 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T03:52:38.290 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T03:52:38.316 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-06T03:52:38.318 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023/remote/smithi040/crash 2024-06-06T03:52:38.319 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T03:52:38.348 INFO:teuthology.orchestra.run.smithi040.stderr:tar: /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T03:52:38.349 INFO:teuthology.orchestra.run.smithi040.stderr:tar: Error is not recoverable: exiting now 2024-06-06T03:52:38.351 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023/remote/smithi104/crash 2024-06-06T03:52:38.351 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T03:52:38.485 INFO:teuthology.orchestra.run.smithi104.stderr:tar: /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T03:52:38.486 INFO:teuthology.orchestra.run.smithi104.stderr:tar: Error is not recoverable: exiting now 2024-06-06T03:52:38.487 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023/remote/smithi131/crash 2024-06-06T03:52:38.488 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T03:52:38.523 INFO:teuthology.orchestra.run.smithi131.stderr:tar: /var/lib/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T03:52:38.523 INFO:teuthology.orchestra.run.smithi131.stderr:tar: Error is not recoverable: exiting now 2024-06-06T03:52:38.524 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-06T03:52:38.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-06-06T03:52:38.563 INFO:tasks.cephadm:Compressing logs... 2024-06-06T03:52:38.564 DEBUG:teuthology.orchestra.run.smithi040:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T03:52:38.606 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T03:52:38.608 DEBUG:teuthology.orchestra.run.smithi131:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T03:52:38.632 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T03:52:38.632 INFO:teuthology.orchestra.run.smithi131.stderr:: No such file or directory 2024-06-06T03:52:38.632 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-06T03:52:38.633 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-06-06T03:52:38.633 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T03:52:38.638 INFO:teuthology.orchestra.run.smithi040.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-06T03:52:38.638 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T03:52:38.638 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T03:52:38.638 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-06-06T03:52:38.638 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-06-06T03:52:38.638 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-06T03:52:38.639 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-06T03:52:38.639 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-06T03:52:38.639 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.0.log: -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-06T03:52:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-06T03:52:38.641 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-06T03:52:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mon.a.log: 90.3% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T03:52:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-06T03:52:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-06-06T03:52:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.cephadm.log: 84.2% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-06T03:52:38.642 INFO:teuthology.orchestra.run.smithi040.stderr: 94.7% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-06-06T03:52:38.643 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-06T03:52:38.643 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-06-06T03:52:38.643 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-06T03:52:38.644 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-06-06T03:52:38.644 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-06-06T03:52:38.644 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mgr.z.log: 87.1% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-06T03:52:38.644 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-06T03:52:38.644 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T03:52:38.644 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-06-06T03:52:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-06T03:52:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-06-06T03:52:38.646 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.11.log: 92.1% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-06-06T03:52:38.647 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T03:52:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-06-06T03:52:38.660 INFO:teuthology.orchestra.run.smithi104.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-06T03:52:38.660 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T03:52:38.660 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T03:52:38.660 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-06T03:52:38.660 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-06-06T03:52:38.660 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-06T03:52:38.660 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-06T03:52:38.660 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-06T03:52:38.661 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-06-06T03:52:38.661 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-06-06T03:52:38.661 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-06T03:52:38.661 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-06-06T03:52:38.661 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T03:52:38.661 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-06T03:52:38.661 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-06-06T03:52:38.661 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T03:52:38.663 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T03:52:38.663 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-06-06T03:52:38.675 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T03:52:38.676 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-06-06T03:52:38.677 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T03:52:38.681 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-06-06T03:52:38.681 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T03:52:38.692 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T03:52:38.693 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T03:52:38.717 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 93.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T03:52:38.794 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 87.1% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-06T03:52:38.809 INFO:teuthology.orchestra.run.smithi104.stderr: 83.8% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-06T03:52:38.964 INFO:teuthology.orchestra.run.smithi104.stderr: 91.1% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T03:52:38.984 INFO:teuthology.orchestra.run.smithi104.stderr: 92.0% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-06T03:52:39.011 INFO:teuthology.orchestra.run.smithi040.stderr: 90.2% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-06T03:52:39.085 INFO:teuthology.orchestra.run.smithi104.stderr: 90.4% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T03:52:39.100 INFO:teuthology.orchestra.run.smithi131.stderr: 91.9% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-06T03:52:39.933 INFO:teuthology.orchestra.run.smithi040.stderr: 90.3% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-06T03:52:39.977 INFO:teuthology.orchestra.run.smithi104.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T03:52:40.375 INFO:teuthology.orchestra.run.smithi131.stderr: 94.9% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-06-06T03:52:41.348 INFO:teuthology.orchestra.run.smithi040.stderr: 94.9% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-06-06T03:52:41.678 INFO:teuthology.orchestra.run.smithi040.stderr: 94.9% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-06-06T03:52:41.960 INFO:teuthology.orchestra.run.smithi131.stderr: 95.3% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-06-06T03:52:44.279 INFO:teuthology.orchestra.run.smithi104.stderr: 92.1% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-06T03:52:47.775 INFO:teuthology.orchestra.run.smithi040.stderr: 91.2% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-06T03:52:47.852 INFO:teuthology.orchestra.run.smithi040.stderr: 91.9% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-06-06T03:52:50.947 INFO:teuthology.orchestra.run.smithi131.stderr: 95.4% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-06-06T03:52:51.353 INFO:teuthology.orchestra.run.smithi131.stderr: 91.5% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-06-06T03:52:51.773 INFO:teuthology.orchestra.run.smithi040.stderr: 91.5% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-06T03:52:52.148 INFO:teuthology.orchestra.run.smithi040.stderr: 91.2% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-06T03:52:52.150 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-06-06T03:52:52.150 INFO:teuthology.orchestra.run.smithi040.stderr:real 0m13.529s 2024-06-06T03:52:52.150 INFO:teuthology.orchestra.run.smithi040.stderr:user 0m50.781s 2024-06-06T03:52:52.150 INFO:teuthology.orchestra.run.smithi040.stderr:sys 0m1.546s 2024-06-06T03:52:53.362 INFO:teuthology.orchestra.run.smithi131.stderr: 91.3% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-06-06T03:52:54.183 INFO:teuthology.orchestra.run.smithi131.stderr: 91.4% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-06T03:52:54.447 INFO:teuthology.orchestra.run.smithi131.stderr: 91.3% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-06T03:53:13.382 INFO:teuthology.orchestra.run.smithi131.stderr: 95.4% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-06-06T03:53:13.384 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-06-06T03:53:13.384 INFO:teuthology.orchestra.run.smithi131.stderr:real 0m34.761s 2024-06-06T03:53:13.384 INFO:teuthology.orchestra.run.smithi131.stderr:user 1m47.498s 2024-06-06T03:53:13.384 INFO:teuthology.orchestra.run.smithi131.stderr:sys 0m3.253s 2024-06-06T03:53:59.074 INFO:teuthology.orchestra.run.smithi104.stderr: 91.3% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-06T03:54:19.304 INFO:teuthology.orchestra.run.smithi104.stderr: 91.5% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-06-06T03:54:40.741 INFO:teuthology.orchestra.run.smithi104.stderr: 93.5% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-06-06T03:54:45.610 INFO:teuthology.orchestra.run.smithi104.stderr: 91.6% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-06T03:54:45.856 INFO:teuthology.orchestra.run.smithi104.stderr: 91.4% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-06T03:55:36.357 INFO:teuthology.orchestra.run.smithi104.stderr: 96.1% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-06-06T03:56:21.450 INFO:teuthology.orchestra.run.smithi104.stderr: 95.5% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-06-06T03:56:39.960 INFO:teuthology.orchestra.run.smithi104.stderr: 95.6% -- replaced with /var/log/ceph/77c25d68-23b4-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-06-06T03:56:39.961 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-06-06T03:56:39.961 INFO:teuthology.orchestra.run.smithi104.stderr:real 4m1.338s 2024-06-06T03:56:39.962 INFO:teuthology.orchestra.run.smithi104.stderr:user 4m33.243s 2024-06-06T03:56:39.962 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m10.894s 2024-06-06T03:56:39.962 INFO:tasks.cephadm:Archiving logs... 2024-06-06T03:56:39.963 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023/remote/smithi040/log 2024-06-06T03:56:39.965 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T03:56:47.447 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023/remote/smithi104/log 2024-06-06T03:56:47.448 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T03:57:25.264 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023/remote/smithi131/log 2024-06-06T03:57:25.265 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T03:57:37.791 INFO:tasks.cephadm:Removing cluster... 2024-06-06T03:57:37.792 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 --force 2024-06-06T03:57:37.996 INFO:teuthology.orchestra.run.smithi040.stdout:Deleting cluster with fsid: 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:57:38.335 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 --force 2024-06-06T03:57:38.631 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:57:39.142 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77c25d68-23b4-11ef-bc9b-c7b262605968 --force 2024-06-06T03:57:39.331 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: 77c25d68-23b4-11ef-bc9b-c7b262605968 2024-06-06T03:57:39.740 INFO:tasks.cephadm:Removing cephadm ... 2024-06-06T03:57:39.741 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T03:57:39.760 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T03:57:39.776 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T03:57:39.791 INFO:tasks.cephadm:Teardown complete 2024-06-06T03:57:39.791 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-06T03:57:39.807 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-06T03:57:39.807 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T03:57:39.810 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T03:57:39.819 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T03:57:39.928 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-06T03:57:39.939 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-06T03:57:39.939 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T03:57:39.942 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T03:57:39.944 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T03:57:39.955 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-06-06T03:57:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-06-06T03:57:39.958 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-06-06T03:57:39.962 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T03:57:39.962 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-06-06T03:57:39.962 INFO:teuthology.orchestra.run.smithi131.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 +682us[ +705us] +/- 69ms 2024-06-06T03:57:39.962 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 8 -640us[ -616us] +/- 80ms 2024-06-06T03:57:39.962 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -152us[ -129us] +/- 70ms 2024-06-06T03:57:39.963 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:57:39.964 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T03:57:39.964 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-06-06T03:57:39.964 INFO:teuthology.orchestra.run.smithi040.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +704us[ +734us] +/- 69ms 2024-06-06T03:57:39.964 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 8 -635us[ -605us] +/- 80ms 2024-06-06T03:57:39.965 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -129us[ -100us] +/- 70ms 2024-06-06T03:57:39.965 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:57:39.973 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T03:57:39.974 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-06-06T03:57:39.975 INFO:teuthology.orchestra.run.smithi104.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 +695us[ +691us] +/- 69ms 2024-06-06T03:57:39.975 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -634us[ -634us] +/- 80ms 2024-06-06T03:57:39.975 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 -147us[ -147us] +/- 71ms 2024-06-06T03:57:39.975 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:57:39.976 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-06T03:57:39.986 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-06T03:57:39.986 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-06T03:57:39.996 DEBUG:teuthology.orchestra.run.smithi040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T03:57:40.092 DEBUG:teuthology.orchestra.run.smithi104:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T03:57:40.191 DEBUG:teuthology.orchestra.run.smithi131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T03:57:40.257 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T03:57:40.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:57:40.289 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T03:57:40.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:57:40.315 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T03:57:40.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:57:40.343 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-06T03:57:40.354 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-06T03:57:40.365 INFO:teuthology.task.internal:Duration was 2292.292073 seconds 2024-06-06T03:57:40.365 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-06T03:57:40.376 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-06T03:57:40.376 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T03:57:40.378 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T03:57:40.381 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T03:57:40.422 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T03:57:40.423 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T03:57:40.436 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T03:57:40.877 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-06T03:57:40.877 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2024-06-06T03:57:40.878 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T03:57:40.903 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2024-06-06T03:57:40.903 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T03:57:40.955 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2024-06-06T03:57:40.955 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T03:57:40.979 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-06T03:57:40.979 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T03:57:40.982 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T03:57:40.984 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T03:57:41.044 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-06T03:57:41.045 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T03:57:41.052 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T03:57:41.054 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T03:57:41.785 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-06T03:57:41.872 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-06T03:57:41.872 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T03:57:41.898 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T03:57:41.927 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T03:57:41.953 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-06T03:57:41.968 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T03:57:41.971 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T03:57:41.973 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T03:57:41.992 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2024-06-06T03:57:42.014 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2024-06-06T03:57:42.016 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2024-06-06T03:57:42.039 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T03:57:42.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:57:42.063 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T03:57:42.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:57:42.088 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T03:57:42.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:57:42.102 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-06T03:57:42.114 INFO:teuthology.task.internal:Transferring archived files... 2024-06-06T03:57:42.114 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023/remote/smithi040 2024-06-06T03:57:42.115 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T03:57:42.168 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023/remote/smithi104 2024-06-06T03:57:42.168 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T03:57:42.202 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743023/remote/smithi131 2024-06-06T03:57:42.203 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T03:57:42.237 INFO:teuthology.task.internal:Removing archive directory... 2024-06-06T03:57:42.237 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T03:57:42.240 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T03:57:42.242 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T03:57:42.286 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-06T03:57:42.300 INFO:teuthology.task.internal:Not uploading archives. 2024-06-06T03:57:42.300 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-06T03:57:42.310 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-06T03:57:42.311 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T03:57:42.313 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T03:57:42.315 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T03:57:42.326 INFO:teuthology.orchestra.run.smithi040.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 03:57 /home/ubuntu/cephtest 2024-06-06T03:57:42.327 INFO:teuthology.orchestra.run.smithi104.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 03:57 /home/ubuntu/cephtest 2024-06-06T03:57:42.341 INFO:teuthology.orchestra.run.smithi131.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 03:57 /home/ubuntu/cephtest 2024-06-06T03:57:42.342 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-06T03:57:42.353 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-06T03:57:42.381 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} duration: 2292.292072534561 failure_reason: 'Command failed (workunit test kernel_untar_build.sh) on smithi040 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d746ca5346f34eef7e826a1eb71e59e02b682ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=efd308eb8d134c02906dff01dfe59e18 status: fail success: false 2024-06-06T03:57:42.381 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-06T03:57:42.476 INFO:teuthology.run:FAIL